Using Context Rules

You can use the surrounding context of a match to help determine how likely it is that your potential match should actually be considered as a match by adjusting its confidence rating.

You can also tell the Detection Rule to return a portion of the surrounding context for manual review.

In the following example, in addition to providing a regular expression to match Social Security Numbers, we also look to see if someone has written the text “SSN” before and after the match, which might be a label indicating it is indeed a social security number. In which case, we change our confidence score to “VERY_LIKELY.” We then provide two possible matches in our payload, the first of which contains the string “SSN”.

curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--header 'Content-Type: application/json' \
--data-raw '{
     "config": {
          "detectionRules": [
               {
                    "detectors": [
                         {
                              "regex": {
                                   "isCaseSensitive": false,
                                   "pattern": "\\d{3}-\\d{2}-\\d{4}"
                              },
                              "contextRules": [
                                   {
                                        "regex": {
                                             "pattern": "SSN",
                                             "isCaseSensitive": false
                                        },
                                        "proximity": {
                                             "windowBefore": 20,
                                             "windowAfter": 20
                                        },
                                        "confidenceAdjustment": {
                                             "fixedConfidence": "VERY_LIKELY"
                                        }
                                   }
                              ],
                              "minNumFindings": 1,
                              "minConfidence": "POSSIBLE",
                              "detectorType": "REGEX",
                              "displayName": "SSN Match Detector"
                         }
                    ],
                    "name": "SSN Match Detection Rule",
                    "logicalOp": "ALL"
               }
          ],
          "contextBytes": 20
     },
     "payload": [
          "My SSN is 555-55-5555",
          "Here it is : 555-55-5555"
     ]
}
'

In the results, you can see the confidence for the first finding in the payload has been set to VERY_LIKELY while the second item is only LIKELY.

{
    "findings": [
        [
            {
                "finding": "555-55-5555",
                "beforeContext": "My SSN is ",
                "detector": {
                    "name": "SSN Match Detector",
                    "uuid": "781f63d6-7675-4bfe-a8ec-af5e184a34ac"
                },
                "confidence": "VERY_LIKELY",
                "location": {
                    "byteRange": {
                        "start": 10,
                        "end": 21
                    },
                    "codepointRange": {
                        "start": 10,
                        "end": 21
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "SSN Match Detection Rule"
                ]
            }
        ],
        [
            {
                "finding": "555-55-5555",
                "beforeContext": "Here it is : ",
                "detector": {
                    "name": "SSN Match Detector",
                    "uuid": "781f63d6-7675-4bfe-a8ec-af5e184a34ac"
                },
                "confidence": "LIKELY",
                "location": {
                    "byteRange": {
                        "start": 13,
                        "end": 24
                    },
                    "codepointRange": {
                        "start": 13,
                        "end": 24
                    }
                },
                "matchedDetectionRuleUUIDs": [],
                "matchedDetectionRules": [
                    "SSN Match Detection Rule"
                ]
            }
        ]
    ]
}