Using Context Rules
You can use the surrounding context of a match to help determine how likely it is that your potential match should actually be considered as a match by adjusting its confidence rating.
You can also tell the Detection Rule to return a portion of the surrounding context for manual review.
In the following example, in addition to providing a regular expression to match Social Security Numbers, we also look to see if someone has written the text “SSN” before and after the match, which might be a label indicating it is indeed a social security number. In which case, we change our confidence score to “VERY_LIKELY.” We then provide two possible matches in our payload, the first of which contains the string “SSN”.
curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--header 'Content-Type: application/json' \
--data-raw '{
"policy": {
"detectionRules": [
{
"detectors": [
{
"regex": {
"isCaseSensitive": false,
"pattern": "\\d{3}-\\d{2}-\\d{4}"
},
"contextRules": [
{
"regex": {
"pattern": "SSN",
"isCaseSensitive": false
},
"proximity": {
"windowBefore": 20,
"windowAfter": 20
},
"confidenceAdjustment": {
"fixedConfidence": "VERY_LIKELY"
}
}
],
"minNumFindings": 1,
"minConfidence": "POSSIBLE",
"detectorType": "REGEX",
"displayName": "SSN Match Detector"
}
],
"name": "SSN Match Detection Rule",
"logicalOp": "ALL"
}
],
"contextBytes": 20
},
"payload": [
"My SSN is 555-55-5555",
"Here it is : 555-55-5555"
]
}
'
In the results, you can see the confidence for the first finding in the payload has been set to VERY_LIKELY while the second item is only LIKELY.
{
"findings":[
[
{
"finding":"555-55-5555",
"beforeContext":"My SSN is ",
"detector":{
"name":"SSN Match Detector",
"uuid":"6131f41c-dbdd-47a9-8c6f-1819c9baf388"
},
"confidence":"VERY_LIKELY",
"location":{
"byteRange":{
"start":10,
"end":21
},
"codepointRange":{
"start":10,
"end":21
},
"rowRange":null,
"columnRange":null,
"commitHash":""
},
"matchedDetectionRuleUUIDs":[
],
"matchedDetectionRules":[
"SSN Match Detection Rule"
]
}
],
[
{
"finding":"555-55-5555",
"beforeContext":"Here it is : ",
"detector":{
"name":"SSN Match Detector",
"uuid":"6131f41c-dbdd-47a9-8c6f-1819c9baf388"
},
"confidence":"LIKELY",
"location":{
"byteRange":{
"start":13,
"end":24
},
"codepointRange":{
"start":13,
"end":24
},
"rowRange":null,
"columnRange":null,
"commitHash":""
},
"matchedDetectionRuleUUIDs":[
],
"matchedDetectionRules":[
"SSN Match Detection Rule"
]
}
]
],
"redactedPayload":[
"",
""
]
}
Updated over 1 year ago