Creating an Inline Detection Rule
In addition to using pre-defined Detection Rules, you may define Detection Rules within the body of your scan method by either supplying:
- the identifier of one of Nightfall's native detectors
- the UUID of an a Detector defined through the UI
- a Regular Expression
- a Word List.
Built In Detectors
Out of the box, Nightfall comes with an extensive library of native detectors.
In the example below two of Nightfall's native Detectors (detectorType = "NIGHTFALL_DETECTOR") are being used:
- US_SOCIAL_SECURITY_NUMBER
- CREDIT_CARD_NUMBER.
When defining a Detection Rule in line, you configure the minimum confidence level (minConfidence) and minimum number of times the match must be found (minNumFindings) for the rule to be triggered. .
In the payload body, you can see that we are submitting a list of three different strings to scan (payload). The first will trigger the U.S. Social Security Detector. The last will trigger the credit card Detector. The middle example will trigger neither.
For more information on the parameters related to redaction, see Using Redaction.
curl --request POST \
--url https://api.nightfall.ai/v3/scan \
--header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--header 'Content-Type: application/json' \
--data '{
"policy": {
"detectionRules": [
{
"detectors": [
{
"detectorType": "NIGHTFALL_DETECTOR",
"nightfallDetector": "US_SOCIAL_SECURITY_NUMBER",
"minNumFindings": 1,
"minConfidence": "LIKELY",
"displayName": "US Social Security Number"
},
{
"detectorType": "NIGHTFALL_DETECTOR",
"nightfallDetector": "CREDIT_CARD_NUMBER",
"minNumFindings": 1,
"minConfidence": "LIKELY",
"displayName": "Credit Card Number",
"redactionConfig": {
"maskConfig": {
"maskingChar": "👀",
"charsToIgnore": ["-"]
}
}
}
],
"name": "My Match Rule",
"logicalOp": "ANY"
}
]
},
"payload": [
"The customer social security number is 458-02-6124",
"No PII in this string",
"My credit card number is 5310-2768-6832-9293"
]
}'
Below is the response payload to the previous request.
{
"findings": [
[
{
"finding": "458-02-6124",
"detector": {
"name": "US Social Security Number",
"uuid": "e30d9a87-f6c7-46b9-a8f4-16547901e069"
},
"confidence": "VERY_LIKELY",
"location": {
"byteRange": {
"start": 39,
"end": 50
},
"codepointRange": {
"start": 39,
"end": 50
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"My Match Rule"
]
}
],
[],
[
{
"finding": "5310-2768-6832-9293",
"redactedFinding": "👀👀👀👀-👀👀👀👀-👀👀👀👀-👀👀👀👀",
"detector": {
"name": "Credit Card Number",
"uuid": "74c1815e-c0c3-4df5-8b1e-6cf98864a454"
},
"confidence": "VERY_LIKELY",
"location": {
"byteRange": {
"start": 25,
"end": 44
},
"codepointRange": {
"start": 25,
"end": 44
}
},
"redactedLocation": {
"byteRange": {
"start": 25,
"end": 44
},
"codepointRange": {
"start": 25,
"end": 44
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"My Match Rule"
]
}
]
],
"redactedPayload": [
"",
"",
"My credit card number is 👀👀👀👀-👀👀👀👀-👀👀👀👀-👀👀👀👀"
]
}
Regular Expression Example
The following example shows a Detection Rule composed of two Detectors defined using regular expressions – one for the format of an International Standard Recording Code (ISRC) and one for the format of an International Standard Musical Work Code (ISWC) – matching either of which will trigger the Detection Rule (by using the logicalOp “Any”).
We will provide a payload of two strings, one of which will match the ISRC and one of which will match the ISWC.
curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--header 'Content-Type: application/json' \
--data-raw '{
"config": {
"detectionRules": [
{
"detectors": [
{
"regex": {
"isCaseSensitive": false,
"pattern": "[A-Z]{2}-?\\w{3}-?\\d{2}-?\\d{5}"
},
"minNumFindings": 1,
"minConfidence": "POSSIBLE",
"detectorType": "REGEX",
"displayName": "ISRC Code Detector"
},
{
"regex": {
"isCaseSensitive": false,
"pattern": "T-[0-9]{3}\\.[0-9]{3}\\.[0-9]{3}-[0-9]"
},
"minNumFindings": 1,
"minConfidence": "POSSIBLE",
"detectorType": "REGEX",
"displayName": "ISWC Code Detector"
}
],
"name": "ISRC and ISWC Code Detection Rule",
"logicalOp": "ANY"
}
]
},
"payload": [
"Non Matching Payload",
"US-S1Z-99-00001 is an example ISRC Code: ",
"The ISWC for Symphony No. 9 is T-905.029.737-5"
]
}
'
The returned response demonstrates how findings are returned, with a finding per payload entry and the Detection Rule and Detector that matched the payload, if any.
The byte range that triggered the match is also provided. In the case of the 2nd item in the payload, since the match occurred at the beginning of the string, it has a location where the byteRange start is 0. In the case of the 3rd payload entry the location offset is 31.
{
"findings": [
[],
[
{
"finding": "US-S1Z-99-00001",
"detector": {
"name": "ISRC Code Detector",
"uuid": "d8be87c9-4b44-41fd-b78c-8d638fe56069"
},
"confidence": "LIKELY",
"location": {
"byteRange": {
"start": 0,
"end": 15
},
"codepointRange": {
"start": 0,
"end": 15
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"ISRC and ISWC Code Detection Rule"
]
}
],
[
{
"finding": "T-905.029.737-5",
"detector": {
"name": "ISWC Code Detector",
"uuid": "faf4c830-f2ac-4934-bf9c-ff20f5a6f420"
},
"confidence": "LIKELY",
"location": {
"byteRange": {
"start": 31,
"end": 46
},
"codepointRange": {
"start": 31,
"end": 46
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"ISRC and ISWC Code Detection Rule"
]
}
]
]
}
Word List Example
The following example shows how a word list may be used instead of a regular expression.
curl --location --request POST 'https://api.nightfall.ai/v3/scan' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'x-api-key: NF-rEpLaCeM3w1ThYoUrNiGhTfAlLKeY123' \
--data-raw '{
"config": {
"detectionRules": [
{
"detectors": [
{
"wordList": {
"values": [
"cat",
"dog",
"rat"
],
"isCaseSensitive": false
},
"minNumFindings": 1,
"minConfidence": "POSSIBLE",
"displayName": "animals",
"detectorType": "WORD_LIST"
}
],
"name": "WordListExamples",
"logicalOp": "ANY"
}
]
},
"payload": [
"THE CAT SAT ON THE MAT",
"The dog and the rat are on the west bank of the river",
"No one here but use chickens"
]
}'
Below is the resulting payload with the findings detected in our different payload strings.
Note that since the isCaseSensitive flag is set to "false" for the detector, so the first string in our payload matches a word from our word list.
Also note that the confidence level for a word list match defaults to "LIKELY" so you should not set a minConfidence level higher than that if you want matches to result.
{
"findings": [
[
{
"finding": "cat",
"detector": {
"name": "animals",
"uuid": "c033e224-034a-417f-9c0d-0c8d13f462bb"
},
"confidence": "LIKELY",
"location": {
"byteRange": {
"start": 4,
"end": 7
},
"codepointRange": {
"start": 4,
"end": 7
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"WordListExamples"
]
}
],
[
{
"finding": "dog",
"detector": {
"name": "animals",
"uuid": "c033e224-034a-417f-9c0d-0c8d13f462bb"
},
"confidence": "LIKELY",
"location": {
"byteRange": {
"start": 4,
"end": 7
},
"codepointRange": {
"start": 4,
"end": 7
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"WordListExamples"
]
},
{
"finding": "rat",
"detector": {
"name": "animals",
"uuid": "c033e224-034a-417f-9c0d-0c8d13f462bb"
},
"confidence": "LIKELY",
"location": {
"byteRange": {
"start": 16,
"end": 19
},
"codepointRange": {
"start": 16,
"end": 19
}
},
"matchedDetectionRuleUUIDs": [],
"matchedDetectionRules": [
"WordListExamples"
]
}
],
[]
],
"redactedPayload": [
"",
"",
""
]
}
Updated almost 2 years ago