Creating a Webhook Server
Learn how to set up a server to handle results of file scans and alerts sent based on policy alert configurations.
After 18th August 2024, this page would permanently be moved to a new location. You can access this page from a new URL which is present here. If you have saved or bookmarked the current URL, kindly update it with the new URL, since there will be no 301 redirect from the current URL to the new URL.
Webhook Challenges
Nightfall will send a POST request with a JSON payload with a single field challenge
containing randomly-generated bytes when it sends a message to a user-provided webhook address. This is to ensure that the caller owns the server.
{"challenge": "z78woE1uDFu7tPrPvEBV"}
In order to authenticate your webhook server to Nightfall, you must reply with (1) a 200 HTTP Status Code, and (2) a plaintext request body containing only the value of the challenge
key.
If Nightfall receives the expected value back, then the file scan operation will proceed; otherwise it will be aborted.
When a server responds successfully to a challenge request, the validity of that URL will be cached for up to 24 hours, after which it will need to be validated again.
If the webhook cannot be reached, you will receive an error with the code "40012" and the description "Webhook URL validation failed" when you initiate the scan.
If the webhook challenge fails, you will receive an error with the code "42201" and the description "Webhook returned incorrect challenge response" when you initiate the scan.
Webhook Signature Verification
When a customer signs up for the developer platform, Nightfall automatically generates a unique siging secret for them.
This secret is used to sign requests to the customer's configured webhook URL.
Signing Secret Security
The signing secret should never be stored in plaintext, as a leak compromises the authenticity of webhook requests.
If you has any concerns that their signing secret may have leaked, you can request rotation at any time by reaching out to Nightfall Customer Success.
For security purposes, the webhook includes a signature header containing an HMAC-SHA256 digital signature that customers may use to authenticate the client.
In order to authenticate requests to the webhook URL, customers may use the following algorithm:
- Check for the presence of the headers
X-Nightfall-Signature
andX-Nightfall-Timestamp
. If these headers are not both present, discard the request. - Read the entire request body into a string
body
. - Verify that the value in the
X-Nightfall-Timestamp
header (the POSIX time in seconds) occurred recently. This is to protect against replay attacks, so a threshold on the order of magnitude of minutes should be reasonable. If a request occurred too far in the past, it should be discarded. - Concatenate the timestamp and body with a colon delimiter, i.e.
timestamp:body
. - Compute the HMAC SHA-256 hash of the payload from the previous step, using your unique signing secret as the key. Encode this computed value in hex.
- Compare the value of the
X-Nightfall-Signature
header to the value computed in the previous step. If the values match, authentication is successful, and processing should proceed. Otherwise, the request must be discarded.
The snippet below shows how you might implement this authentication validation in Python:
from datetime import datetime, timedelta
import hmac
import hashlib
from flask import request
SIGNING_SECRET = "super-secret"
given_signature = request.headers.get('X-Nightfall-Signature')
req_timestamp = request.headers.get('X-Nightfall-Timestamp')
now = datetime.now()
if now-timedelta(minutes=5) <= datetime.fromtimestamp(int(req_timestamp)) <= now:
raise Exception("could not validate timestamp is within the last few minutes")
computed_signature = hmac.new(
SIGNING_SECRET.encode(),
msg=F"{req_timestamp}:{request.get_data(as_text=True)}".encode(),
digestmod=hashlib.sha256
).hexdigest().lower()
if computed_signature != given_signature:
raise Exception("could not validate signature of inbound request!")
Example Webhook Server
An example implementation of a simple webhook server is below.
import hmac
import hashlib
from os import getenv, path, mkdir
from flask import Flask, request
import requests
app = Flask(__name__)
output_dir = "findings"
SIGNING_SECRET = getenv("NF_SIGNING_SECRET")
@app.route("/", methods=['POST'])
def hello():
content = request.get_json(silent=True)
challenge = content.get("challenge")
if challenge:
return challenge
else:
verify_signature()
print(F"Received request metadata: {content['requestMetadata']}")
print(F"Received errors: {content['errors']}")
if not content["findingsPresent"]:
print(F"No findings for {content['uploadID']}")
return "", 200
print(F"S3 findings valid until {content['validUntil']}")
response = requests.get(content["findingsURL"])
save_findings(content["uploadID"], response.text)
return "", 200
def verify_signature():
if SIGNING_SECRET is None:
return
given_signature = request.headers.get('X-Nightfall-Signature')
nonce = request.headers.get('X-Nightfall-Timestamp')
computed_signature = hmac.new(
SIGNING_SECRET.encode(),
msg=F"{nonce}:{request.get_data(as_text=True)}".encode(),
digestmod=hashlib.sha256
).hexdigest().lower()
if computed_signature != given_signature:
raise Exception("could not validate signature of inbound request!")
def save_findings(scan_id, finding_json):
if not path.isdir(output_dir):
mkdir(output_dir)
output_path = path.join(output_dir, f"{scan_id}.json")
with open(output_path, "w+") as out_file:
out_file.write(finding_json)
print(F"Findings for {scan_id} written to {output_path}")
if __name__ == "__main__":
app.run(port=8075)
You can test your webhook with a tool such as ngrok which allows you expose a web server running on your local machine to the internet.
In the above example, the webhook server is running on port 8075. To route ngrok requests to this server, once you run the python script (having installed the necessary dependencies such getenv and Flask), you would run ngrok as follow:
./ngrok http 8075
See the section on Alerting for details about the json payloads for the different messages sent to webhook servers.
Updated 2 months ago