DETECT hunt Build detections

ProductionShip

Section 16

Detection as code

Framework v0.136.0, updated November 17, 2026.

Remember

  • If the live rule and the repository disagree, the repository no longer tells you what runs. Alarm on the difference.
  • Write the logic once in Sigma and let the build produce the query for each target. A hand-edited query can never be regenerated.
  • Commit the converted query for every target. The diff is what catches a converter upgrade changing your production logic.
  • Prove each detection on a real engine with a fixture it must match and one it must ignore. A query that converts cleanly can still match nothing.
  • Give the build two credentials: read-only for queries, and a deploy credential no pull request build can read.
  • Your git host, your CI system, and your SIEM will all be replaced before your detections are. The rule file and the record are the parts you keep.

Detection as code makes the repository the program's source of truth and the SIEM a copy of it. Whether a program has it comes down to one question: can you read the repository at a commit and say what ran in production that day, thresholds included? Storing YAML in version control gets you part of the way, because a file can still be copied into a console by hand and edited there. The rules below close the rest of the distance. The last column names what proves each rule, and it says where the kit's check is still planned.

Rule What goes wrong without it What proves it
Keep one directory per detection, named for its id, carrying the rule, the record, the score, the variables, the fixtures, and the approved queries. Logic, intent, and tests are reviewed in three places, so a reviewer approves a query change and misses the record that no longer describes it. Lint fails when the directory is missing a file the detection's stage requires.
Write the logic once in Sigma, and generate the query for every target in the registry from it. Never edit a generated query. The same detection becomes a different detection on each platform, and a SIEM migration means rewriting every one of them. Convert regenerates each target from rule.yml, so a hand edit shows up as a diff on the next build.
Commit the converted query for each target beside the rule, and fail the build when a conversion stops matching its approved copy. A converter upgrade changes what runs in production, and the new query reaches the SIEM before anybody has read it. The golden diff compares every conversion with its committed copy on each pull request.
Pin the converter, every query backend, and the ATT&CK release the technique ids are checked against. A dependency that floats rewrites a production query with no commit behind it, and a detection that stopped matching is the first sign. Each backend is capped below its next major version, and lint rejects a technique id outside the pinned map.
Declare every threshold, window, allowlist, and scope on the record with a default, bounds, and an evidence line. Tuning edits the query, so each change needs a full re-review and the reason for the old number is gone. Lint rejects a variable with no default or no evidence, and a threshold or window with no bounds. Substituting each value into the queries is planned.
Commit one event the detection must match and one it must ignore, and replay both on a real engine. A query that converts cleanly can still match nothing, and the proof of it arrives as an incident the detection should have caught. The replay job loads the fixtures into each engine and compares the hits with tests/expected.yml.
Scrub credentials, tokens, internal host names, and personal data out of a fixture before it is committed. A captured event carries a live secret into version control, and git history keeps it after the fixture is cleaned. A secret scan on every pull request, with a line in the review checklist as the backstop. The kit adds no scan of its own, so turn on the repository host's secret scanning.
Deploy from the build, make the production console read-only, and alarm on a live rule that no longer matches the repository. An edit made in the console during a shift leaves production running a query no test has ever covered. A daily check that compares each running query with its approved copy and fails when they differ. The kit's tools/running.py does it for Splunk, along with the schedule, the alert condition, and whether alerting is on. On another SIEM, compare them by hand at each precision check.
Route every change that reaches production through a pull request approved by somebody other than its author. Under deadline pressure the checks get skipped, and a week later nobody can say which detections skipped them. Branch protection and required status checks on the default branch, with an owner named for the detections directory.
Deploy a new detection with alerting off, count seven days of alerts, and switch alerting on only at or under the threshold on the record. A detection reaches the analyst queue before anybody has counted what it costs, and the first measurement is the complaint from the SOC. The deploy job packages a detection at deploy with alerting off; its vetting and install steps are placeholders a team fills in for its own stack. The pull request that moves it to tune runs a check that counts its first seven days live against the threshold, and merging that pull request switches alerting on.
Retire inside the repository: set the rule status to deprecated, switch alerting off, and keep the record and the retirement note. A deleted detection takes its coverage claim, its blind spots, and the reason it retired away with it. Lint checks the Sigma status against the stage, and the record keeps its retirement record section.

Most checks in the third column are the reference pipeline's jobs. An approval and an owner on the detections directory are repository settings, because a build can't enforce a rule about itself. The secret scan and the 30-day precision check are jobs I haven't written yet; today the review checklist covers the first and a person runs the second.

On this page

16.1A common pipeline

The rule path

RepositoryOne directory per detection: the Sigma rule, the record, the score, the variables, the fixtures, and the approved queries.
->
BuildLint, convert for every target, diff each conversion against its approved copy, replay the two fixtures.
->
DeployOne job, one credential, alerting off for seven days, then on under the threshold on the record.

The event path

1CollectEach log source writes its events where it can, in its own schema and on its own delay.
2Land in object storageEvents arrive as compressed JSON objects, partitioned by date, kept long enough to replay a detection across them.
3Announce the new objectsOne notification per object tells the next step that events arrived, so nothing has to poll the store.
4Normalize the fieldsVendor field names are mapped into one set, so a rule written once resolves against every source.
5Run the detectionThe query runs on each object as it arrives or on a schedule, and emits an alert when it matches.
6Raise the alertThe alert carries the detection id, a link to the record, the severity, and the events that matched.
7Triage in the SOCAn analyst works the alert, records a disposition and the minutes it took, and closes it.
8Tune or retirePrecision over 30 days and minutes per true positive go back to the detection, and a change to either becomes a pull request.
The way back. What an analyst records about each alert returns as a pull request: a tuning change to one declared variable, or a retirement proposal with the trigger that fired.
The event path runs left to right and wraps. The rule path above it writes into step 5 and nowhere else, so what runs there is what the repository converted and the build approved.

The parts differ by platform, and the order rarely does. Events land in object storage, a notification tells a runtime they arrived, the runtime runs the queries the build deployed, and the alerts reach a queue an analyst works. What the analyst records about each alert is what sends the next tuning change back to the repository. The steps below name the AWS parts; the Azure and GCP equivalents sit in the same order.

Step What happens Example parts on AWS What the repository controls
1. Collect Each log source writes its events where it can, in its own schema and on its own delay. CloudTrail, GuardDuty, VPC flow logs, EKS audit logs, and whatever the identity provider and the EDR export A row per source in the inventory, with the normal event rate DET-0000 alerts against.
2. Land in object storage Events arrive as compressed JSON objects, partitioned by date, kept long enough to replay a detection across them. An S3 bucket per account or per source, with versioning on and retention set by its bucket policy Nothing yet. Research reads field names and the 7-day baseline out of this store.
3. Announce the new objects One notification per object tells the next step that events arrived, so nothing has to poll the store. S3 event notifications into SQS, SNS, or EventBridge, with a dead-letter queue behind them Nothing. The age of the oldest message deserves the same silence alert a log source gets.
4. Normalize the fields Vendor field names are mapped into one set, so a rule written once resolves against every source. OCSF or ECS names, applied by the ingest function, an Iceberg table, or the SIEM's own parsers One field map per target, written from real events, so a conversion resolves the names the store has.
5. Run the detection The query runs on each object as it arrives or on a schedule, and emits an alert when it matches. A Lambda or Panther rule per event, a scheduled search in Splunk, Sentinel, or Elastic, or scheduled SQL in Athena or Snowflake The rule itself. CI converts rule.yml for this target, diffs it against the approved copy, and deploys it.
6. Raise the alert The alert carries the detection id, a link to the record, the severity, and the events that matched. SNS or EventBridge into a case manager such as Jira, ServiceNow, or TheHive The Response section of the record, linked from the alert, which is what an analyst reads at 3 a.m.
7. Triage in the SOC An analyst works the alert, records a disposition and the minutes it took, and closes it. Case management, with the disposition stored in a field somebody can count Nothing, and every number that decides this detection's future comes out of that field.
8. Tune or retire Precision over 30 days and minutes per true positive go back to the detection, and a change to either becomes a pull request. A scheduled report out of case management, read at the monthly review A tuning pull request that adds one named exclusion, or a retirement proposal with the trigger that fired.

Whatever runs at step 5, the rule file stays the same. A streaming runtime evaluates one event at a time, which suits an object store that notifies on arrival. A scheduled search in a SIEM reads a window of indexed events. Scheduled SQL in a warehouse reads the same window out of a table. What changes per target is the conversion, and section 9 covers what the warehouse variant adds on top.

Steps 1 to 5 are infrastructure, so each one has a name on every cloud:

Step On Azure On Google Cloud
1. Collect Azure Activity log, Entra ID sign-in logs, and Defender for Cloud alerts Cloud Audit Logs and Security Command Center findings
2. Land in object storage Blob Storage, or Data Lake Storage Gen2 A Cloud Storage bucket
3. Announce the new objects Event Grid on Microsoft.Storage.BlobCreated, into Storage queues or Service Bus Pub/Sub notifications for Cloud Storage, or an Eventarc trigger
4. Normalize the fields A Function or a Data Factory job writing the agreed names A Dataflow or Cloud Run job writing the agreed names
5. Run the detection A Function, or a scheduled analytics rule in Microsoft Sentinel A Cloud Run function, or a scheduled query in BigQuery

The pipeline above is a shape, and what fills it is whatever the team already runs. The tables below say what each choice gives this job and where it stops. I read each one against the vendor's own documentation on 2026-10-06. They are documented mechanisms and not products I run, and the names move fast enough that you should check again before you build on one.

16.2Where the detections live

A detection repository asks four things of its host: a default branch nobody can push to, a check that must pass before a merge, a named owner for the detections directory, and a scan that stops a credential reaching the history. Every host below does at least three.

Host Protected branch and required checks Owner for a directory Secret scanning on a push
GitHub Rulesets, or the older branch protection rules CODEOWNERS, required through the pull request rule Push protection, paid on a private repository
GitLab Protected branches, with pipelines that must succeed CODEOWNERS, with approval enforced on Premium and above Secret push protection, on Ultimate only
Bitbucket Cloud Branch restrictions and merge checks .bitbucket/CODEOWNERS, added as default reviewers Nothing at push time, so add a pipeline step
Bitbucket Data Center Branch permissions and the required builds check .bitbucket/CODEOWNERS, added as reviewers On by default, but it reports after the push lands
Azure Repos Branch policies for reviewers, build validation, and status checks Automatically included reviewers on a path filter, with no CODEOWNERS file Push protection through the paid Advanced Security add-on
Gitea and Forgejo Protected branches with required approvals and status checks CODEOWNERS; Gitea can require the owner's approval and Forgejo can only request it Nothing
Google Secure Source Manager Branch rules with required approvals and checks .securesourcemanager/CODEOWNERS Rejects the push, with a documented skip option

Two hosts are closed to new customers, so neither should start a detection repository: AWS CodeCommit since July 2024, and Cloud Source Repositories since June 2024.

The build needs two credentials and no more: a read-only query credential scoped to the indexes a baseline check reads, and a deploy credential only the deploy job can read. Keep the deploy credential out of pull request builds, because a pull request from a fork runs code a stranger wrote.

16.3What runs the checks

The reference pipeline in section 17 declares its jobs for GitHub Actions: lint, convert, score, and a replay matrix with a service container per engine on every change, then deploy after a merge, and a promotion check on the pull request that switches a detection's alerting on. Any CI system can carry them. What differs is where the jobs are declared, whether a runner can sit inside the network where the SIEM runs, and what a build started by a stranger is allowed to read.

CI system Where the jobs are declared Runners you can own What a build on a fork's pull request reads
GitHub Actions Files under .github/workflows Self-hosted runners, per repository or organization No secrets until a maintainer approves the run
GitLab CI/CD .gitlab-ci.yml Self-managed runners, per group or per project Protected variables only where the project allows them
Azure Pipelines azure-pipelines.yml Self-hosted agent pools Nothing, until an administrator turns fork secrets on
Jenkins Jenkinsfile Agents are the whole model What the fork trust setting allows, and Collaborators is the recommended one
CircleCI .circleci/config.yml Container runners and machine runners What the context restrictions allow
Buildkite A pipeline file the agent uploads Every agent is yours Whatever your own secret store hands the plugin
Bitbucket Pipelines bitbucket-pipelines.yml Self-hosted runners Secured variables, masked in the log
Tekton, with Pipelines as Code Pipeline runs under .tekton Your own cluster What the repository policy allows, after an ok-to-test comment
Concourse A pipeline file set with fly set-pipeline Workers are yours Only what the pipeline's own credential manager gives it

Woodpecker CI, TeamCity, Argo Workflows, Dagger, and the beta successor to Drone all do the same job. Ask any of them the same four questions: where the jobs are declared, whether a job can run beside a SIEM that has no public address, what a stranger's pull request can read, and whether the repository can refuse a merge on the answer.

16.4What ships the rule into the platform

Delivery takes one of two shapes. The build pushes, calling the platform's API after a merge, which is simple and leaves the platform with no idea the repository exists. Or the platform pulls, reconciling against the repository on its own, which closes the drift question and gives you one more system to operate. Of the platforms in 16.5, only Microsoft Sentinel offers the pull shape itself. Everywhere else the pull side means Argo CD or Flux plus a controller that speaks the vendor's API.

How the rule ships What it reaches What to watch
The platform's own repository connection Microsoft Sentinel, from GitHub or Azure Repos Five connections per workspace, and deleting a file leaves the rule running
Terraform or OpenTofu Sentinel, Elastic, Splunk saved searches, Datadog, Sumo Logic, and Google SecOps A rule edited in the console drifts until the next apply silently overwrites it
The vendor's content CLI Elastic detection-rules, Splunk contentctl-ng, Panther's upload tool, SecOps Content Manager Upload adds and replaces, so a deleted detection needs a delete call of its own
A REST call from the build Elastic's import endpoint, Sekoia's rules catalog, Exabeam, Devo The script is yours to maintain, and the API host is often pinned to one region
GitOps reconciliation with Argo CD or Flux Anything expressed as a Kubernetes resource Neither reaches a SaaS rule without a controller that speaks the vendor's API
Packages pulled onto the node, as falcoctl does for Falco Rules shipped as OCI artifacts Follow mode installs new versions on its own, with no deploy step in between

16.5The platform on the other end

Sigma covers a detection whose logic is a match on fields inside one event, which is most of them. A correlation across events, a stateful sequence, or a scheduled warehouse query goes in as its own query file, with the same record, score, variables, and fixtures beside it. Review happens on that file, because no conversion can regenerate it.

Platform Query language The rule in version control What deploys it
Splunk Enterprise Security SPL One YAML file per detection, built into an app contentctl-ng, then the Admin Config Service on Splunk Cloud
Microsoft Sentinel KQL Bicep or ARM for a scheduled alert rule, or YAML converted into it A repository connection, an az deployment, or Terraform
Microsoft Defender XDR KQL A custom detection rule The Bicep security extension, still in preview
Elastic Security KQL, Lucene, EQL, and ES|QL One TOML file per rule The detection-rules CLI, through the detection engine API
Google SecOps YARA-L 2.0 over UDM Rule text beside the state that says whether it runs The rules API, Content Manager, or the Terraform resource
CrowdStrike Falcon Next-Gen SIEM Its correlation rule query A Terraform resource per rule The CrowdStrike Terraform provider
CrowdStrike Falcon LogScale LogScale query A package: a manifest with alerts, parsers, and queries beside it Installed per repository or view, so a fleet rollout is many installs
Panther Python, or Simple Detections YAML A .py and .yml pair per detection The upload command, with an API token the build carries
Datadog Cloud SIEM Its own query syntax A Terraform resource per rule The Datadog Terraform provider
Sumo Logic Cloud SIEM Its rule expressions Terraform, with a different resource per rule shape The Sumo Logic Terraform provider
Sekoia.io Sigma, used natively The Sigma rule itself A post to the rules catalog, scripted by you
Grafana Loki LogQL A ruler rules file lokitool, or the ruler API
IBM QRadar AQL, plus the rule engine A content export The content management tool; the SaaS product reached end of life in April 2026
A warehouse: Snowflake, Databricks, BigQuery, Athena SQL A .sql file beside the schedule that runs it A task or alert, a scheduled query, or a scheduler rule

Sigma reaches these platforms unevenly, and the gap is worth measuring before a team commits to one rule format. Sekoia reads Sigma directly. Splunk, Elastic, OpenSearch, Kusto, Loki, CrowdStrike, Panther, and SQL all have a converter that works. Google SecOps converts only rules whose fields already have a UDM name, and the QRadar converter that still builds is the one IBM maintains. Correlation across events converts for fewer targets again, so a rule that counts or sequences is portable in theory more than in practice.

Section 17 takes the pipeline I run against this framework's own example detections, job by job.

16.6Where this pipeline breaks

What breaks What you see The check that finds it
The notification queue wedges The bucket keeps filling, every log source reports healthy, and nothing has run for six hours. Alert on the age of the oldest message, the way DET-0000 alerts on a silent source.
A normalization change renames a field Every rule still converts and every build still passes, and the live queries match nothing. Field names come from real events at research, and re-validation replays the fixtures against the current schema.
Somebody deploys by hand during an incident The live rule and the approved copy diverge, so the detection's tests prove nothing about what runs. The daily running check, which compares the running query with its approved copy.
Retention is shorter than the longest window You can't re-measure a detection before tuning it, so the tuning argument comes down to opinion. Set retention from the longest window any detection declares, and check it again when a window moves.
Alert delivery fails The failure is itself an alert that nobody receives, and the queue looks quiet because it is empty. A dead-letter queue behind the alert topic, with what lands in it counted as a program metric.
A retired detection keeps running The rule is gone from the repository and still live on the platform, because an upload adds and replaces but never deletes. Make the delete its own deploy step, and count live rules against the repository at each review.
A notification fires before the object is finished The runtime reads a partial file, so the detection misses the events that were still being written. Subscribe to the event that marks the write complete, which on Data Lake Storage Gen2 means FlushWithClose.

These failures make no noise, with one exception: the retention gap announces itself on the day you need the data. A detection that throws an error gets noticed; a detection that runs, matches nothing, and reports success stays unnoticed for as long as nobody asks.

Next step The reference pipeline