DETECT hunt Reference

Section 17

The reference pipeline

Framework v0.136.0, updated November 17, 2026.

StageWhat the definition of done countsThe job that fails when it is skipped
Intake6 scores, each with an evidence line, and 1 computed scorescore
Hypothesis1 hypothesis sentence, 1 named log source, 1 named blind spotlint
Research1 emulation run, fixtures saved, every field checked against a real event, 7 days of baseline counts, lab torn downlint
Buildlint passes, 1 conversion per target, 1 event it must match, 1 event it must ignorelint convert
Validatefixtures pass on every replay engine, every converted query approved (matching its approved copy on a re-validation), 1 approval by someone other than the author (one-person team: the author after 24 hours)lint convert replay (one leg per engine) golden-diff review
Deploy7 days live with alerting off, alert volume at or under its threshold, then alerting ondeploy promote-check field-check (scheduled) running-check (scheduled)
Tuneprecision checked every 30 days, running query matches the approved copy at each checklint (core lock, on pull requests) field-check (scheduled) running-check (scheduled) precision-check (scheduled, planned)
Retirestatus deprecated, alerting off, record kept, retirement note written, replacement linked if one existsdeploy (the next package leaves it out)

Remember

  • Copy the repository layout and replace the example detection with your own.
  • The scorer writes every score, and CI fails any score that disagrees with the formula.
  • Convert once, commit the approved copies, then review every difference after that.
  • Lint also fails when AGENTS.md no longer matches framework.json.
  • The Splunk replay leg needs only its container's password and HEC token. The promotion check needs a read-only Splunk account in the siem-read environment, and the two daily checks need the same account in siem-read-daily. The deploy job needs the deploy credential in the production environment.

If you're using the repository as a template, replace the example and point the secrets at a Splunk Cloud stack. Each detection is a directory. It carries rule.yml (Sigma), ads.md (the record), score.yml (the inputs, an evidence line for each, and a CI-written score), the fixtures, and the approved copies of the converted queries.

Stage Sigma status CI jobs Artifact
1. Intake no rule file yet score score.yml with stage set to intake
2. Hypothesis no rule file yet lint The Goal, Categorization, Strategy Abstract, and Blind Spots and Assumptions sections of ads.md
3. Research no rule file yet lint The Technical Context section of ads.md; the emulation plan and run report; tests/events.jsonl
4. Build experimental lint, convert rule.yml, tests/events.jsonl, tests/expected.yml
5. Validate test lint, convert, replay (one leg per engine), golden-diff, review Passing CI run; approved pull request
6. Deploy stable deploy, promote-check, field-check (scheduled), running-check (scheduled) The pull request that moves the detection from deploy to tune, carrying the promotion check's count for its first 7 days live
7. Tune stable lint (core lock, on pull requests), field-check (scheduled), running-check (scheduled), precision-check (scheduled, planned) A tuning pull request that cites the alert ids that prompted it
8. Retire deprecated deploy (the next package leaves it out) A retirement note in ads.md: reason, date, and the id of the replacement detection

The definitions of done call these tools in the repository:

Tool Jobs it backs What it does
lint lint Checks the layout and parses the detection. It confirms the id and tag agree, the Sigma status matches the stage, the ADS sections are present, and every named fixture exists. It also validates the score record, rejects any ATT&CK id outside the pinned map, and fails when AGENTS.md no longer matches framework.json.
convert convert, golden-diff Turns each Sigma rule into all 13 targets in targets.yml, and fails when any output differs from its approved copy.
score score Computes each record's score and sorts the backlog.
metrics score Adds to the score job's metrics the numbers only the file layout, incidents.csv, and notifications.csv can tell.
package deploy Builds a Splunk app from the approved queries. A detection at deploy runs with alerting off, one at tune runs with it on, and a retired one is left out, which takes it out of production.
replay replay, promote-check Loads the fixtures into a real instance of each replay engine (SQLite, Splunk, Kusto, Elasticsearch, ES|QL, OpenSearch). On the pull request that moves a detection to tune, it counts the first 7 days live against the threshold.
history promote-check Reads from git when each detection changed stage and when it went live, so no record carries a date by hand.
fields field-check Once a day, counts the events from each live rule's log source and how many carry each field the rule reads. It fails a source that sent nothing and a field that was empty in every event.
running running-check Once a day, compares each live rule's saved search in Splunk with what the package writes from main. It fails a changed query, schedule, or alert condition, a disabled search, alerting at the wrong stage, and a search in the app with no live detection behind it.
next none; run by hand Runs the checks a detection's current stage needs, in one command, and prints what is still missing, the count the stage must reach, and the step that moves the detection on.
intake none; run by hand Prints the weekly intake meeting's agenda from the records and git: new candidates, the ones blocked on telemetry, the queue in scored order, the work in flight, and what has not moved in a month.

The precision-check job is planned and not yet written. The other seven targets (EQL, Loki, LogScale, NetWitness, Panther, Microsoft Defender XDR advanced hunting, Google SecOps) have no public test instance. Their conversions stay drafts until someone runs them on a real one. Separate upkeep tools reshape fixtures for each engine, refresh the pinned ATT&CK map, and regenerate the agent instructions.

I cap each query backend below its next major version in requirements.txt. A major release can change the query a backend emits, and the golden check catches that. When pysigma-backend-sqlite 2.0.0 renamed the table placeholder, it failed the check on DET-0001. To move a cap, you open a pull request that updates the approved copies in the same change.

These have run on the example: lint, convert with the golden check on all 13 targets, score, metrics, packaging with one detection at deploy and one at tune, and the promotion check against a stubbed search. The two daily checks ran against a real Splunk in a container. The field check ran on the fixtures with fields renamed, blanked, and removed, and the running check on the packaged app after edits made through the console's own API. Every replay adapter has run end to end against a real engine in a container: SQLite, Splunk, the Kusto emulator, Elasticsearch, and OpenSearch each returned the two fixtures DET-0001 must match and none of the three it must ignore. The site's own build also copies the kit and breaks it on purpose, one case per check: a rule only some targets can convert, a regular expression, a silent re-approval, each stage move the lifecycle refuses, and each count the promotion check fails on. The build stops if any tool misses its break. These haven't run anywhere yet and stay placeholders until they do: the container services inside the GitHub Actions workflow, and the AppInspect and ACS steps.

Next step Detection as code