Appendix A
Appendix A: Definitions of done and stalls
Every stage's definition of done in full, with the mistake that most often stalls it.
| Stage | You may leave when | The usual stall | If it can't be met |
|---|---|---|---|
| 1. Intake | The request is written down. It names the behavior, the asset, and who asked. Six scores are recorded with one line of evidence each. | Requests arrive as nothing more than an alert name, with no note about the threat behind it, so you build them in the order they showed up. | A candidate whose scores can't be evidenced stays at intake, and the missing evidence becomes a question for the next intake meeting. A candidate with telemetry readiness 0 is blocked, and its log source request goes to the platform owner. |
| 2. Hypothesis | The record carries a one-sentence hypothesis: when an attacker does X, log source Y records Z. It names the log source. It names one thing the rule cannot see. | The hypothesis describes one tool's fingerprint. When that tool changes, the detection goes quiet. | If no log source you collect records the behavior, the candidate goes back to intake with telemetry readiness 0 and a log source request. If the only hypothesis anyone can write names one tool's fingerprint, rescore its robustness at intake before going on. |
| 3. Research | The behavior has been emulated from a believable starting identity. The events it produced are saved as fixtures. Every field the rule will use was read from a real event. Its normal rate has been counted over 7 days of production data. The lab is torn down and the teardown verified by run tag. | Field names come from documentation or memory, so the detection is almost right and never fires. The other common failure is skipping the 7-day count, which spends the first week in production on noise the count would have shown. | If the events never reach the SIEM, or a field the rule needs arrives unparsed, the candidate goes back to intake with its telemetry readiness lowered and a log source request. If the 7-day count shows the behavior is routine here, rewrite the hypothesis before building anything. |
| 4. Build | The rule file passes lint. It converts cleanly into every target in the registry. It has one event it must match and one it must ignore. | You write the detection straight in the SIEM's own query language. Then it runs on one product only, you can't compare its changes against an approved version, and it can't move when the SIEM changes. | If a target can't express the rule, mark that target convert-only in targets.yml and say so in the record. If Sigma can't express it at all, as with a correlation across events, write it as a query file beside the same record and fixtures, and review that file directly. |
| 5. Validate | The fixtures run against a real instance of every engine that has one, and each returns what it should. Every converted query matches the approved copy, or the difference is explained and accepted. A second engineer has approved the change. On a one-person team the approval comes from a colleague outside the team, a peer elsewhere, or the author after a 24-hour wait. | Whenever the conversion output changes, someone overwrites the approved copy to match it. After that the comparison can't fail, so it catches no changes. | If a fixture fails or the reviewer asks for changes, the pull request goes back to build and stays open. If the reviewer rejects the hypothesis itself, the detection goes back to hypothesis. |
| 6. Deploy | The detection is live with alerting off. Over its first 7 days live it returned no more alerts than the threshold on its record, and at least one if research counted the behavior in its baseline. A pull request then moves it to tune, and the move switches alerting on. | Alerting goes on before the 7-day check, and the SOC pays for the noise in triage time. | If the first 7 days run over the threshold, alerting stays off and the detection stays at deploy. The fix is a pull request that goes back through validate, and the 7 days start again from its merge. If they return nothing though research counted the behavior, check the fields and the index against a real event. |
| 7. Tune | Over the last 30 days its precision is at or above its floor. Every tuning change went through a pull request and back through validate, and the query running in production matches the approved copy. | Someone tunes the detection by adding exclusions in the SIEM's own interface, so the rule in the repo no longer matches the rule running in production. | If precision stays under the floor for two 30-day periods in a row, propose retirement. If an incident or an exercise shows the detection missed what it claims to catch, fix this detection. The missed events become a positive fixture, and the change goes back through validate and its 7-day count. |
| 8. Retire | Status is deprecated and the detection is off in production. A retirement note records why, and any replacement is at intake with a link to that note. | Nothing is ever retired, so analysts keep triaging alerts from detections that no one owns or understands anymore. | A retirement is not undone. If the technique still matters, the replacement is a new detection at intake with a link to the retirement note. |