An open framework for running and measuring a detection engineering program.
Writing a detection is the easy part. The hard parts are deciding which one to write, proving it works, and admitting when it has stopped earning its alerts. So here every detection is scored before anyone builds it, tested against real events before it ships, and retired when its alerts cost more than they return.
On this page
Starting points
Follow the step-by-step checklist, score the candidate, and translate the query for the target platform.
Ship a detectionScore a backlogQuery languagesTranslate a queryDetection libraryTemplatesSet up the weekly routine, score the program's maturity, plan the quarter, and report it.
Operating rhythmScore your maturityPlan OKRs and a roadmapQuarterly reportFind your ceilingMap your ATT&CK coverageAnswer three questions about log sources, incident history, and engineering hours, and get the first 10 detections and a plan for the first month.
Find a starting pointFirst stepsSizing the practiceSee what leadership gets, the objections a team raises, what has been checked against outside sources, and the license.
What an executive getsHandling objectionsChecks against outside sourcesSizing the practiceLicense and creditGrab the paper, both workbooks, the data file, and the agent instructions.
Downloadsframework.jsonAGENTS.mdSee which AI uses to adopt first, and whether to use the SIEM's AI, buy, or assemble. Then grade a product, and prove a triage agent before it closes an alert.
Choosing AIGrade an AI productThe closed loopWorking with agentsWhy it exists
I've built a detection engineering program from scratch and led incident response teams, and four things stayed true the whole way through. The backlog is longer than the team can ever build. Every live detection charges analysts a few minutes of triage a day, whether or not it ever catches anything. Counting detections per tactic flatters the coverage map. And rules go into production far more often than they come out, mostly because nobody gets promoted for deleting one. Leave all four alone and a program that only builds will bury its own analysts. The assumptions page walks through each of these realities and what follows from them.
What it adds
The best ideas here are other people's. The record format, the rule syntax, the robustness scale, and the basic shape of the score all come from community work that was proven long before I showed up. The Borrowed practices page gives full credit to where they came from.
What I added is the machinery that keeps a detection program honest, and it starts with the least glamorous detection you'll ever ship. Detection zero is an alert for any log source that goes quiet. You build it first because every other rule depends on its logs showing up, and a rule with no logs fails silently, which looks exactly like a quiet week. After that, every stage of the lifecycle ends on a definition of done that is a hard count. CI checks the count at every stage and fails the pipeline when it comes up short, so "done" stops being a matter of opinion. I owe one confession here. The check for tune's 30-day precision isn't written yet, so a person computes that number. Scoring happens at intake, before anyone spends an engineering hour. The score blocks any candidate whose telemetry doesn't exist. It charges the candidate for the analyst minutes it will cost every day, plus the compute a SQL detection burns. It also discounts a candidate that overlaps with what the live set already covers.
People are budgeted in minutes, because minutes run out before anything else does. The framework tracks three hard ceilings: analyst triage, detection upkeep, and ready telemetry. One program-wide cap on analyst minutes per true positive gives every detection a precision floor. A live detection that stays below its floor for two 30-day periods in a row has met a retirement trigger. The trigger is arithmetic, so the monthly review can skip the argument about whether it fired (it will find something else to argue about). Pulling a detection starts a second loop in the lifecycle. The replacement enters intake with the old retirement note attached, so the next attempt starts from why the last one was pulled.
The closed loop pushes the same machinery further. Agents run every stage and people keep the approvals. Each check is written against the way an agent would fake its way past it. An agent told to make the tests pass will find the cheapest way to make the tests pass. If you're shopping for AI for detection and response, the same tests come as a buying form. It says which uses to adopt first, what to ask any product, and how to prove a triage agent before you let it close an alert.
What an executive gets
Once a quarter, leadership gets one page. It puts the raw rule count right next to coverage weighted by how hard each detection is to evade, which tends to be the moment the heatmap gets less green. It shows the share of confirmed incidents that a live detection caught first, along with median precision and the analyst minutes spent per true positive.
The page also names whichever capacity ceiling currently binds the program and pairs it with one costed ask, so the conversation is about a number and a price. Last comes a maturity score. It's set by the program's weakest dimension and graded on evidence that anyone outside the team could verify, which rules out grading yourself on vibes. The quarterly report template lays out the page, and the maturity page explains how the grading works.
Contents
The model, and where a program enters it.
Assumptions4 premises. The operational realities of the backlog, alert load, coverage, and retirement.First steps3 questions. Answer three questions about current telemetry and environment to get the first 10 detections.Sizing the practice9 practices. What to adopt first. Scales from one engineer to several teams.The lifecycle8 stages. Every stage ends on a countable definition of done.How one detection goes from idea to production.
Ship a detection8 steps. The engineer's checklist, with the commands and a clear done-when for each step.Prioritization6 inputs. A score that decides what to build next.Emulation3 scopes. How a detection gets real events to test against.Detection record11 sections. What a finished detection documents.Query languages13 targets. One detection written in six query languages, with practical tips for each. The others are listed for reference.Working with agents9 limits. What an agent drafts, and what a person approves.The closed loop6 machine stages. Agents run the stages, people keep the approvals, and each check is written against the way an agent would fake it.Choosing AI12 questions. Which AI uses to adopt first, whether to buy or build, and how to prove a triage agent.The weekly routine, and how to know it is working.
Operating rhythm4 meetings. The standing meetings, the post-incident review, and who owns what.Metrics31 metrics. Each defined once and owned by one stage or by the program.ATT&CK coverage706 techniques. Coverage per technique on ATT&CK 20.1, in the browser.Maturity model11 × 5. Each dimension scored on evidence someone else can check.Scaling3 ceilings. How many detections the team can carry.Objections10 answers. What to say when people push back on the rollout.