DETECT hunt Build detections

IdeaMeasure

Section 7

Worked examples

Framework v0.136.0, updated November 17, 2026.

Do it in the browser Score a backlog

Remember

  • Work a score in order: value, cost, their ratio, the readiness factor, then marginal coverage.
  • A cheap detection still loses when an attacker can evade it by renaming a tool.
  • Retire a detection when a trigger fires, and write the retirement note before you switch it off.

The examples below take DET-0001 to DET-0005 through the decisions one at a time. DET-0101, the password spray detection whose record is on the site, appears only in the ordered backlog. The environment details are illustrative, and the repository's own scorer computes every number when this paper is generated, so the numbers can't drift from the code.

On this page

7.1Scoring one candidate

DET-0001 detects Kerberoasting (ATT&CK T1558.003). Each of its scores answers one question and carries one line of evidence. The evidence line makes the score arguable, because a reviewer can dispute its source.

Input Score The question it answers The evidence line on the record
threat relevance (T) 4 How recently, and how close to you, has this technique been used? sector ISAC report, Q2 2026, section 3
impact (I) 4 What does the attacker reach if the technique succeeds? service account escalation path to tier-0
robustness (B) 4 How hard is it for an attacker to slip past this detection using the data you have? Score it with the five Summiting the Pyramid levels: 1 matches something the attacker can change in seconds, 5 matches something the technique cannot work without. event 4769, RC4 ticket encryption, non-machine account
telemetry readiness (R) 4 Is the data this detection needs already in the SIEM, and in good enough shape to use? DC security logs parsed, no enrichment
build effort (E) 2 How many hours of work from writing the hypothesis to passing validation? 6 hours, comparable to DET-0007
run cost (C) 2 How many analyst minutes a day will this detection consume? Expected alerts per day multiplied by the typical minutes to triage one. For a detection that runs as scheduled SQL, add the warehouse compute it burns, converted the same way your finance team converts it. 10 analyst minutes per day, estimated before any 7-day count

On the balanced preset, where every weight is 1, the arithmetic goes like this:

  1. Add the value side. value = 4 + 4 + 4 = 12.
  2. Add the cost side. cost = 2 + 2 = 4.
  3. Divide value by cost. 12 / 4 = 3.00.
  4. Apply the readiness factor. Readiness is 4 out of 5, so the factor is (4/5) = 0.80. 3.00 × 0.80 = 2.40.

A score of 2.40 only means something when you compare it with the rest of the backlog. On the balanced preset, scores run from 0.06 to 7.50, and the median of the six example records in this section is 1.50. This candidate sits above the middle, so it's worth building before most of what's waiting. The scorer writes the score into the record, and CI fails a score typed in by hand.

7.2Two candidates in a different order

DET-0002 detects a scheduled task created by a non-administrator (T1053.005). It's a reasonable detection, and it still ranks below DET-0001 for three reasons: a smaller value side, a higher run cost, and lower readiness.

Input DET-0001 Kerberoasting DET-0002 Scheduled task
threat relevance (T) 4 3
impact (I) 4 2
robustness (B) 4 3
telemetry readiness (R) 4 3
build effort (E) 2 2
run cost (C) 2 4
value 12 8
cost 4 6
score 2.40 0.80
  1. Add the value side. value = 3 + 2 + 3 = 8.
  2. Add the cost side. cost = 2 + 4 = 6.
  3. Divide value by cost. 8 / 6 = 1.33.
  4. Apply the readiness factor. Readiness is 3 out of 5, so the factor is (3/5) = 0.60. 1.33 × 0.60 = 0.80.

Run cost and readiness make most of the gap. The run cost evidence on DET-0002 reads "80 analyst minutes per day", which scores 4; on DET-0001 it reads "10 analyst minutes per day, estimated before any 7-day count", which scores 2. Both are built on parsed logs, but DET-0001's source is the better prepared of the two.

Changing the weights doesn't rescue it. Each preset leans the score a different way, and the order stays the same under every one:

Preset DET-0001 DET-0002 Order
balanced 2.40 0.80 unchanged
threat_led 3.20 1.10 unchanged
asset_led 3.20 1.00 unchanged
telemetry_led 1.92 0.48 unchanged

That's why I wrote the weights down. When somebody argues for a different lean, you change one number and the backlog re-sorts in front of them.

7.3A candidate the score blocks

DET-0004 detects a program download to a PLC from an engineering workstation (ATT&CK for ICS T0843). It's the highest-value candidate in the sample: impact 5, robustness 5, threat relevance 4. Its value side adds to 14, the highest in the sample, and it scores 0.00.

Telemetry readiness is 0: no OT network monitoring log source ingested. A readiness of 0 sets the score to zero and marks the record blocked. A detection can't alert on telemetry nobody collects, so the work item becomes a log source onboarding ticket. You argue for it as an onboarding ticket, with the detection attached as the reason.

Once you ingest and parse the log source, readiness moves to 4. The same value and the same cost (build effort 5, run cost 1) give 14 / 6 × 0.80 = 1.87, third by score in this backlog. Nothing else about the candidate changed.

7.4A candidate the live set already covers

DET-0005 detects the same Kerberoasting technique as DET-0001, by matching the name of the tool that requests the ticket. It's cheap, at two hours to build and four analyst minutes a day. On its inputs alone it scores 3.60, better than DET-0001, because it costs so little.

Input Score The evidence line on the record
threat relevance (T) 4 same sector ISAC report as DET-0001, Q2 2026, section 3
impact (I) 4 service account escalation path to tier-0, same as DET-0001
robustness (B) 1 matches the requesting process name, a filename the attacker renames in seconds (Summiting level 1)
telemetry readiness (R) 4 DC security logs parsed, no enrichment
build effort (E) 1 2 hours, the query is a filter on an existing search
run cost (C) 1 4 analyst minutes per day from the 7-day baseline

The record is scored for the day DET-0001 goes live, because that's when the comparison matters. In the example set no live record covers T1558.003 yet, because DET-0001 is still at build, so DET-0005's record carries the marginal state by hand. In a repository with a live set the pipeline compares the candidate against it, finds T1558.003 covered at robustness 4 against this candidate's 1, and marks it equal. That halves its score to 1.80. A low cost doesn't make up for low robustness. If an attacker renames the binary, this detection is worth nothing, while the behavioral detection that's already live keeps firing.

The marginal factor is there for this case. Without it, a backlog fills up with cheap variants of what already works. Each one scores well on its own, and the coverage chart climbs while nothing new gets detected.

That doesn't make the candidate automatically wrong. If somebody can write down a variant this one catches and DET-0001 misses, it earns its place at half score. If nobody can, close it.

7.5A detection somebody else's deadline decides

DET-0003 detects an administrator logging in to the cardholder database outside a change window. On the numbers it scores 1.20, mid-table. It still goes first, because it carries a mandate: PCI DSS 4.0, requirement 10.4.1, due 2026-11-30. Measured from 2026-09-19, the as-of date the example is computed on, that's 72 days away, inside the 90-day window, so it sorts to the front by due date.

The scorer orders the whole six-record backlog like this. Live records stay in the list so every candidate is compared against the same set:

Order Id Detection Score Status Why it sits there
1 DET-0003 PCI cardholder DB admin login outside change window 1.20 mandated mandate due 2026-11-30
2 DET-0101 Password spray against Entra ID sign-in 2.60 in_progress by score
3 DET-0001 Kerberoasting 2.40 in_progress by score
4 DET-0005 Kerberoasting, RC4 ticket request by a known tool name 1.80 scored by score
5 DET-0002 New scheduled task by non-admin user 0.80 live by score
6 DET-0004 OT engineering workstation PLC program download 0.00 blocked readiness 0

Keep mandated work in the same backlog with its reason recorded, so the time it takes shows up in the same place as everything else.

7.6Retiring a detection

Retirement is arithmetic, then a check against the triggers. The figures in the two cases below are illustrative inputs, so use the method with your own measurements.

Measure the detection over 30 days: how many alerts it raised, how many were true positives, and how many minutes a typical triage took. Its floor is the triage minutes divided by the cap on analyst minutes per true positive, 120 by default. Compare precision with the floor in two consecutive 30-day windows. One window below the floor triggers nothing, and two in a row is a retirement trigger.

Case Alerts, 30 days True positives Precision Triage mins Its floor Mins per true positive Verdict
Keeps its place 60 22 36.7% 25 20.8% 68 min clears the floor
Retires 900 27 3.0% 14 11.7% 467 min below the floor, second window in a row

The detection that stays is expensive per alert and still clears its floor. Its 60 alerts for 22 real findings cost 1500 analyst minutes, about 68 minutes per true positive and inside the 120-minute cap. The detection that retires is cheap per alert and costly in total. Its 900 alerts at 14 minutes each come to 12,600 minutes, or 210 analyst hours for 27 findings.

Then check the triggers. Any one of them proposes retirement:

  1. Its true-positive share stayed below its floor for two 30-day periods in a row.
  2. It fired at least 10 times in 180 days and was never right.
  3. The log source it needs has been missing for 30 days straight, so its readiness fell to 0. Suspend it with alerting off and raise a log source request, and retire it only if the source owner declines to restore the source.
  4. The technique or the system it protects is gone, for example because the platform was decommissioned.
  5. Another live detection covers the same technique at an equal or higher robustness level and catches everything this one catches. Retire the weaker one in the same change that ships the stronger one.

When one fires, decide between tuning and retiring. Tune when the noise has one identifiable source you can exclude without losing the behavior, such as a service account, a backup window, or a scanner. Retire when the only way to quiet it is to narrow it until it no longer detects the thing it was written for. If you can't say which exclusion would fix it, you're retiring it.

To retire it, set the Sigma status to deprecated, disable the alert action, and keep the record and its history. Write a note saying which trigger fired and what you measured. If the technique still matters, its replacement enters intake with a link back to that note. That's the second loop in the lifecycle.

A detection that hasn't fired at all in 365 days gets a review before anyone retires it, because either its data stopped or its hypothesis was wrong. The silence alert on its log source tells you which.