Section 7
Worked examples
Do it in the browser Score a backlog
Remember
- Work a score in order: value, cost, their ratio, the readiness factor, then marginal coverage.
- A cheap detection still loses when an attacker can evade it by renaming a tool.
- Retire a detection when a trigger fires, and write the retirement note before you switch it off.
The examples below take DET-0001 to DET-0005 through the decisions one at a time. DET-0101, the password spray detection whose record is on the site, appears only in the ordered backlog. The environment details are illustrative, and the repository's own scorer computes every number when this paper is generated, so the numbers can't drift from the code.
On this page
7.1Scoring one candidate
DET-0001 detects Kerberoasting (ATT&CK T1558.003). Each of its scores answers one question and carries one line of evidence. The evidence line makes the score arguable, because a reviewer can dispute its source.
| Input | Score | The question it answers | The evidence line on the record |
|---|---|---|---|
| threat relevance (T) | 4 | How recently, and how close to you, has this technique been used? | sector ISAC report, Q2 2026, section 3 |
| impact (I) | 4 | What does the attacker reach if the technique succeeds? | service account escalation path to tier-0 |
| robustness (B) | 4 | How hard is it for an attacker to slip past this detection using the data you have? Score it with the five Summiting the Pyramid levels: 1 matches something the attacker can change in seconds, 5 matches something the technique cannot work without. | event 4769, RC4 ticket encryption, non-machine account |
| telemetry readiness (R) | 4 | Is the data this detection needs already in the SIEM, and in good enough shape to use? | DC security logs parsed, no enrichment |
| build effort (E) | 2 | How many hours of work from writing the hypothesis to passing validation? | 6 hours, comparable to DET-0007 |
| run cost (C) | 2 | How many analyst minutes a day will this detection consume? Expected alerts per day multiplied by the typical minutes to triage one. For a detection that runs as scheduled SQL, add the warehouse compute it burns, converted the same way your finance team converts it. | 10 analyst minutes per day, estimated before any 7-day count |
On the balanced preset, where every weight is 1, the arithmetic goes like this:
- Add the value side. value = 4 + 4 + 4 = 12.
- Add the cost side. cost = 2 + 2 = 4.
- Divide value by cost. 12 / 4 = 3.00.
- Apply the readiness factor. Readiness is 4 out of 5, so the factor is (4/5) = 0.80. 3.00 × 0.80 = 2.40.
A score of 2.40 only means something when you compare it with the rest of the backlog. On the balanced preset, scores run from 0.06 to 7.50, and the median of the six example records in this section is 1.50. This candidate sits above the middle, so it's worth building before most of what's waiting. The scorer writes the score into the record, and CI fails a score typed in by hand.
7.2Two candidates in a different order
DET-0002 detects a scheduled task created by a non-administrator (T1053.005). It's a reasonable detection, and it still ranks below DET-0001 for three reasons: a smaller value side, a higher run cost, and lower readiness.
| Input | DET-0001 Kerberoasting | DET-0002 Scheduled task |
|---|---|---|
| threat relevance (T) | 4 | 3 |
| impact (I) | 4 | 2 |
| robustness (B) | 4 | 3 |
| telemetry readiness (R) | 4 | 3 |
| build effort (E) | 2 | 2 |
| run cost (C) | 2 | 4 |
| value | 12 | 8 |
| cost | 4 | 6 |
| score | 2.40 | 0.80 |
- Add the value side. value = 3 + 2 + 3 = 8.
- Add the cost side. cost = 2 + 4 = 6.
- Divide value by cost. 8 / 6 = 1.33.
- Apply the readiness factor. Readiness is 3 out of 5, so the factor is (3/5) = 0.60. 1.33 × 0.60 = 0.80.
Run cost and readiness make most of the gap. The run cost evidence on DET-0002 reads "80 analyst minutes per day", which scores 4; on DET-0001 it reads "10 analyst minutes per day, estimated before any 7-day count", which scores 2. Both are built on parsed logs, but DET-0001's source is the better prepared of the two.
Changing the weights doesn't rescue it. Each preset leans the score a different way, and the order stays the same under every one:
| Preset | DET-0001 | DET-0002 | Order |
|---|---|---|---|
| balanced | 2.40 | 0.80 | unchanged |
| threat_led | 3.20 | 1.10 | unchanged |
| asset_led | 3.20 | 1.00 | unchanged |
| telemetry_led | 1.92 | 0.48 | unchanged |
That's why I wrote the weights down. When somebody argues for a different lean, you change one number and the backlog re-sorts in front of them.
7.3A candidate the score blocks
DET-0004 detects a program download to a PLC from an engineering workstation (ATT&CK for ICS T0843). It's the highest-value candidate in the sample: impact 5, robustness 5, threat relevance 4. Its value side adds to 14, the highest in the sample, and it scores 0.00.
Telemetry readiness is 0: no OT network monitoring log source ingested. A readiness of 0 sets the score to zero and marks the record blocked. A detection can't alert on telemetry nobody collects, so the work item becomes a log source onboarding ticket. You argue for it as an onboarding ticket, with the detection attached as the reason.
Once you ingest and parse the log source, readiness moves to 4. The same value and the same cost (build effort 5, run cost 1) give 14 / 6 × 0.80 = 1.87, third by score in this backlog. Nothing else about the candidate changed.
7.4A candidate the live set already covers
DET-0005 detects the same Kerberoasting technique as DET-0001, by matching the name of the tool that requests the ticket. It's cheap, at two hours to build and four analyst minutes a day. On its inputs alone it scores 3.60, better than DET-0001, because it costs so little.
| Input | Score | The evidence line on the record |
|---|---|---|
| threat relevance (T) | 4 | same sector ISAC report as DET-0001, Q2 2026, section 3 |
| impact (I) | 4 | service account escalation path to tier-0, same as DET-0001 |
| robustness (B) | 1 | matches the requesting process name, a filename the attacker renames in seconds (Summiting level 1) |
| telemetry readiness (R) | 4 | DC security logs parsed, no enrichment |
| build effort (E) | 1 | 2 hours, the query is a filter on an existing search |
| run cost (C) | 1 | 4 analyst minutes per day from the 7-day baseline |
The record is scored for the day DET-0001 goes live, because that's when the comparison matters. In the example set no live record covers T1558.003 yet, because DET-0001 is still at build, so DET-0005's record carries the marginal state by hand. In a repository with a live set the pipeline compares the candidate against it, finds T1558.003 covered at robustness 4 against this candidate's 1, and marks it equal. That halves its score to 1.80. A low cost doesn't make up for low robustness. If an attacker renames the binary, this detection is worth nothing, while the behavioral detection that's already live keeps firing.
The marginal factor is there for this case. Without it, a backlog fills up with cheap variants of what already works. Each one scores well on its own, and the coverage chart climbs while nothing new gets detected.
That doesn't make the candidate automatically wrong. If somebody can write down a variant this one catches and DET-0001 misses, it earns its place at half score. If nobody can, close it.
7.5A detection somebody else's deadline decides
DET-0003 detects an administrator logging in to the cardholder database outside a change window. On the numbers it scores 1.20, mid-table. It still goes first, because it carries a mandate: PCI DSS 4.0, requirement 10.4.1, due 2026-11-30. Measured from 2026-09-19, the as-of date the example is computed on, that's 72 days away, inside the 90-day window, so it sorts to the front by due date.
The scorer orders the whole six-record backlog like this. Live records stay in the list so every candidate is compared against the same set:
| Order | Id | Detection | Score | Status | Why it sits there |
|---|---|---|---|---|---|
| 1 | DET-0003 | PCI cardholder DB admin login outside change window | 1.20 | mandated | mandate due 2026-11-30 |
| 2 | DET-0101 | Password spray against Entra ID sign-in | 2.60 | in_progress | by score |
| 3 | DET-0001 | Kerberoasting | 2.40 | in_progress | by score |
| 4 | DET-0005 | Kerberoasting, RC4 ticket request by a known tool name | 1.80 | scored | by score |
| 5 | DET-0002 | New scheduled task by non-admin user | 0.80 | live | by score |
| 6 | DET-0004 | OT engineering workstation PLC program download | 0.00 | blocked | readiness 0 |
Keep mandated work in the same backlog with its reason recorded, so the time it takes shows up in the same place as everything else.
7.6Retiring a detection
Retirement is arithmetic, then a check against the triggers. The figures in the two cases below are illustrative inputs, so use the method with your own measurements.
Measure the detection over 30 days: how many alerts it raised, how many were true positives, and how many minutes a typical triage took. Its floor is the triage minutes divided by the cap on analyst minutes per true positive, 120 by default. Compare precision with the floor in two consecutive 30-day windows. One window below the floor triggers nothing, and two in a row is a retirement trigger.
| Case | Alerts, 30 days | True positives | Precision | Triage mins | Its floor | Mins per true positive | Verdict |
|---|---|---|---|---|---|---|---|
| Keeps its place | 60 | 22 | 36.7% | 25 | 20.8% | 68 min | clears the floor |
| Retires | 900 | 27 | 3.0% | 14 | 11.7% | 467 min | below the floor, second window in a row |
The detection that stays is expensive per alert and still clears its floor. Its 60 alerts for 22 real findings cost 1500 analyst minutes, about 68 minutes per true positive and inside the 120-minute cap. The detection that retires is cheap per alert and costly in total. Its 900 alerts at 14 minutes each come to 12,600 minutes, or 210 analyst hours for 27 findings.
Then check the triggers. Any one of them proposes retirement:
- Its true-positive share stayed below its floor for two 30-day periods in a row.
- It fired at least 10 times in 180 days and was never right.
- The log source it needs has been missing for 30 days straight, so its readiness fell to 0. Suspend it with alerting off and raise a log source request, and retire it only if the source owner declines to restore the source.
- The technique or the system it protects is gone, for example because the platform was decommissioned.
- Another live detection covers the same technique at an equal or higher robustness level and catches everything this one catches. Retire the weaker one in the same change that ships the stronger one.
When one fires, decide between tuning and retiring. Tune when the noise has one identifiable source you can exclude without losing the behavior, such as a service account, a backup window, or a scanner. Retire when the only way to quiet it is to narrow it until it no longer detects the thing it was written for. If you can't say which exclusion would fix it, you're retiring it.
To retire it, set the Sigma status to deprecated, disable the alert action, and keep the record and its history. Write a note saying which trigger fired and what you measured. If the technique still matters, its replacement enters intake with a link back to that note. That's the second loop in the lifecycle.
A detection that hasn't fired at all in 365 days gets a review before anyone retires it, because either its data stopped or its hypothesis was wrong. The silence alert on its log source tells you which.