The Anonymiser That Passed

A deny-list cleared an article for publication. A second check, written the same afternoon, found the article identified its author anyway.

Symptom

An article carried the status line ready to publish · anonymization verified. It had passed the automated gate: a deny-list of company names, people, product names and domains, run against every artifact at build time. Zero hits. Signed off.

A second check, written the same afternoon for an unrelated reason, flagged it immediately.

The article was not leaking a name. It was leaking an identity, in a single introductory paragraph that read roughly like this:

Context: I'm a CEO at a mid-size B2B [sector] company, ~[NN] people across [N] entities ([four named verticals]).

The bracketed values are redacted here for the obvious reason. What matters is the shape, not the payload: a sector, a headcount band, and a portfolio breakdown, in one sentence.

Read it as a person and it is context-setting. Read it as a query and it is a filter: sector, plus headcount band, plus a very specific portfolio shape. In one region that set has perhaps a dozen members. With the portfolio detail, fewer.

Root cause

The gate answered the wrong question.

A deny-list answers "does this text contain a forbidden string?" That is a useful question and it catches the obvious mistake — pasting a customer name into a case study. But de-anonymisation in practice almost never works that way. It works by intersection. Each fact on its own is harmless and shared by thousands of companies. Together they identify one.

Nothing in the pipeline was measuring intersection. Every check was a membership test against a list, and every one of them passed, correctly, because no forbidden member was present.

There is a second-order failure underneath. The phrase anonymization verified was written by the same process that ran the deny-list, so the sign-off inherited the deny-list's blind spot while sounding like a broader guarantee. A green light that only covers one failure mode should say which one.

Cost

Nothing was published, so the direct cost is zero. That is luck, not process — the article had been sitting in a content folder marked ready for 84 days and would have gone out with the first publishing run.

The real cost is what the near-miss revealed: a second inconsistency in the same paragraph. It stated one headcount band. The system's own memory file, written months earlier, stated a different one. One of them was wrong and nobody knew which, because no check compares a claim against the band the system already committed to elsewhere.

An anonymisation error is not reversible. Caches, archives and model training runs do not honour a retraction.

Fix

Stop counting forbidden words. Count categories.

Four categories were defined — revenue, headcount, sector, geography — each with a small set of patterns. The build now scans every artifact and reports how many distinct categories it touches. The threshold is two.

KRIZENIE PASIEM — 1 artefaktov nesie 2+ identifikacne kategorie:
   annotated-claude-md    revenue:"€5" + headcount:"~NN people"

That first run also produced a false positive, which is worth keeping in the record: €5 came from an unrelated rule threshold — stakes >€5K — not from a revenue claim. The pattern was tightened to require a magnitude suffix, so a rule threshold like €5K no longer reads as a revenue band, while an annual-revenue figure still does. A gate that cries wolf gets switched off within a week.

The offending paragraph was rewritten to carry one category instead of three. It kept the sector, dropped the headcount and the portfolio breakdown, and lost nothing a reader needed.

Prevention

Three rules came out of this, and all three are enforced by the build rather than by remembering:

1. Numbers about the system are free; numbers about the company are banded, and only one band per artifact. Runs, sessions, tokens, failure rates, latency — publish all of it. Revenue, headcount, sector, geography — at most one.
2. A ratio is safer than an absolute and usually more interesting. "About 12% of decisions with a financial impact touch the agent" carries the insight. The revenue figure carries the identity and no insight at all.
3. A sign-off names the check it ran. Not anonymization verified but deny-list: 0 hits; cross-check: 1 category. The second form cannot quietly grow into a promise nobody made.

The deny-list stays. It was not wrong, it was narrow — and the failure was believing a narrow check was a broad one.