Every redaction tool says roughly the same thing: AI-powered, accurate, secure. Adjectives are cheap. So we did something a little uncomfortable instead — we built a brutal test document, ran our own engine against it, and wrote down every number, including the ones that made us wince.
This is the full scorecard.
The test
We generated a 25-page synthetic document seeded with 645 sensitive data items across 45 categories — the kind of data that actually appears in HR files, medical records, financial transfers, IT logs, and identity documents. US formats and international ones: Social Security numbers, UK National Insurance numbers, Australian Tax File Numbers, EU VAT registrations, IBANs printed the way banks actually print them, ICD-10 diagnosis codes, crypto wallets, API keys, passwords, GPS coordinates, and two photo-realistic ID portraits to test face detection.
Every item was itemised in an answer key before scanning. No grading on a curve: an item was either boxed or it wasn't.
The first run: 71.5%
Our first scan found 461 of 645 items. That's the number most vendors would quietly bury.
Here's why we're publishing it: the misses weren't 184 random failures. They were a handful of systematic gaps, each repeated across every page:
- International phone numbers were detected — but the box clipped the country code. A redaction that looks complete is worse than a clean miss, because nobody re-checks a box that's already drawn.
- 15-digit Amex numbers slipped through a pattern built for 16-digit cards.
- Passwords, API keys, VAT numbers, and non-US government IDs had no category at all.
- Written biometric identifiers ("Fingerprint Scan: MATCH_0x9A2B") were invisible to a detector that only looked at images.
- GDPR special-category data — religion, political affiliation, sexual orientation, trade-union membership, criminal records — wasn't flagged, despite being the most legally sensitive text on the page under UK and EU law.
What we fixed
We expanded the engine from 57 to 67 detection categories and rebuilt the patterns the benchmark broke. Some of the fixes are the kind of thing you only find by testing against an itemised key:
- A leading word-boundary bug meant "(770) 351-2701" was being boxed as "770) 351-2701" — the opening parenthesis left visible. The same class of bug was dropping the minus sign from GPS coordinates, which doesn't hide a location; it flips the hemisphere.
- IBANs are printed in four-character groups ("DE89 3704 0044 …"). Our pattern only knew the unspaced machine form.
- "US Passport 523741892" — a table layout with no colon after the label — defeated an anchor that required punctuation. Real forms don't punctuate for your convenience.
- A labelled partial SSN ("SSN (last 4) 1892") is four bare digits. Nothing can safely claim four bare digits — except the label sitting next to them. Now the label is the evidence.
The final score
- 645 of 645 items detected. Zero false positives.
- All 45 categories reconciled exactly — not in aggregate, category by category.
- Both ID photographs detected by the face-detection pass.
- Scan time for the 25-page document: 41 seconds.
Then we rebuilt the test to be harder: 25 pages where every page is a structurally different document type — a medical intake, a wire transfer, a deposition transcript, a server config file, a payroll stub, a visa application. No repeated layouts to learn from. That fixture scored 97.8%, and the misses it exposed became the next round of fixes.
We also ran a real 60+ page commercial lease — scanned pages, handwriting, signatures, the works. 46 of its 60+ pages needed OCR. Total scan time: 2 minutes 21 seconds, with 97.7% recall and 99.5% precision against a hand-verified key.
Detection is half the job. Here's the other half.
Finding sensitive text means nothing if the "redacted" file still contains it. Plenty of tools draw a black rectangle over text that remains selectable underneath — the digital equivalent of taping paper over a window.
Our approach: on any page carrying a redaction, the page is rasterized — the redacted content is destroyed at the pixel level, not cut out of a text structure where a mistake can leave it behind. Then every word that touched no redaction box is laid back over the image as an invisible text layer, so the document stays fully searchable while the redacted words were never written into the file in any form.
We verify this forensically, not visually. After processing our benchmark document, we inspected the actual output file:
- The redacted values exist nowhere — not in any page's text layer, and not in the file's raw bytes. We searched for all 645 of them directly.
- The rest of the document still searches, selects, and copy-pastes normally, including on redacted pages.
- Metadata scrubbed: title, author, producer, creator all blank; the timestamp pinned to zero so the file doesn't even reveal when it was processed.
- Annotations stripped. We planted a fake reviewer comment in a test document specifically to prove it gets destroyed. It did — gone from the file structure and from the bytes.
Pages with nothing to redact keep their original text untouched — so a 60+ page lease with redactions on 34 pages stays a usable document instead of becoming a photo album.
Scanned documents get the same treatment, with a HUGE twist that doubles as a safety proof: after the redaction bars are burned in, the scanned page is OCR'd from the redacted image and given an invisible text layer. Your fax or phone photo comes back as a searchable document (THIS IS BIG) — and the redacted content can't leak into that layer for a simple physical reason: by the time recognition runs, the bars are opaque pixels and there is nothing there to read.
Then we destroy the original
Here's the part we think matters most, and the part we've seen no one else commit to: when you process a document, your unredacted original is destroyed. Not archived, not retained for our convenience — replaced in storage by the redacted output, permanently.
Your audit certificate states it in writing: the unredacted source no longer exists to leak. If our storage were breached the day after your redaction, there would be nothing sensitive to find.
(And no — this doesn't strand you if the scan missed something. Anything missed is, by definition, still visible in the redacted output, so you can run additional redaction rounds on it free of charge. What's protected is everything you already removed.)
What we're still honest about
- These benchmarks are synthetic. Real documents are messier, and that's why every scan goes through a mandatory human review step — you see every detection on the page and approve, reject, or draw your own boxes before anything is permanent.
- AI detection is not perfectly deterministic. Two scans of the same document can differ by a handful of items. The review step, not the model, is what makes the output final.
- A benchmark we built can't grade itself forever. We re-run it on every engine change, and when it stops finding problems, we'll build a harder one. That's the point.
Try it on your own worst document
The best benchmark is the document you'd least want leaked. Upload it, scan it, and count what we catch — the first pages are on us.