Verdah · accuracy audit · 100 posts

It catches 42% of AI posts and has never wrongly accused a person.

I labelled 100 LinkedIn posts by hand — 50 written by a model, 50 written by a person — and ran them through my own product. Every detector in this category publishes a percentage. Almost none publish the set it was measured on.

The confusion matrix

Rows are what the post actually was. Columns are what Verdah said.
Actually	Said AI	Said Human	Said Mixed	Total
AI	21	28	1	50
Human	0	50	0	50

71 exact matches out of 100. The errors are all in one direction: it lets AI through, and it does not accuse people. That asymmetry is deliberate and it is the whole design.

What it does badly

56%of AI posts called human

42%of AI posts flagged

0Mixed verdicts on human posts

Twenty-eight of fifty AI posts came back as human. If a machine-written post is specific — real names, real numbers, an unflattering detail — this will miss it, and that is the most common way it fails. Treat a Human verdict as "no strong signal", not as a clearance.

What it does well

0%false accusation rate

100%precision when it says AI

21/21AI calls that were correct

Zero human posts out of fifty were called AI. Every one of the 21 AI calls was correct. On a set this size that is not proof of a perfect false-positive rate — it is one hundred posts, and the honest reading is "no false accusations were observed", not "none can happen".

What retuning can and cannot buy

The verdict comes from a 0–100 score and two thresholds. Sweeping every threshold pair against this same set shows detection is flat — there is no setting that catches substantially more AI without starting to accuse people:

Same 100 posts, same model, thresholds swept across the full range.
Thresholds	AI caught	Precision	False accusations	Exact
33 / 32 — current	21/50 · 42%	21/21 · 100%	0/50 · 0%	71/100
30 / 29 — more aggressive	22/50 · 44%	22/23 · 96%	1/50 · 2%	71/100
53 / 33 — more cautious	17/50 · 34%	17/17 · 100%	0/50 · 0%	67/100

Two more percentage points of detection costs the first false accusation. That trade is not worth making. Detection improves by changing what the model looks for, not by moving a number, and the next version of the prompt is where that work belongs.

What I will not claim

Any accuracy number other than the ones on this page. The set is 100 posts. It is not big enough to extrapolate from and I am not going to.
That this "detects AI content." It detects some AI content, and the numbers above say how much.
That the labels are beyond dispute. I labelled them. Where a post was genuinely ambiguous I had to make a call, and a different person would call a few of them differently.
That 0/50 means zero. It means none were observed in fifty. A larger set will find some.

Method

100 LinkedIn posts, 50 generated and 50 written by people, labelled before scoring — not after.
Each post scored once through the production path, the same one a paying user hits.
Output is a verdict (AI / Human / Mixed) and a confidence band (High / Medium / Low).
No post was excluded after seeing its result.
Scoring includes an explicit instruction not to treat careful, formal, or non-idiomatic English as evidence of machine writing. AI detectors have a documented bias against non-native writers; on this set, structure and specificity decide the verdict, not fluency.

The raw per-post results are available on request while I decide how to publish them without republishing other people's posts.