How AI content detectors actually work, and where they break

Almost every conversation about AI detection is really a conversation about a black box. Someone pastes text into a tool, a number comes back, and a decision gets made: a grade, a payment, a hiring call. Almost nobody asks what produced the number. This is what produces the number.

We build detection systems for a living. TextSight.ai is our text detector; Truthring is our voice one. That gives us an obvious commercial interest in you believing detection works. It also gives us a reason to be precise about where it does not, because the fastest way to destroy a detection product is to let people trust a score it did not earn.

So: four generations of detection method, what each one actually measures, and where each one breaks.

Generation one: perplexity and burstiness

The original idea was elegant, which is usually a warning sign. Take a reference language model. Feed it the text one token at a time and ask how surprised it is by each next word. That surprise, averaged, is perplexity. Low perplexity means the text is highly predictable. Language models are trained to produce likely text, so machine writing tends to sit in the low-perplexity band.

Burstiness was the companion metric: the variance of perplexity across sentences. Human writing lurches. A long technical clause is followed by a four-word fragment. A model, sampling from a smoothed distribution, tends to hold a steadier line. High variance suggested a human; a flat line suggested a machine.

The failure is structural and it was visible from the first month. Perplexity does not measure who wrote something. It measures how predictable the writing is. Those are wildly different properties that happen to correlate, and the correlation collapses exactly where the stakes are highest.

OpenAI launched its AI Text Classifier in January 2023 and killed it in July of the same year. The published numbers above are the reason. A tool that catches a quarter of what it is looking for while wrongly accusing one human in eleven is not a weak tool; it is a coin flip with a confidence interface attached.

GPTZero, which was built on the perplexity idea and named the concept for a general audience, has since stated plainly that it stopped using perplexity and burstiness as its detection mechanism in autumn 2023 and moved to a deep-learning architecture. Perplexity survives in its product as one indicator among several. That is the honest end state for the metric: a weak feature, not a verdict.

It matters that some tools still ship generation-one methods in 2026, usually the free ones. If a detector explains its result to you purely in terms of perplexity and burstiness, you are looking at a 2023 product with a 2026 marketing page.

Generation two: probability curvature

The next idea was sharper. Mitchell and colleagues at Stanford published DetectGPT in 2023 with a specific hypothesis: machine-generated text does not merely have low probability under the generating model, it sits at a local maximum of that model’s probability surface. Perturb it slightly — swap a few words for near-synonyms — and the probability drops noticeably. Perturb human text the same way and it barely moves, because human text was never optimised toward that surface in the first place.

This is a much better question than “is this predictable”. It asks “does this text look like it was drawn from a model’s distribution”, which is closer to the thing you actually want to know.

The problem was cost. DetectGPT needed around a hundred perturbations per document, each requiring a model pass. Fast-DetectGPT, presented at ICLR 2024 by Bao and colleagues, replaced perturbation with a conditional probability curvature computed from a single sampling step, reporting roughly a 340-fold speedup and a large accuracy gain at the same time. Binoculars, published the same year, took another route: score the text under two closely related models and use the ratio of perplexity to cross-perplexity, which normalises away the “this text is just inherently predictable” confound that broke generation one.

These methods are good, they are zero-shot, and they need no labelled training data. Their weakness is that they are still reading the statistical surface of the text. Change the surface and the signal moves.

Generation three: supervised classifiers

What actually replaced perplexity in production was boring: train a large transformer on an enormous labelled corpus of human and machine text, and let it learn whatever separates them.

This is where nearly all serious 2026 detectors sit, ours included. The engineering that matters is not the architecture (everyone is using some transformer encoder) but the data. Specifically, three things:

• Domain coverage. A detector trained on essays will misbehave on product copy, legal drafting, GitHub issues and clinical notes. Human writing conventions differ enormously by genre, and a detector that has not seen a genre treats its conventions as anomalies.
• Generator coverage. Every model family leaves a slightly different fingerprint. A detector trained before a model existed has never seen that model’s output distribution. This is the single largest source of silent decay in deployed detectors.
• Hard negative mining. The technique that separates a usable detector from an unusable one. You find the human documents your model wrongly flags, generate machine text matched to those documents on topic, length and register, and retrain on the pair. You are not teaching the model what AI looks like; you are teaching it what human writing that resembles AI looks like. That is where false positives live.

Generation four: stop guessing, start marking

The fourth approach abandons inference entirely. Instead of asking a classifier to guess, you have the generator mark its own output.

Google DeepMind’s SynthID Text does this by biasing the token sampling process: a tournament over candidate next-tokens that embeds a statistical signature detectable later with a key, without meaningfully degrading output quality. DeepMind open-sourced the text implementation through Hugging Face Transformers in October 2024, which was the first time a major watermarking scheme became available to third parties.

C2PA Content Credentials attack the same problem from the provenance side: a cryptographically signed manifest travelling with a file, recording what made it and what edited it. It is not a detector. It tells you where content came from, not whether a machine wrote it.

Both have the same two structural limits, and they are worth stating flatly because vendors rarely do.

Watermarks only cover watermarked models. A signature tells you nothing about output from any model that did not embed one, which includes every open-weights model anyone can run locally. Absence of a watermark is not evidence of human authorship, and any workflow that treats it that way is broken by design.

Text provenance does not survive copy-paste. C2PA was built around media files carrying metadata. Plain text pasted from a chat window into a document has no manifest, and that is precisely the workflow anyone would use. For images and audio, provenance is a serious answer. For text, it is currently a partial one.

The regulatory tide is nonetheless running this way. Article 50 of the EU AI Act obliges providers of generative systems to mark synthetic outputs in machine-readable form, and obliges deployers to disclose deepfakes and certain published AI text. Those transparency obligations became applicable on 2 August 2026, with a short grace window into December 2026 for marking systems already on the market. Note who is being regulated: the generators. No binding accuracy standard governs detectors themselves.

What independent testing actually found

The most-cited independent evaluation remains Weber-Wulff and colleagues in the International Journal for Educational Integrity in 2023, covering fourteen tools. Their findings set the baseline everyone still argues about:

No tool in that study exceeded 80% overall accuracy. Their conclusion, that detection tools are neither accurate nor reliable enough to serve as sole evidence, is the sentence that appears in university policy documents to this day.

The RAID benchmark, presented at ACL 2024 by Dugan and colleagues, is the other reference point: over six million generations across eleven models, eight domains and eleven adversarial attacks, against twelve detectors. Its two findings are the ones that should change how you buy.

First, detectors report excellent accuracy in-distribution and degrade sharply out of it. Second, and this is the uncomfortable one, the most damaging attack was not sophisticated paraphrasing. It was homoglyph substitution: swapping Latin characters for visually identical characters from other scripts. Several detectors lost thirty to seventy accuracy points to an attack that changes nothing a reader can see and requires no model at all.

The defence is a single Unicode normalisation pass before tokenisation. Detectors that had it were essentially unaffected. Detectors that did not were destroyed. That is not a research frontier; it is a preprocessing checkbox, and the fact that it separated the field in a 2024 benchmark tells you how uneven the engineering was.

The theoretical ceiling

There is a limit here that no amount of engineering removes, and it is worth understanding before you build a policy on top of a detector.

Sadasivan and colleagues at the University of Maryland framed it formally in work later published in TMLR: the best achievable accuracy of any detector is bounded by the total variation distance between the distribution of human text and the distribution of machine text. As models get better at producing text indistinguishable from human writing, that distance shrinks, and the bound on any detector, present or future, tightens toward chance.

The training objective of every frontier language model is, in effect, to minimise exactly that distance. Detection is therefore fighting the industry’s central optimisation target. This does not mean detection is useless today. It means detection is a depreciating asset that must be continuously re-funded, and that any long-run answer has to involve marking at source rather than inference after the fact.

Reading a score honestly

Three practical consequences follow from all of the above.

A percentage is not a probability of guilt. Most detectors output a calibrated likelihood that text resembles machine output, which is not the same as the likelihood that a specific person cheated. Turnitin, which publishes more than most, has stated a document-level false positive rate below 1% but only for documents scoring 20% AI or above, has acknowledged a sentence-level false positive rate around 4%, and since July 2024 has suppressed the display of scores below 20% entirely rather than show numbers it considers unreliable. That last decision is more informative than any accuracy claim.

Short text is not detectable. Every statistical method here needs signal, and signal accumulates with length. Detectors that will happily score a 40-word paragraph are selling you noise. Most serious tools require 200 to 300 words minimum, and confidence keeps climbing well past that.

Ask for the failure conditions, not the accuracy figure. An accuracy number without an operating threshold, a false-positive rate, a domain and an unseen-generator holdout is uninterpretable. The questions that separate real vendors from marketing: what is your false positive rate on non-native English writing, specifically; when was the model last retrained, and against which generators; do you normalise Unicode before tokenising; and what happens to your score distribution on text between 100 and 300 words.

If a vendor cannot answer those four, the number on their screen is decoration.

Where this leaves you

Detection in 2026 is a real technology with a narrowing operating envelope. It works well on long-form, unmodified output from known models in domains the detector was trained on. It degrades under editing, degrades further under paraphrasing, and can be defeated outright by trivial character-level tricks if the vendor skipped their preprocessing homework.

Used as one signal among several, alongside drafting history, domain knowledge, factual verification and human judgement, it is useful. Used as an automated verdict, it will eventually produce an accusation you cannot defend, and the person on the receiving end will be the one whose writing was simply plainer than average.

That is not an argument against detection. It is an argument for building the workflow around it properly, which is a different and considerably more tractable problem. We wrote about how the major tools compare if you want the buying view instead of the mechanics, and if you need detection wired into your own product rather than bought off a shelf, that is what we do.