Key takeaways

  • AI detectors use trained classifiers, statistical tests of how predictable text is, or watermarks that only the model's maker can check.
  • Paraphrasing cut DetectGPT's detection rate from 70.3% to 4.6% at a 1% false positive rate, and seven detectors flagged 61.22% of non-native English essays as AI on average.
  • A Cornell Tech and MIT paper reports that detectors which adapt to batches of new text caught 93.3% of disguised AI text, against 27.0% for Pangram.

AI detectors work in one of three ways. They measure how predictable a text is, run it through a classifier trained on human and AI writing, or check for a hidden watermark the AI company added. None does the job reliably on its own. The papers behind them show high lab scores and steep drops once text is rewritten or comes from a model the detector has not seen.

That gap is the subject of a paper from Cornell Tech and MIT that MIT News covered on November 18, 2026. In the paper, Kevin Ren, Manish Raghavan and Nikhil Garg found that the commercial detector Pangram caught 27.0% of AI text that had been rewritten to look human. Their own method caught 93.3%.

How AI detectors work: three approaches

Trained classifiers. The most common kind. A company labels large sets of human and AI writing and trains a model to tell them apart. OpenAI's own AI classifier from January 2023 worked this way, and so do most commercial tools. Its weakness is built in: it only knows the AI text it was trained on.

Zero-shot statistical tests. These need no training set of AI text. They ask a language model how likely the text is, word by word. The common measure is perplexity, which is how surprised a model is by each next word. AI text tends to be predictable, so it tends to score low.

  • DetectGPT, from Eric Mitchell, Chelsea Finn and colleagues in January 2023, goes one step further. It makes small random edits to a passage and checks how much the model's probability for it falls. AI text sits near a peak of probability, so almost any edit makes it less likely. Human text does not show that pattern as strongly.
  • Binoculars, from Abhimanyu Hans, Tom Goldstein and colleagues in January 2024, compares two closely related models (Falcon-7B and Falcon-7B-Instruct by default). It divides the text's perplexity by how surprised one model is by the other's predictions. That corrects for odd prompts, which make even AI text surprising to read.

Watermarks. Here the AI company changes how its model picks words so the output carries a statistical signature, and later checks for it. It only works for that company's own model. Google's SynthID is the best-known example; our explainer on AI watermarking covers how it works.

How accurate AI detectors are, in the papers

The headline numbers look strong:

Detector Type Result reported by its authors
DetectGPT Zero-shot 0.95 AUROC on fake news from the 20B-parameter GPT-NeoX, vs 0.81 for the best earlier zero-shot method
Binoculars Zero-shot Over 90% of ChatGPT and other LLM text caught at a 0.01% false positive rate
Pangram 3.3.2 Trained classifier 98% balanced accuracy on plain AI rewrites of arXiv abstracts (tested by Ren et al.)
OpenAI AI classifier Trained classifier 26% of AI text caught, 9% of human text wrongly flagged

AUROC runs from 0.5 (a coin flip) to 1.0 (perfect).

OpenAI's numbers are the odd ones out, and OpenAI acted on them. On July 20, 2023, it added a note to the announcement saying the classifier was no longer available because of its low accuracy, as TechCrunch reported.

Why AI detectors fail

Lab numbers hold only while new text looks like the test set. Four things break them.

Rewriting. In a March 2023 paper, Kalpesh Krishna and colleagues ran AI text through an 11-billion-parameter paraphrasing model. DetectGPT's detection rate fell from 70.3% to 4.6%, with the false positive rate held at 1%. A separate team led by Soheil Feizi showed that repeated paraphrasing cut detection for watermarks, trained classifiers, zero-shot tests and retrieval systems alike. Ren et al. saw the same with Pangram: its balanced accuracy fell from 98% to 63.5% on text rewritten by prompts tuned against it.

New models. A classifier trained on one model's writing struggles with another's. In Ren et al.'s tests, a detector trained on Llama 3.3 70B text caught 55% of text from gpt-oss-120b. Pangram, before an update, labeled 34.0% of GPT 5.4 abstracts as AI or AI-assisted. Pangram later reported 99.5% recall on GPT 5.4, the authors note: detectors catch up after each release, then fall behind again.

Unusual human writing. Low perplexity is not proof of AI. Weixin Liang, James Zou and colleagues tested seven popular detectors in March 2023 on 91 TOEFL essays by non-native English speakers and 88 essays by US eighth-graders. The detectors were near perfect on the eighth-graders but flagged 61.22% of the TOEFL essays as AI on average. All seven agreed on 18 of the 91, and 89 of 91 were flagged by at least one. Simpler word choices mean lower perplexity. When the researchers simplified the eighth-graders' word choices, the average false positive rate rose from 5.19% to 56.65%. The Binoculars authors later reported 99.67% accuracy on essays by non-native speakers, so the size of this bias depends on the detector.

Human writing changes too. Ren et al. trained detectors on 2010 arXiv abstracts and tested them on later years. A standard classifier's accuracy on human text fell from 85.9% to 73.4% by 2020, because the way people wrote abstracts had drifted.

There is also a ceiling. The Feizi team's paper proves that the best possible detector's AUROC is tied to how different the human and AI text distributions are. As models write more like people, the most any detector can achieve drops toward a coin flip.

What the Cornell and MIT team changed

The team's fix is called test-time adaptation. A normal detector is frozen after training and judges each text alone. Their detector looks at a batch of new, unlabeled texts and retrains its decision boundary on them, using the fact that AI text from one source tends to look alike. The code is on GitHub.

Results, from the paper:

  • On rewritten AI text, after three rounds of an arms race between rewriter and detector, the adapted detector reached 94.4% balanced accuracy and caught 99% of AI text. A standard classifier reached 69.3% and caught 49.2%.
  • On human text from 2020, the adapted detector stayed at 89.8% to 85.9% accuracy, depending on the variant, against 73.4% for the standard one.

The authors are frank about the limits. Adaptation needs batches of new text, so it suits platforms that see many submissions, not one teacher checking one essay. It needs a supply of verified human writing, which gets harder to find after 2022. And they did not use it to label individual texts, only to estimate how much AI writing a collection contains, because a wrong accusation does real harm. "It isn't entirely clear if this will be a solvable problem," Raghavan told MIT News.

What this means for anyone reading a detector score

A detector score is a guess about whether a text resembles its training data, not proof of who wrote it. The papers agree that detectors work best on long, unedited text from a known model, and worst on short, edited or unusual writing, which is often the text people most want checked. The Cornell and MIT paper will be presented at NeurIPS, the main machine learning conference.

Frequently asked questions

Are AI detectors accurate?

On unedited text from models they know, the best ones report over 90% detection at very low false positive rates. On paraphrased text or text from new models, rates fall sharply: one study saw DetectGPT drop from 70.3% to 4.6%, and OpenAI withdrew its own classifier in July 2023 over low accuracy.

Can AI detectors flag human writing as AI?

Yes. A 2023 study of seven popular detectors found they flagged 61.22% of TOEFL essays by non-native English speakers as AI on average, while being near perfect on essays by US eighth-graders. OpenAI's classifier wrongly flagged 9% of human text.

What is perplexity in AI detection?

Perplexity measures how surprised a language model is by each next word in a text. AI text tends to be predictable and score low, but plain human writing with simple vocabulary can score low too, which causes false positives.

How does AI watermark detection work?

The AI company changes how its model picks words so the output carries a hidden statistical signature, then checks text for it. It only works for that company's own model and only if the company offers the check.

Sources

  1. Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution ShiftarXiv (Ren, Raghavan, Garg)
  2. Helping AI text detectors keep up with changing conditionsMIT News
  3. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability CurvaturearXiv (Mitchell et al.)
  4. Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated TextarXiv (Hans et al.)
  5. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensearXiv (Krishna et al.)
  6. Can AI-Generated Text be Reliably Detected?arXiv (Sadasivan et al.)
  7. GPT detectors are biased against non-native English writersarXiv (Liang et al.)
  8. New AI classifier for indicating AI-written textOpenAI
  9. New AI classifier for indicating AI-written text (forum announcement)OpenAI Developer Community
  10. OpenAI scuttles AI-written text detector over 'low rate of accuracy'TechCrunch
  11. llm_detection code repositoryGitHub

Add ClaudeGPT Si as a preferred source on Google