Do AI detectors work?
The vendor figure and the classroom experience are both real. The reason they feel irreconcilable is that a very small error rate, applied to a very large number of essays, is still a lot of accused students.
Key figures
Turnitin has claimed a false-positive rate under 1% for its AI writing detection.
Vanderbilt worked out what that meant at its own scale: about 750 of 75,000 papers a year wrongly flagged, and switched the detector off.
Independent research found detectors systematically misclassify non-native English writing as AI-generated.
What the vendor claims
Turnitin's published position at launch was roughly 98% accuracy with a false-positive rate below 1%, for documents where more than a fifth of the text is AI-generated. In June 2023 the company acknowledged that the real false-positive rate was higher than it had originally stated.
Two things about that claim are worth holding on to. It comes from the vendor's own testing, not from independent peer review. And it is conditioned on a threshold, which means it does not describe the case people actually worry about: an essay with a sentence or two that reads oddly.
Why 1% is not reassuring
Vanderbilt did the multiplication on its own submissions: about 75,000 papers a year, and at the stated 1% false-positive rate roughly 750 students could be incorrectly flagged as having used AI. Annually. At one university.
This is the whole problem with reading an accuracy figure as reassurance. A 1% error rate is excellent as a statistic and intolerable as a disciplinary process, because the cost of a false positive is not one percent of a student's academic record.
Who gets wrongly accused
The errors are not spread evenly. Researchers at Stanford tested detectors on writing by non-native English speakers and found they consistently misclassify non-native English writing as AI-generated, while correctly identifying native writing.
The mechanism is uncomfortable but simple. Detectors key on the statistical texture of text: how predictable the next word is, how varied the sentence structure is, how idiomatic the vocabulary is. Someone writing carefully in a second language produces exactly the flatter, more regular, less idiomatic prose the detector reads as machine-written.
So the tool does not merely make mistakes. It makes them disproportionately against international students, who are also the least placed to contest an accusation.
What universities did about it
Vanderbilt disabled Turnitin's AI detector in August 2023 and published its reasoning. Alongside the arithmetic above it cited the bias against non-native speakers, the absence of any detailed explanation of how detection works, and the difficulty of the underlying task, concluding that AI detection is already very hard for technology to solve, if it is solvable at all.
It was not an isolated decision. A number of institutions across several countries have since disabled, restricted or moved away from AI writing detection, and several detector companies have changed business model or shut down.
If you have been accused
The single most useful fact to know is that a detector score is not evidence in the way a plagiarism match is. A plagiarism match points at a specific source document that can be examined. A detector score points at nothing: it is a probability produced by an undisclosed method that its own vendor has conceded is less accurate than first claimed, and that independent research shows is biased by the writer's first language.
Version history, drafts and notes are the practical answer, because they show process rather than arguing about output. Keep them.
Sources and method
- GPT detectors are biased against non-native English writers, Liang, Yuksekgonul, Mao, Wu and Zou, Stanford, published in Patterns, 2023. Source of the finding that detectors consistently misclassify non-native English writing as AI-generated.
- Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector, Vanderbilt University, August 2023. Source of the 75,000 submissions and roughly 750 wrongly flagged papers, and of the reasoning for disabling the detector.
- SynthID, Google DeepMind. The alternative approach of watermarking at the point of generation, which yields a positive signal rather than a probability estimate.
- Turnitin's accuracy and false-positive claims are the company's own published figures from its launch materials, together with its June 2023 acknowledgement that the false-positive rate was higher than first stated.
- Every source here was opened and checked on 25 September 2026. Where two credible sources disagree, both are shown with their scope, rather than averaged into one number.