Resource
How Do AI Detectors Work? Perplexity, Burstiness, and Pattern Matching Explained
A plain-English breakdown of the technology behind AI detection — from statistical metrics like perplexity and burstiness to modern pattern-matching engines — and why no single method is foolproof.
July 29, 2026 · Naturalmelo Team
Sentence-level AI detection with Human-like, Mixed, and AI-like classification
What AI Detectors Actually Measure
AI detectors do not detect "AI-ness" — there is no chemical test for whether a language model wrote something. Instead, they measure statistical properties of text and compare those properties against what we know about how large language models (LLMs) generate text. If a piece of writing has statistical properties that closely match what an LLM would produce, the detector flags it. If it has properties more typical of human writing, it passes.
This distinction matters because it reveals both what detectors can and cannot do. A detector can tell you that your text has a statistical profile similar to AI-generated content — it cannot tell you that AI actually wrote it. The difference between "this looks like AI output" and "an AI produced this" is the gap where false positives live.
There are three main approaches to AI detection, and modern tools typically combine them. Understanding each approach helps you understand both why detectors work and why they fail.
Perplexity: How "Surprised" Would an LLM Be?
Perplexity is the foundational metric behind most AI detectors, including GPTZero. The concept is simpler than the name suggests: perplexity measures how "surprised" a language model would be by each word in a sequence. If a model can predict each next word with high confidence, the text has low perplexity — it follows the most probable path. If the model is frequently surprised by unexpected word choices, the text has high perplexity.
AI-generated text tends to have low perplexity because LLMs are trained to produce the most probable next token. Given the prompt "The capital of France is," an LLM will almost always produce "Paris" — very low perplexity. Human writers, by contrast, make idiosyncratic choices: "The capital of France is, surprisingly, not the city most tourists imagine it to be." That is higher perplexity because "surprisingly" was not the most probable next token.
GPTZero popularized perplexity-based detection for education. It computes perplexity scores at the sentence, paragraph, and document levels, then uses the variation between these levels (burstiness) as an additional signal. A document where every sentence has uniformly low perplexity suggests AI generation; a document with spikes and valleys of high and low perplexity suggests human authorship.
- Low perplexity = predictable word choices = possible AI generation
- High perplexity = unexpected word choices = more likely human writing
- Perplexity is measured against a reference model — different models produce different scores for the same text
- Formal writing (academic papers, legal documents) naturally has lower perplexity regardless of authorship
Burstiness: Why Human Writing Has Spikes and Valleys
Burstiness is perplexity's essential companion metric. While perplexity measures average predictability, burstiness measures variation in predictability. Human writing is "bursty" — it alternates between complex, unpredictable sentences and simple, straightforward ones. A long, winding sentence with an unexpected metaphor might be followed by a three-word fragment. That variation is a human signature.
AI-generated text tends to be more uniform. Sentence lengths cluster around a mean. Complexity stays consistent. Each paragraph has roughly the same structure. This uniformity is not because AI cannot vary its output — it can, when prompted to — but because the default generation mode produces the most statistically probable sequence, and variation is statistically less probable.
A detector that uses burstiness looks at how much the complexity and predictability of writing change from sentence to sentence and paragraph to paragraph. A document where every sentence has roughly the same structure and complexity profile — low burstiness — raises a flag. A document with natural variation — high burstiness — passes.
Perplexity asks "how predictable is each word?" Burstiness asks "how much does predictability vary?" Together they capture both the micro (word-by-word) and macro (sentence-by-sentence) patterns that distinguish human from AI writing. But neither is definitive on its own — and both can be gamed.
- High burstiness: sentence length and complexity vary naturally — human hallmark
- Low burstiness: uniform sentence structure and complexity — AI hallmark
- Burstiness catches what perplexity misses: AI text that uses occasional rare words but maintains uniform structure
- Experienced writers who consciously vary their sentences can produce high-burstiness AI-assisted text
Pattern Matching: The Rule-Based Approach
Not all detection relies on statistical modeling. Rule-based detectors scan text for specific phrases and structures that LLMs overuse. These are the "tells" of AI writing: hollow openings ("In today's rapidly evolving digital landscape"), stacked connectors ("Moreover, Furthermore, Additionally"), overused transitions ("It is worth noting that"), and hype phrases ("revolutionary," "game-changing," "unprecedented").
Naturalmelo's hybrid detection engine uses 12 rule-based patterns (P01–P12) as its first pass. These patterns catch the templated phrasing that statistical models sometimes miss — especially the formulaic opening and closing sentences that LLMs default to. Rule-based detection is fast (milliseconds, no API call), transparent (each hit shows exactly which pattern was triggered), and language-aware (separate Chinese-language patterns exist for zh-CN text).
The limitation of rule-based detection is coverage. It catches the most common AI patterns but cannot catch everything — an LLM prompted to "avoid clichés and vary your sentence openings" will produce text that triggers fewer rules. That is why hybrid systems combine rules with statistical or LLM-based review.
- P01–P05: Structural patterns — hollow openings, formulaic closings, stacked connectors, uniform paragraph length, overused transitions
- P06–P09: Phrase patterns — hype adjectives, hedging clusters, generic examples, fake precision ("studies show," "research indicates")
- P10–P12: Style patterns — passive voice clusters, nominalization chains, abstract subject sentences
- Rule-based detection is transparent: each hit names the specific pattern triggered, giving writers actionable feedback
LLM-as-Judge: Using AI to Detect AI
The third approach is using an LLM — typically a smaller, faster model — to evaluate whether text "reads like AI." This is the LLM-as-judge pattern: the detector sends the text to a model with instructions to identify AI-generated passages and explain its reasoning. The model returns structured output (sentence indices, confidence scores, pattern labels) that the detector renders as highlights.
LLM-based detection has advantages: it catches patterns that rules miss, it can consider context and coherence (not just individual sentences), and it can provide natural-language explanations for why text was flagged. The disadvantage is cost and speed — an LLM call takes 2–5 seconds and costs money, which is why hybrid systems use rules first and only call the LLM on sampled or ambiguous sentences.
Naturalmelo's hybrid engine takes exactly this approach: rules run first on every sentence, then an LLM review spot-checks the tricky cases. The merge step deduplicates overlapping hits and takes the maximum severity. This two-tier design means the checker is fast and free for the common cases (rule hits only) while still providing the depth of LLM review where it matters.
Why No Detection Method Is Perfect
Every detection approach has failure modes, and understanding them is more useful than memorizing accuracy statistics. The fundamental problem is that "AI-generated text" and "human-written text" are not distinct categories with a clear boundary — they are overlapping distributions.
Formal academic writing and AI output overlap significantly because both favor low perplexity, consistent structure, and formal register. Non-native English writing and AI output overlap because both tend toward more predictable word choices and simpler sentence structures. Heavily edited writing (multiple rounds of revision, grammar checking) and AI output overlap because editing smooths out the idiosyncrasies that distinguish human from AI text.
These overlaps are not bugs — they are consequences of what detectors actually measure. A perplexity-based detector is not "wrong" when it flags formal writing; it is correctly reporting that the text has statistical properties similar to AI output. The error is treating that statistical similarity as proof of AI generation.
A detector is a measuring tool, not a lie detector. It tells you what the statistical profile of your text looks like — not who wrote it. Treating detector output as probabilistic evidence rather than binary judgment transforms it from a threat into a useful writing feedback signal.
- Formal writing → low perplexity → AI flag risk: Academic prose, legal documents, and technical writing naturally score higher on AI detection
- ESL writing → predictable patterns → AI flag risk: Non-native speakers are flagged at roughly 2× the rate of native speakers
- Heavily edited text → smoothed idiosyncrasies → AI flag risk: Multiple rounds of editing remove the "burstiness" that distinguishes human writing
- Short text → insufficient signal → unreliable scores: Anything under 300 words lacks enough data for meaningful statistical analysis
What This Means for Writers and Students
Understanding how detectors work changes how you should think about them. Instead of asking "will this pass detection?" ask "does my writing sound like me?" — because the statistical properties detectors measure correlate with authentic, individual voice.
If your writing has natural variation in sentence length and structure (burstiness), uses concrete specific examples (not generic ones), makes unexpected but appropriate word choices (perplexity), and avoids the templated phrases that LLMs default to (rule patterns) — it will likely pass detection. Not because you gamed the system, but because you wrote like a person.
If your writing does trigger detection, the feedback is actionable: vary your sentence openings, add specific personal examples, replace generic transitions with content-specific ones, and break up uniform paragraph structures. These are good writing practices independent of AI detection — they make your writing better, not just less detectable.
The detect → fix → verify loop (check, humanize flagged sentences, re-check) operationalizes this understanding. It treats the detector not as a judge but as an editor — a tool that highlights where your writing sounds templated so you can make it sound more like you.
Quick Tips
Detectors measure statistics, not authorship. A high AI score means your text has a statistical profile similar to AI output. It does not mean AI wrote it. Understanding this distinction helps you interpret scores without panic.
Burstiness is your friend. Natural variation in sentence length, structure, and complexity is the strongest human signal. Let some sentences run long, keep others short. Your writing should breathe.
Use concrete specifics over generic statements. "A 2024 Stanford study of 1,200 essays found..." beats "Studies show..." every time. Specific details are hard for AI to generate accurately and signal authentic engagement with your topic.
Treat detection as editing feedback, not judgment. A flagged sentence is not a crime scene — it is an editing suggestion. It says "this sounds templated" not "you cheated." Rewrite it in your voice and move on.
Frequently Asked Questions
Common questions about how AI detection technology works.
Q: How accurate are AI detectors?
Accuracy varies significantly by tool, text type, and language background. Turnitin claims a 1% false positive rate at the document level; independent research suggests higher rates, especially for formal academic writing and non-native English. No detector achieves 100% accuracy, and most institutions now treat scores as probabilistic evidence rather than definitive proof. The most reliable approach combines multiple detection signals (rules + perplexity + LLM review) rather than relying on any single method.
Q: Why does my original writing get flagged by AI detectors?
Original writing can trigger AI detectors for several reasons: formal academic prose naturally has lower perplexity (more predictable word choices, as AI models are trained on academic text); non-native English writing tends toward simpler, more predictable sentence structures; heavily edited text loses the idiosyncrasies that distinguish human from AI output. A flag does not mean you used AI — it means your writing shares statistical properties with AI-generated text, which is common for certain writing styles.
Q: What is the difference between perplexity and burstiness?
Perplexity measures how "surprised" a language model would be by each word in a sequence — low perplexity means predictable word choices (AI-like), high perplexity means unexpected choices (human-like). Burstiness measures how much perplexity varies from sentence to sentence — human writing has natural spikes and valleys; AI writing tends to be more uniform. Together they capture both the micro (word-by-word) and macro (sentence-by-sentence) signatures of authorship.
Q: Can detectors identify which AI model wrote a piece of text?
Generally, no. While some detectors claim to distinguish between ChatGPT, Claude, and Gemini output, the differences between models are smaller than the differences between AI and human writing. Most detectors are designed to answer "does this look AI-generated?" not "which AI generated this?" Turnitin's model was trained primarily on GPT-3.5/4 output but detects patterns from newer models as well — the statistical signature of LLM generation is more consistent across models than most people assume.
See how your writing measures up
Run a free AI writing check to see which sentences trigger detection — and get specific, actionable feedback on how to make your writing sound more like you.
Run AI writing check