The Misclassification of Autistic Writing as AI-Generated

AI summary

Overview

Research area: Natural Language Processing, specifically AI-text detection models and algorithmic bias against disabled writers.

Technical level: Intermediate. Readers need some familiarity with logistic regression, odds ratios, and detector-related concepts such as perplexity, burstiness, and tokenization, but the paper explains these terms in plain language.

Scope: The paper empirically tests whether OpenAI's GPT-2 detection model flags Reddit posts from autism-focused communities as AI-generated more often than posts from general subreddits, using a corpus of roughly 60,000 posts and two logistic regression experiments.

What This Paper Is About

AI-detection tools are widely used in schools and publishing despite evidence that they are inaccurate and biased against groups such as non-native English speakers. Autistic writers have long claimed anecdotally that their writing is disproportionately flagged as AI-generated, but this claim had not been formally tested. The authors build a Reddit corpus of "likely-autistic" and "general-Reddit" writing and measure the GPT-2 detector's false-positive rates across the two groups.

Key Contributions

1. First empirical test of the autism false-positive claim. The authors convert a widely repeated anecdotal claim about autistic writers into a testable hypothesis and examine it against a large, real-world corpus rather than a small set of personal writing samples.

2. A large, two-group Reddit corpus construction pipeline. They combine PRAW-collected posts (approximately 2021–2024) with a Pushshift archive (2010–2020) drawn from 13 autism-related subreddits and 12 general subreddits, yielding around 60,000 posts, with filtering to remove autism keywords from the general group, one post per user, and a minimum of 1000 characters.

3. Demonstration of a word-count confound and a follow-up experiment that neutralizes it. Recognizing that the likely-autistic subcorpus had a lower mean word count, they run a second experiment truncating every post to exactly 300 words, which strengthens the observed effect instead of removing it.

4. Comparison of textual features against the assumed profile of AI text. They compute perplexity, burstiness, mean word length, and mean sentence length for both subcorpora and find that the autism-linked writing does not match the low-perplexity, low-burstiness profile usually attributed to AI-generated text.

Main Findings

• The detector flags few posts overall, but unevenly. In Experiment 1, 1.7% of all 59,947 posts were flagged as AI-generated. The likely-autistic group had 1.9% flagged, versus 1.5% for general-Reddit.

• Subcorpus is a significant predictor. A logistic regression with the model's AI-probability score as the outcome showed a significant effect of subcorpus (estimate 0.220, standard error 0.067, z = 3.305, odds ratio 1.246, p = 0.001). Based on the odds ratio, posts from the likely-autistic subcorpus had a 25% greater chance of being classified as AI-generated.

• Perplexity and word count also mattered in Experiment 1. Perplexity had a significant negative effect (estimate −0.232, odds ratio 0.793, p < 0.001), meaning higher-perplexity posts were less likely to be flagged. Word count also had a significant negative effect (estimate −0.439, odds ratio 0.644, p < 0.001). Date, mean word length, mean sentence length, and burstiness were not significant in this model.

• The effect grows when word count is controlled. In Experiment 2, restricting to posts longer than 300 words and truncating each to exactly 300 words left approximately 33,000 of the original approximately 60,000 posts. Of the 33,216 total posts, 1.4% were flagged: 1.7% of likely-autistic posts and 1.2% of general-Reddit posts.

• The odds ratio rises to 1.501 in Experiment 2. Subcorpus remained significant (estimate 0.406, standard error 0.096, z = 4.226, p < 0.001), meaning likely-autistic posts had a 50% greater chance of being classified as AI-generated.

• Date becomes significant once word count is normalized. The binary post-2020 indicator had a small but significant negative effect (estimate −0.206, odds ratio 0.814, p = 0.030), indicating posts written in or after 2020 were less likely to be flagged than older posts. The authors suggest this may reflect the GPT-2 detector failing to catch text written by newer LLMs such as ChatGPT, producing false negatives.

• Perplexity and burstiness lose significance in Experiment 2. Perplexity (estimate 0.042, odds ratio 1.043, p = 0.335) and burstiness (estimate 0.010, odds ratio 1.010, p = 0.816) were not significant, as were mean word length and mean sentence length.

• The textual-feature assumptions did not hold. The general-Reddit subcorpus had a much larger mean word count. Mean word length and mean sentence length were slightly higher in the likely-autistic subcorpus, and perplexity and burstiness were close to equal, with the likely-autistic subcorpus trending only slightly higher in both. This contradicts the expectation that autistic writing would show the low perplexity and low burstiness associated with AI text.

• One other detection tool returned identical output. A second freely available AI-detection tool produced the exact same predictions and probability scores as the OpenAI model for the same texts, rounded to the hundredth decimal place, raising the possibility that other products use the OpenAI model on the back end.

Methodology in Plain English

The researchers needed two comparable sets of writing: one very likely written by autistic people, and one representing ordinary Reddit writing. They chose 13 subreddits devoted to autism topics and 12 popular subreddits with a similar style (long, first-person narratives about mental health, social situations, and requests for advice). Because nobody can be certain of any Reddit author's neurotype, they label the groups "likely-autistic" and "general-Reddit" rather than "autistic" and "non-autistic."

To reduce contamination, they removed posts from the general group whose authors also appeared in the autism group or whose text contained autism keywords, and they filtered out posts containing autism keywords alongside phrases like "my daughter" or "my nephew." They kept only one randomly chosen post per user and discarded posts under 1000 characters, the minimum length OpenAI claims is needed for an accurate prediction.

Each post was tokenized with the RoBERTa AutoTokenizer and truncated to 480 tokens, then run through OpenAI's freely downloadable RoBERTa GPT-2 detector, which returns a binary label and a probability score. The authors treated probabilities above 0.5 as a "Fake" (AI-generated) prediction.

For statistics, they fit a logistic regression in R using the glm function with a binomial distribution, no random effects, and no interaction terms. The outcome was the detector's AI-probability score, and the predictors were subcorpus, mean sentence length, mean word length, perplexity, burstiness, and a binary indicator for whether the post was written after January 1, 2020. Perplexity was computed with HuggingFace's evaluate package and burstiness as the coefficient of variation of sentence lengths, modeled after the zippy package. A second experiment repeated the whole procedure on posts truncated to exactly 300 words to neutralize the word-count difference between groups.

Why This Matters

Impact on research. The study provides quantitative evidence for a bias that had only been reported anecdotally. It also shows that a bias signal survives in a deliberately noisy dataset, leading the authors to suspect the real-world difference between autistic and non-autistic writers may be larger than what they measured. The finding that feature-based explanations do not easily account for the bias suggests the detector is picking up on something other than the features researchers usually assume.

Real-world applications:

• Education and academic integrity. False accusations of AI use can damage a student's academic standing and psychological well-being. The authors cite Chaka's argument that any AI content probability percentage, however negligible, inflicts reputational damage on the essay and the student.

• Employment. Autistic people already face extremely high rates of unemployment, and reputational damage or lost educational opportunities from false accusations could worsen employment outcomes.

• Admissions and hiring screening. The paper notes that Ivy League admissions officers reportedly use automated AI-detection models alongside their own judgment, and that one officer's criteria for spotting AI text echoed descriptions of autistic narrative styles.

• Legal and institutional risk. Schools and companies using biased detection technology could expose themselves to ableism and other discrimination lawsuits.

Industry relevance. Detection tools such as CopyLeaks, Turnitin, GoWinston, and GPTZero advertise largely unverified accuracy rates up to 99.6%, and a 2024 survey reported that two thirds of American teachers used AI detectors regularly. The paper also warns that some commercial detectors may be running the same OpenAI model underneath and inheriting its bias. Even if automated detectors fall out of fashion, the authors note that humans are often worse than automated models at identifying AI-generated content, so the underlying problem may persist through human judgment alone.

Future Directions

1. Test more detection tools. The authors state it would be more informative to gather predictions from more than one AI-detection tool, as several of the referenced experiments did, and to investigate whether other commercial products share the same back-end model.

2. Use controlled corpora with verified diagnoses. A more controlled experiment would verify participants' results of autism evaluations when designating the two comparison groups, and would match subcorpora more objectively by topic rather than only by style and format.

3. Analyze lexical items and constructions. The authors suggest that examining the distribution of frequent lexical items and constructions in both subcorpora could explain why they did not observe lower perplexity and burstiness in the likely-autistic group despite descriptions of autistic language as repetitive and clichéd.

4. Investigate the short-text threshold more deeply. Since shorter posts were flagged significantly more often even above the 1000-character minimum OpenAI recommends, the authors call for exploring this pattern in greater depth.

Target Audience

This paper is most useful to researchers in NLP and algorithmic fairness, educators and university administrators weighing AI-detection policies, academic integrity and publishing professionals, and disability studies scholars. It is also relevant to autistic writers and students who have been affected by false accusations, and to developers and auditors of AI-detection tools who want evidence of group-level false-positive disparities. Prior exposure to basic statistical modeling will help, but the paper's central claim and results are accessible without it.

Authors’ abstract

Recent findings suggest that detection models for artificial intelligence (AI) cannot accurately identify AI-generated text and may exhibit bias against certain minority groups. In the present study, anecdotal claims that autistic writers more often have their work flagged as AI-generated are examined empirically. A corpus of approximately 60,000 Reddit posts split into "likely-autistic" and "general-Reddit" subcorpora is used to compare the distribution of probabilities output by the OpenAI GPT-2 detection model. Differences in textual features between subcorpora are observed and compared to reported features of AI-generated text. Results showed that while less than two-percent of either subcorpus was flagged as AI-generated by the model, significantly more texts from the likely-autistic subcorpus were flagged. Connections between features of text with likely-autistic authors and AI-generated text were not straightforward. The widespread use of AI-detection models with a potential bias against autistic writers in their output prompts ethical scrutiny, and the authors recommend further critical examination of the models themselves as well as their use in academic contexts.