StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectors

AI summary

Overview

• Research area: Adversarial robustness of AI-generated-text detectors; reinforcement learning for text generation (LLM paraphrasing); AI content provenance.
• Technical level: Intermediate. The paper assumes familiarity with detectors, AUROC/TPR metrics, and RL fine-tuning concepts such as GRPO and LoRA, but explains each component.
• Scope: A single paper introducing StealthRL, an RL-trained paraphrase policy on Qwen3-4B-Instruct that is trained against a two-detector ensemble and then evaluated against four detectors (plus held-out transfer targets) on the filtered MAGE test pool of 15,310 human and 14,656 AI samples.

What This Paper Is About

AI-text detectors are usually tested on "clean" AI text with no evasion attempt, but real adversaries can iteratively rewrite text until it slips past detection. The authors build StealthRL, a reinforcement learning system that trains a language model to paraphrase AI-generated text so that it keeps its meaning but is classified as human by detectors. The goal is to measure how badly current detectors fail under this kind of adaptive, semantics-preserving attack, and to release a reproducible protocol for stress-testing them.

Key Contributions

1. Multi-detector RL attack: StealthRL trains a paraphrase policy with Group Relative Policy Optimization (GRPO) and LoRA adapters on Qwen3-4B-Instruct against a two-detector ensemble (RoBERTa at 0.6 weight, Fast-DetectGPT at 0.4 weight), with a composite reward balancing detector evasion and E5-embedding semantic similarity. The authors describe it as one of the first works to apply RL for adversarial detector evasion across multiple detector families simultaneously.
2. Demonstration of catastrophic robustness failure: A mean TPR@1%FPR of 0.024 across four detectors, with strong transfer to two held-out detectors (Binoculars and MAGE) not seen during training.
3. Comprehensive analysis: Detector score distributions explaining why evasion succeeds, an LLM-judge Likert quality evaluation on 500 matched samples per method, and per-detector AUROC with 95% bootstrap confidence intervals (500 iterations, seed 42).
4. Released evaluation protocol and artifacts: A protocol measuring evasion, transfer, and fidelity at security-relevant operating points, with code at github.com/suraj-ranganath/StealthRL and a model checkpoint at huggingface.co/suraj-ranganath/StealthRL.

Main Findings

• Near-total evasion at strict operating points: StealthRL (M2) reaches a 0.024 mean TPR@1%FPR and a 97.6% attack success rate, reducing mean AUROC from 0.79 (no attack, M0) to 0.43.
• Three of four detectors collapse: At 1% FPR, StealthRL achieves 0.002 TPR on Binoculars, 0.002 on Fast-DetectGPT, and 0.002 on RoBERTa; the MAGE detector is the most robust at 0.089.
• Attacks transfer to unseen detectors: Binoculars and MAGE were held out during training, yet StealthRL hits 0.002 TPR@1%FPR on Binoculars and 0.089 on MAGE, which the authors read as evidence of shared architectural vulnerabilities rather than detector-specific brittleness.
• AUROC vs. TPR@1%FPR diverge on RoBERTa: RoBERTa's AUROC under attack is 0.691 while its TPR@1%FPR is 0.002. The authors explain that AUROC measures global rank separability while the strict-threshold metric measures one operating point, and that the policy shifts scores leftward toward the decision boundary rather than fully collapsing discriminability.
• Per-detector AUROC under attack: Binoculars 0.055, Fast-DetectGPT 0.089, RoBERTa 0.691, MAGE 0.891; the MAGE detector remains the strongest held-out target.
• Supplementary 5% FPR results: Mean TPR@5%FPR of 0.076 for M2 (Binoculars 0.005, Fast-DetectGPT 0.003, RoBERTa 0.022, MAGE 0.272), versus 0.566 for no attack.
• Evasion costs text fidelity: StealthRL's E5 similarity is 0.901 versus 0.974 for simple paraphrasing (M1) and 0.973 for adversarial paraphrasing (M3); its mean LLM-judged quality is 2.514 versus 3.780 and 3.770. It also has a much higher perplexity of 148.686 compared to 26.662 for unmodified text.
• Learned paraphrasing beats character-level obfuscation on quality: Homoglyph substitution (M5, SilverSpeak) achieves comparable evasion (mean ASR 0.972 vs. StealthRL's 0.976) but lower judged linguistic quality (1.972 vs. 2.514), while scoring higher on judged similarity (2.920 vs. 2.644).
• Single-shot attack requirement: The reported attack uses one policy generation per input with no iterative refinement and no detector queries at evaluation time; detector access is needed only during offline training.

Methodology in Plain English

The authors start with 10,000 AI-generated samples from the MAGE training split — no human-written samples are used during training. A base model, Qwen3-4B-Instruct, is given LoRA adapters (rank 32, alpha 32, dropout 0.05) and trained with GRPO for 3 epochs, batch size 16, group size 8, and learning rate 2.8 × 10⁻⁴. For each input the model generates 8 candidate paraphrases. Each candidate is scored by a reward that combines two things: how strongly it evades the detector ensemble (1 minus the weighted average of RoBERTa and Fast-DetectGPT confidence, with weights 0.6 and 0.4), and how well it preserves meaning (cosine similarity of E5 embeddings between original and paraphrase, weight 0.1 against a detector weight of 1.0). A KL penalty of 0.05 against the frozen reference policy keeps outputs fluent. Advantages are computed by group-relative normalization, then the policy is updated with a clipped policy gradient.

At test time the policy generates a single paraphrase per input using temperature 1.0, top-p 0.9, and a 512-token limit, with the prompt "Paraphrase the following text while preserving its meaning: [TEXT]". No reranking or candidate selection is applied. Evaluation uses the full filtered MAGE test pool of 15,310 human and 14,656 AI samples (100–500 token filter). Detection thresholds are set from the 99th percentile of detector scores on the human samples so that the false positive rate is 1%, and that threshold is held fixed for AI and attacked samples. Six attack settings are compared: M0 (no attack), M1 (simple paraphrase), M2 (StealthRL), M3 (detector-guided adversarial paraphrasing of Cheng et al.), M4 (AuthorMist, RL against a single RoBERTa detector), and M5 (homoglyph substitution). A separate LLM judge, gpt-5-nano, rates 500 matched samples per method (M1–M5) on 1–5 Likert scales for linguistic quality and semantic similarity.

Why This Matters

The paper argues that current AI-text detectors learn brittle surface statistics — token frequency distributions, perplexity patterns, embedding geometry — rather than deeper linguistic understanding, and that meaning-preserving paraphrasing is enough to defeat them. Because the attack transfers to detectors never seen in training, the authors conclude that ensemble defenses offer limited robustness gains. The evasion is achievable with modest compute: a single LoRA fine-tuning run.

Real-world applications affected:

• Academic integrity systems: Institutions that rely on detector scores to flag student work could face false accusations from human writers at low FPR settings, while adversarial students evade detection entirely.
• Publishing and editorial screening: Journals and platforms using AI-text detection for provenance checks cannot rely on these tools against a motivated author.
• Content moderation and misinformation: Actors generating synthetic content at scale can modify outputs to bypass moderation pipelines.
• Detector development and benchmarking: The released code and protocol give detector builders a security-relevant stress test at the 1% FPR operating point, where deployment decisions actually happen.

Industry relevance: the findings affect any vendor or platform deploying AI-text detection, and they push the field toward adversarial training and semantic-aware detection rather than clean-distribution accuracy.

Future Directions

• Adversarial training for detectors: Fine-tuning detectors on adversarially generated examples, which the authors note faces a scalability problem because the space of paraphrasing strategies is vast.
• Semantic-aware detection: Building detectors that rely on deeper linguistic features instead of surface-level statistical cues.
• Provable robustness: Establishing theoretical guarantees on detector robustness under bounded perturbations.
• Improving StealthRL's fidelity: Closing the quality gap (E5 similarity 0.901, Likert quality 2.514) via constrained decoding, rejection sampling, or multi-objective Pareto-optimal RL.
• Broader evaluation: Extending beyond MAGE to datasets such as RAID, additional languages, and watermark-based detectors, which the authors explicitly did not evaluate.

Target Audience

Detector researchers and engineers building or deploying AI-text detection systems; adversarial machine learning researchers studying evasion and robustness; AI safety and content-provenance teams; and policy or integrity officers who need to understand the limits of detection tools before deploying them in high-stakes settings. Readers should be comfortable with detection metrics such as AUROC and TPR at fixed false positive rates, and with the basics of reinforcement learning fine-tuning.

Authors’ abstract

AI-text detectors face a critical robustness challenge: adversarial paraphrasing attacks that preserve semantics while evading detection. We introduce StealthRL, a reinforcement learning framework that stress-tests detector robustness under realistic adversarial conditions. StealthRL trains a paraphrase policy against a multi-detector ensemble using Group Relative Policy Optimization (GRPO) with LoRA adapters on Qwen3-4B, optimizing a composite reward that balances detector evasion with semantic preservation. We evaluate six attack settings (M0-M5) on the full filtered MAGE test pool (15,310 human / 14,656 AI) against four detectors: RoBERTa, Fast-DetectGPT, Binoculars, and MAGE. StealthRL achieves near-zero detection on three of the four detectors and a 0.024 mean TPR@1%FPR, reducing mean AUROC from 0.79 to 0.43 and attaining a 97.6% attack success rate. Critically, attacks transfer to two held-out detectors not seen during training, revealing shared architectural vulnerabilities rather than detector-specific brittleness. We additionally conduct LLM-based quality evaluation via Likert scoring on 500 matched samples per method, analyze detector score distributions to explain why evasion succeeds, and provide per-detector AUROC with bootstrap confidence intervals. Our results expose significant robustness gaps in current AI-text detection and establish StealthRL as a principled adversarial evaluation protocol. Code and evaluation pipeline are publicly available at https://github.com/suraj-ranganath/StealthRL.