DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning

AI summary

Overview

Research area: Natural Language Processing, specifically AI-generated text detection and representation learning for human–AI collaborative ("hybrid") text.

Technical level: Intermediate. The paper assumes familiarity with contrastive learning, embedding spaces, K-nearest-neighbor retrieval, and hierarchical clustering, though the core ideas are explained in accessible terms.

Scope: The paper proposes a tree-structured representation learning framework (DETree) plus a large hybrid-text benchmark (RealBench) for detecting how, and how much, AI was involved in producing a piece of text.

What This Paper Is About

Most AI-text detectors only answer a binary question: was this written by a human or a machine? But real writing often mixes both, such as a human draft polished by an LLM, an LLM output revised by a person, or text passed through several different models. The paper argues that these different collaboration processes leave distinguishable traces, and that those traces have a natural hierarchical structure: some processes are more similar to each other than others. DETree's goal is to learn text representations that respect that structure, so a detector can identify not just whether AI was involved, but how it was involved, and generalize to new, unseen LLMs.

Key Contributions

1. A representation-learning classification paradigm based on a Hierarchical Affinity Tree (HAT). Instead of treating every generation process as a flat category, DETree builds a tree from an inter-class similarity matrix, where leaf nodes are specific categories and internal nodes capture their affinities. The tree is built with agglomerative hierarchical clustering, then refined by an editable top-down subtree reorganization algorithm guided by three prior assumptions about where hybrid text belongs. The overall algorithm has computational complexity O(N² log N).

2. A Tree-Structured Contrastive Loss (TSCL). The loss aligns the embedding space with the HAT: for each text, categories sharing a closer lowest common ancestor are treated as positives and more distant ones as negatives, with a per-level contrastive term averaged over the node's depth. A Virtual Class Prototype mechanism supplies a learnable anchor per class so that small mini-batches can still cover many categories.

3. RealBench, a large-scale hybrid-text benchmark. Built by aggregating original samples from MAGE, M4, TuringBench, OUTFOX, and RAID, then applying hybrid construction strategies (paraphrasing, extension, polishing, translation). It spans 1,204 text categories and approximately 16.4 million text samples, plus 11 perturbation-based attack types.

4. A retrieval-based few-shot adaptation scheme. For out-of-distribution settings, the method rebuilds its retrieval database from a small number of target-domain samples, adjusting decision boundaries under severe distribution shift. A K-means-based database compression step reduces retrieval candidates while keeping class representations balanced.

Main Findings

• TSCL measurably aligns representations with the tree: after training with TSCL, 99.32% of triplets satisfy the hierarchical similarity constraint defined in Theorem 3.1.

• Supervised detection improves with the tree and the loss: DETree trained on RealBench under prior1 reaches an average AvgRec of 97.88 and F1 of 97.75 across MAGE, M4-monolingual, M4-multilingual, and TuringBench. The best comparison method, DeTeCTive, reaches 96.94 AvgRec and 96.74 F1 on the same average. Removing TSCL drops DETree to 96.50 AvgRec and 96.32 F1.

• Prior assumptions matter, and "hybrid = AI-involved" wins: among the three priors, prior1 performs best (97.88 AvgRec), prior3 next (97.68), and prior2 worst (96.17). The authors interpret this as evidence that AI stylistic signals dominate once any AI is involved in generation.

• Out-of-distribution AUROC gains: DETree achieves improvements of +15.55 AUROC on MAGE-Paraphrase, an average of +7.94 on DetectRL, and +10.39/+15.01 on the GPT4o-Edited and Llama3.1-Edited subsets of Beemo. In the few-shot setting, MAGE Unseen reaches 99.58 / 99.81 AUROC with 5 / 10 shots, versus 72.07 / 74.22 for UAR.

• Fine-grained AI risk detection: on the HART three-level task under prior1, DETree records ALL-set AUROC / TPR@5%FPR of 0.988 (95.3%) for Level-3, 0.992 (98.5%) for Level-2, and 0.998 (99.5%) for Level-1. The strongest baseline, HART(Binoculars), reaches 0.883 (61%) at Level-3 and 0.838 (58%) at Level-1.

• Humanized-by-tool text is detected without training on it: although the training data contains no samples humanized by commercial tools, the model identifies them. Machine-humanized human text is hard to distinguish from machine-generated base text, but distinguishable when the base text is human-written.

• Robustness and deployability: DETree maintains stable performance across all attack types when trained with adversarial samples, and still outperforms an adversarially trained RoBERTa-large in most attack scenarios without adversarial exposure. RoBERTa-large degrades substantially under Synonym, Perplexity, and Homoglyph attacks. Compressing RealBench's full training set into 10K representatives slightly improves accuracy while reducing storage and computation (the figure caption describes RealBench as ~12M samples). Training on only 10% of the full category set keeps the performance drop within 3%.

Methodology in Plain English

The researchers start from an intuition: if you embed texts produced by different collaboration processes, similar processes naturally land near each other. So they first fine-tune a text encoder with supervised contrastive learning, treating each category (for example, "Llama3_polish_GPT-4o" or "human_polish_Gemini1.5") as its own class. They then compute how similar every pair of categories is, using the dot product of category centroid vectors, and use that similarity matrix to grow a tree. Closely related categories end up under the same branch; distantly related ones separate early. Because plain hierarchical clustering forces strict binary splits, they add a reorganization step that tries different groupings of a subtree and keeps whichever gives the best Silhouette Score, guided by assumptions about whether hybrid text should sit with human text, with AI text, or in its own group.

Once the tree exists, they retrain the encoder with a contrastive loss derived from it. For any text, categories closer in the tree are pulled together in embedding space and distant ones are pushed apart, level by level, averaged over the depth of the label's node. A learnable prototype vector per class stands in for classes that a mini-batch does not happen to cover.

At inference, the detector uses K-nearest-neighbor retrieval over a database of class representatives rather than a fixed classifier head. To keep the database small and balanced, each class is clustered with K-means and the direction of each cluster's mean vector becomes a representative. When the detector faces a new domain, a handful of target-domain samples are added to rebuild the database and shift the decision boundary. Training uses RoBERTa-large fine-tuned with LoRA, embeddings taken from layers 17–19, k set to 5 or 50, and a contrastive temperature of 0.07.

Why This Matters

The paper shifts AI-text detection from a binary verdict toward structured provenance: not just "was AI involved" but "how, and in what combination." That matters because tolerance for AI involvement is scenario-dependent, and because new LLMs keep appearing faster than detectors can be retrained on them.

Real-world applications:

• Academic integrity: distinguishing lightly AI-polished student work from fully AI-generated submissions, which the paper notes existing detectors handle poorly.
• Publishing and journalism: checking whether a draft was human-written and LLM-refined versus machine-generated, to enforce originality policies.
• Content provenance and platform moderation: labeling documents by generation pipeline, including multi-model chains, to support transparency requirements.
• Copywriting and marketing: where LLM-based editing may be acceptable, enabling permissive but still audited workflows.

Industry relevance: the retrieval-based design and the compression result, where a full training set is reduced to 10K representatives with a slight accuracy gain, point toward deployable detectors that can be updated with a few in-domain samples rather than full retraining. The few-shot adaptation results, including 99.62 / 99.88 AUROC on DetectRL Multi-Domain, suggest a practical path for handling new models as they are released.

Future Directions

• Adversarial evasion through fine-tuning: the authors state they have not explored evasion where a model is fine-tuned specifically to bypass detection.
• Scaling beyond three collaborators: the paper focuses on hybrid texts involving three authors and calls extension to more collaborators an open direction.
• Handling entirely unseen, rare domains: the method still needs a small number of in-domain samples to adjust decision boundaries when facing rare or unseen domains.
• New LLMs and shifts in writing practice: as models and collaboration patterns change, whether the HAT structure and retrieval database stay meaningful without rebuilding is not reported.

Target Audience

Researchers and graduate students in NLP working on AI-text detection, representation learning, or contrastive learning; practitioners building provenance, moderation, or integrity tooling who need detection that generalizes to new models; and policy or academic-integrity specialists interested in how fine-grained the distinction between human and AI contributions can currently be made. Readers should be comfortable with embedding spaces, retrieval-based classification, and hierarchical clustering, though the paper's high-level story is accessible without deep math.

Authors’ abstract

Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class). We observe that representations of texts generated through different processes exhibit inherent clustering relationships. Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes. Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. Our code and dataset are available at https://github.com/heyongxin233/DETree.