Structurally Human, Semantically Biased: Detecting LLM-Generated References with Embeddings and GNNs

AI summary

Overview

Research area: machine learning and scientometrics, specifically detecting LLM-generated bibliographies by combining citation-graph topology with text embeddings and graph neural networks.

Technical level: Intermediate (assumes familiarity with random forests, graph neural networks, and embedding models, but the paper's argument is accessible).

Scope: A paired, large-scale comparison of human citation graphs and LLM-generated citation graphs built from 10,000 focal papers, showing that LLM reference lists imitate human graph structure closely while remaining separable through semantic embeddings.

What This Paper Is About

The authors ask whether reference lists produced by large language models can be distinguished from genuine human reference lists once both are turned into citation graphs. Earlier work found that LLM-suggested bibliographies look plausible and match many human bibliometric patterns, so the question is whether any detectable signal remains, and where it lives. The paper tests this by pitting structural graph features against semantic title/abstract embeddings across random forests and several GNN architectures.

Key Contributions

1. A paired experimental design over 10,000 focal papers (274,951 references) from SciSciNet, comparing ground truth, GPT-4o-generated (from parametric knowledge), and field-matched randomized citation graphs, with each focal paper's variants kept together across data splits.
2. A demonstration that structural descriptors alone (degree, closeness, eigenvector centrality, clustering coefficient, edge count) leave ground truth and GPT-4o graphs nearly indistinguishable, while still cleanly rejecting random graphs.
3. Evidence that semantic title/abstract embeddings restore separability: a random forest on aggregated embeddings reaches 0.8346 ± 0.0063 mean accuracy for ground truth vs. GPT, and GNNs with embedding node features reach 93% test accuracy per the abstract (per-model test values in Table 3 range from 93.10 to 93.78).
4. Robustness checks spanning a second generator (Claude Sonnet 4.5), multiple embedding backbones (OpenAI text-embedding-3-large, SPECTER2), subfield-level and temporally constrained random baselines, and cross-generator generalization.

Main Findings

• Structure is shared. A random forest on graph-level structural aggregates separates ground truth from GPT-4o at mean accuracy 0.6079 ± 0.0058 (mean F1 0.6061 ± 0.0055), essentially chance, despite near-identical centrality medians and interquartile ranges between the two classes.
• Random graphs are easy to reject. The same structural model reaches 0.8956 ± 0.0024 accuracy (F1 0.8946 ± 0.0026) for ground truth vs. random and 0.9275 ± 0.0059 (F1 0.9272 ± 0.0060) for GPT vs. random, confirming that realistic citation topology is far from random even under field matching.
• Embeddings sharply improve detection. Aggregated embeddings raise ground truth vs. GPT random-forest accuracy to 0.8346 ± 0.0063 (F1 0.8345 ± 0.0063), with 0.9077 ± 0.0062 (F1 0.9070 ± 0.0063) for ground truth vs. random and 0.9527 ± 0.0017 (F1 0.9526 ± 0.0017) for GPT vs. random.
• GNNs confirm the split. With structural node features, GNN test accuracies for ground truth vs. GPT cluster near chance (GCN 57.73 ± 2.10, GraphSAGE 55.59 ± 1.93, GAT 57.40 ± 2.44, GIN 51.71 ± 2.70). With embedding node features, they rise to 93.10 ± 0.55 (GCN), 93.14 ± 0.51 (GraphSAGE), 93.78 ± 0.43 (GAT), and 93.28 ± 0.57 (GIN). Table 3 reports further test numbers in the 88 to 97 range for GPT vs. random comparisons across models and feature types.
• The signal is semantic, not dimensional. Replacing node embeddings with i.i.d. vectors of matched dimensionality collapses random forest and GNN accuracy to chance.
• Effects survive generator swaps. Ground truth vs. Claude Sonnet 4.5 random-forest separability is approximately 0.77, with clean rejection of the random baseline. An RF trained on GPT-4o reaches approximately 0.72 when the generator is swapped at test time, and training on GPT-4o while testing on Claude Sonnet 4.5 yields above-chance generalization for all GNNs.
• PCA is not the story. The first two principal components explain only about 6% of the variance in the 3072-dimensional title embeddings, and several hundred components are needed to explain more than 90%, so the 2-D visualization is illustrative rather than decisive.
• Splits do most of the work early. In the random forest, average leaf depth clusters around approximately 10 levels.
• Temporal structure differs slightly. Roughly 80% of focal-reference pairs preserve temporal order under the original field-level reshuffling, fewer than 1% violate it under the temporally constrained baseline, while about 6% of GPT-generated suggestions point to papers published after the focal paper.

Methodology in Plain English

The authors start from a public dataset of 10,000 focal papers drawn from SciSciNet, published in Q1 journals between 1999 and 2021, with between 3 and 54 references, at least one citation, a defined "top field," a valid DOI, and an available abstract. For each focal paper, GPT-4o is prompted with the title, authors, year, venue, and abstract and asked to suggest as many references as the paper actually cites (using gpt-4o-2024-08-06, with a separate gpt-4o-mini-2024-07-18 pass to parse the output). Suggested references are checked against SciSciNet through fuzzy matching on titles and authors. The whole pipeline is repeated with Claude Sonnet 4.5 using identical prompts and filters.

Each focal paper becomes a node, and citations among the suggested or cited references become edges. A field-matched random baseline is created by shuffling ground truth references uniformly within each top field and reattaching them, which preserves each paper's reference count and field-level distributions of citation frequency and year while breaking latent structure. Variants include a subfield-level shuffle (292 subfields nested within 19 top fields) and a temporally constrained shuffle restricted to papers published no later than the focal paper. Graphs are size-matched by randomly removing references from ground truth and random graphs, resulting in 9,218 graphs per group for GPT-4o and 9,908 for Claude; 779 GPT-4o graphs and 89 Claude graphs were dropped. Directed edges are made undirected.

Two feature families are compared. The structural family uses degree centrality, closeness centrality, eigenvector centrality, clustering coefficient, and edge count, normalized at node level and aggregated to graph level with mean, median, interquartile range, and max-to-mean ratios. The semantic family uses 3072-dimensional OpenAI text-embedding-3-large vectors for focal-paper titles and abstracts and for reference titles, with SPECTER2 (768-dimensional) as a robustness check, using the adhoc-query version for titles only and the proximity version for title and abstract combinations. A random forest is trained on graph-level aggregates, and then GCN, GAT, GIN, and GraphSAGE models are trained on node features with 5-dimensional structural vectors or 3072-dimensional embeddings. Data are split 70/15/15 with stratified splits that keep a focal paper's variants together, using the Adam optimizer and sweeps of 500 hyperparameter setups per configuration, reported across multiple seeds.

Why This Matters

Impact on research: the paper reframes LLM bibliography detection as a semantic problem rather than a topological one. Audits that rely on citation graph shape alone will systematically under-detect machine-generated reference lists, while content-aware pipelines combining graph structure with text embeddings are the appropriate tool. It also supplies a reusable protocol, paired graphs plus domain-matched randomization, for stress-testing bibliographic authenticity without manual curation.

Real-world applications:

• Automated literature-review and bibliography-generation tools that need to flag or filter LLM-suggested references before they enter a manuscript.
• Citation recommendation and discovery systems, where LLM-mediated suggestion could feed back into which papers receive attention and credit.
• Peer review and editorial screening, where semantically cohesive but machine-generated reference lists might otherwise pass structural inspection.
• Bibliometric and research-integrity studies, where the paired-generation design can be reused to quantify systemic tilt toward recency or prestige.

Industry relevance: publishers, academic search platforms, and vendors of research assistants all have an interest in distinguishing human from machine reference lists at scale. The finding that detection must target content signals means investment in embedding-based or hybrid text-plus-graph pipelines, not simply graph-statistics dashboards.

Future Directions

1. Probing which semantic dimensions drive separability and what they represent, including recency, prestige, method versus theory orientation, and author overlap.
2. Extending the analysis to full-text signals rather than title and abstract text only, which the authors explicitly identify as a limitation of the current study.
3. Testing settings where models have access to external databases, since the present work deliberately restricts generation to parametric knowledge.
4. Moving from detection to correction and debiasing, using the identified semantic fingerprint as a target for adjusting LLM reference selection.

Target Audience

Researchers and practitioners working on LLM detection, research integrity, scientometrics, and graph machine learning; developers building literature-review or citation-recommendation tools; and editors or reviewers who need to reason about whether a bibliography was machine-generated. The paper's progressive structure, from interpretable features to GNNs, also makes it useful for students learning how to design controlled comparisons between graph and text signals.

Authors’ abstract

Large language models are increasingly used to curate bibliographies, raising the question: are their reference lists distinguishable from human ones? We build paired citation graphs, ground truth and GPT-4o-generated (from parametric knowledge), for 10,000 focal papers ($\approx$ 275k references) from SciSciNet, and added a field-matched random baseline that preserves out-degree and field distributions while breaking latent structure. We compare (i) structure-only node features (degree/closeness/eigenvector centrality, clustering, edge count) with (ii) 3072-D title/abstract embeddings, using an RF on graph-level aggregates and Graph Neural Networks with node features. Structure alone barely separates GPT from ground truth (RF accuracy $\approx$ 0.60) despite cleanly rejecting the random baseline ($\approx$ 0.89--0.92). By contrast, embeddings sharply increase separability: RF on aggregated embeddings reaches $\approx$ 0.83, and GNNs with embedding node features achieve 93\% test accuracy on GPT vs.\ ground truth. We show the robustness of our findings by replicating the pipeline with Claude Sonnet 4.5 and with multiple embedding models (OpenAI and SPECTER), with RF separability for ground truth vs.\ Claude $\approx 0.77$ and clean rejection of the random baseline. Thus, LLM bibliographies, generated purely from parametric knowledge, closely mimic human citation topology, but leave detectable semantic fingerprints; detection and debiasing should target content signals rather than global graph structure.