How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluation

AI summary

Overview

Research area: Automated fact-checking (AFC), natural language processing, information retrieval, and benchmark evaluation.

Technical level: Intermediate. The paper uses standard NLP/IR terminology (recall@K, macro-F1, BM25, dense retrieval, zero-shot LLMs) but is written around a clear two-stage pipeline that a non-specialist can follow.

Scope: A cross-benchmark evaluation of nine fact-checking model configurations spanning sparse baselines, fine-tuned transformers, zero-shot LLMs, and the two top systems from the AVeriTeC 2025 shared task, tested under identical conditions on four datasets from scientific, open-web, and climate domains.

What This Paper Is About

Automated fact-checking systems typically work in two stages: first retrieve evidence documents, then predict whether a claim is Supported, Refuted, or Not Enough Information (NEI). The problem is that most systems are built and measured on a single benchmark, are rarely compared against simple baselines, and are assumed to generalise even though nobody has tested the full retrieve-then-verify pipeline across structurally different domains at once. This paper runs that cross-benchmark test to find out whether reported leaderboard progress reflects genuine language understanding or dataset-specific engineering.

Key Contributions

1. A unified two-stage cross-benchmark evaluation. The authors evaluate evidence retrieval and veracity prediction separately and jointly across four structurally diverse datasets (AVeriTeC, SciFact, ClimateCheck, ClimateFEVER) under identical preprocessing and conditions, which prior work had not done (earlier studies such as Thakur et al. 2021 and Calamai et al. 2025 examined only a single pipeline stage in isolation).

2. Nine model configurations spanning the full method spectrum. These range from random and sparse retrieval baselines (TF-IDF, BM25), through fine-tuned transformers (Longformer, DistilRoBERTa) and logistic regression classifiers, to zero-shot LLMs (Llama 3.1-8B and Llama 3.1-70B), plus the AVeriTeC 2025 winner (AIC CTU) and runner-up (Sanctuary).

3. A quantified decomposition of the retrieval-versus-veracity bottleneck. By substituting gold-annotated evidence for retrieved evidence, the authors measure how much veracity performance is limited by retrieval rather than by the veracity model itself.

4. Released artifacts for reproducibility. Code, pre-processed datasets, and all results are published at the project repository (https://github.com/aidausmanova/FCBench).

Main Findings

• Classical baselines remain competitive. Claim-only TF-IDF + logistic regression outperforms evidence-conditioned zero-shot LLMs and AVeriTeC 2025 top-performing systems on ClimateCheck, the largest and most informal corpus. TF-IDF also consistently outperforms BM25 across all four datasets, which the authors attribute to BM25's document-length normalisation penalising long Wikipedia passages and scientific abstracts.

• Rankings are strongly domain- and metric-dependent. No single system dominates all four benchmarks. The best system on SciFact reaches macro-F1 0.700 but only 0.315 on ClimateCheck. On AVeriTeC, Sanctuary leads on accuracy (0.709 vs. 0.606) and macro-F1 (0.482 vs. 0.371) over the shared-task winner AIC CTU, even though AIC CTU won the 2025 shared task. A single system (Sanctuary) spans 0.39 macro-F1 across datasets.

• Retrieval quality is the primary bottleneck. Replacing retrieved evidence with gold annotations improves accuracy by 14-22 percentage points across LLM models. The largest gains are for Llama 70B on ClimateFEVER (+22 pp) and Llama 8B on ClimateFEVER (+20 pp); the smallest are on ClimateCheck (+14-15 pp), where the large corpus limits coverage even with gold relevance assumed.

• Retrieved evidence can actively harm veracity prediction. In a matched ablation with identical model and prompt, adding BM25-retrieved evidence changes accuracy by +21.6 (Llama 8B, AVeriTeC), +0.3 (SciFact), -24.4 (ClimateCheck), and +3.9 (ClimateFEVER); for Llama 70B the corresponding changes are +6.8, +0.3, -19.4, and +20.1. The authors argue the negative result on ClimateCheck is a genuine out-of-domain failure rather than evidence that claim-only models are superior.

• Retrieval rankings reverse across domains. On SciFact, AIC CTU achieves Recall@5 = 0.731 versus TF-IDF's 0.217, rewarding dense semantic retrieval. On AVeriTeC, TF-IDF leads at Recall@5 (0.126) while Sanctuary reaches only 0.077, favouring lexical overlap.

• Climate corpora are hard for every retriever. Retrieval results are low across all methods on ClimateCheck and ClimateFEVER. Mean Jaccard word overlap between claims and gold evidence is 0.122 on ClimateFEVER, 0.093 on AVeriTeC, 0.087 on SciFact, and only 0.047 on ClimateCheck, correlating with the vocabulary gap between informal claims and formal abstracts.

• Claim-only labels carry exploitable signal. Relative to random prediction, claim-only macro-F1 improves by 13.7 points on AVeriTeC, 6.5 on SciFact, 26.7 on ClimateCheck, and 15.0 on ClimateFEVER. The authors treat gaps below roughly 5 points as within the annotation-noise floor, citing a reported Cohen's kappa of 0.334 for ClimateFEVER evidence annotations in prior work and kappa = 0.55 from their own two-annotator re-labelling.

• Zero-shot LLMs are unstable. Llama 70B is competitive on AVeriTeC (accuracy 0.640) but near-random on ClimateFEVER (accuracy 0.240). Fine-tuned transformers are the most reliable non-SOTA systems, beating TF-IDF + LogReg on all datasets.

• Dataset-specific veracity outcomes. On ClimateCheck, Longformer achieves the highest accuracy (0.618), followed by DistilRoBERTa (0.617). On ClimateFEVER, peak accuracy is 0.45 (AIC CTU), followed by Sanctuary (0.446) and Longformer (0.441). On SciFact, Sanctuary leads at accuracy 0.702 / macro-F1 0.700, with AIC CTU second at accuracy 0.566 / macro-F1 0.482. On AVeriTeC, BM25 + Llama 70B is second at accuracy 0.708, while fine-tuned transformers reach 0.562-0.575.

• The AVeriTeC 2025 winner does not transfer as well. Under the official AVeriTeC protocol (Hungarian METEOR at the 0.25 threshold), AIC CTU scores 0.536 on AVeriTeC, but Sanctuary is stronger out-of-domain (SciFact: 0.490 vs. 0.390; ClimateCheck: 0.460 vs. 0.000; ClimateFEVER: 0.325 vs. 0.149). The authors attribute AIC CTU's near-zero ClimateCheck score to architectural scaling limits, since it builds FAISS vectors for all evidence documents: workable for AVeriTeC's 32,818 web documents but not for ClimateCheck's 394,269 abstracts.

• The climate subset of AVeriTeC is too small to analyse. Only six claims (1.2%) are climate-related, and Sanctuary shows no corresponding degradation on them (83.3 vs. 70.9), so the authors do not interpret this subset as isolating topic effects.

Methodology in Plain English

The researchers assembled four existing fact-checking benchmarks that differ along two structural axes: the evidence type (scientific abstracts versus web/Wikipedia text) and the claim origin (expert-written versus real-world/social-media). They standardised everything: an 80/10/10 train/development/test split, duplicate removal by exact string matching, gibberish filtering using language identification and perplexity-based filtering, HTML stripping and Unicode normalisation for scraped web pages, and truncation of long documents (4,096 tokens for Longformer, 512 tokens for DistilRoBERTa).

They then ran nine model configurations through both pipeline stages. For retrieval, they measured Recall@K and F1@K at K = 5, 10, and 20 against gold-annotated relevant documents, with AVeriTeC scored using the official Hungarian METEOR protocol at a 0.25 threshold. For veracity, they measured accuracy and macro-averaged F1, separating "claim-only" baselines (which see the claim text but no evidence) from "evidence-conditioned" models (which see both). This claim-only condition acts as a diagnostic probe for whether labels can be predicted from claim text alone, following hypothesis-only analyses in NLI research.

Robustness checks included three fixed random seeds with mean ± standard deviation (single runs for LLMs at temperature 0.1), 95% bootstrap confidence intervals with disjointness as the significance criterion, an oracle experiment swapping retrieved evidence for gold evidence, a matched ablation isolating the effect of adding retrieved evidence, a Jaccard word-overlap analysis, and a manual error analysis that classified sampled errors as genuine model errors, annotation mistakes, or debatable errors. Appendix F repeats the ablation with a frontier closed-weight model under three evidence conditions and tests an iterative FIRE-style agent. Prompt sensitivity is not evaluated, and the paper reports that no evaluation of multi-domain or multilingual fact-checking was performed.

Why This Matters

Impact on research. The paper argues that leaderboard rankings should not be taken as proof of real-world utility until systems are evaluated across diverse domains. It calls for cross-domain AFC evaluations with mandatory classical baselines and explicit decoupling of the retrieval and veracity stages. Because simple baselines outperform sophisticated systems on the largest, most informal corpus here, reported gains in single-benchmark papers may reflect dataset-specific engineering rather than improved language understanding.

Real-world applications:

• Newsroom and social-media fact-checking triage, where a claim's domain is unknown in advance and a system that collapses out-of-domain cannot be trusted for deployment.
• Climate misinformation monitoring, where ClimateCheck's informal social-media claims must be checked against a 394,269-abstract scientific corpus, a setting where every tested retriever performed poorly.
• Scientific claim verification, where SciFact is the one dataset that rewards semantic reasoning and where Sanctuary's sentence-chunking architecture aligns with rationale-sentence annotations.
• Benchmark auditing and dataset curation, since the paper quantifies annotation noise and shows claim-only baselines can match or beat evidence-conditioned systems on datasets with weak evidence annotations.

Industry relevance. Organisations building or procuring fact-checking systems need to know when a simpler approach suffices. The paper's evidence that TF-IDF plus logistic regression beats shared-task winners on the largest informal corpus, and that retrieved evidence can reduce LLM accuracy by 24.4 points under domain shift, is directly actionable for system design and procurement decisions.

Future Directions

1. Improving retrieval for large and informal corpora. The authors suggest domain-adaptive pretraining and adaptive retrieval as ways to boost performance on domain-specific datasets such as ClimateCheck and ClimateFEVER, noting the ClimateCheck 2025 challenge as a venue.

2. Extending coverage beyond four datasets. The study's limitations note that FEVER (300K+ claims) and FEVERous were excluded because they exceeded the computational budget, and that multi-domain and multilingual fact-checking were not evaluated.

3. Testing prompt sensitivity and newer generative systems. All LLM experiments used a single prompt template per dataset, so prompt sensitivity remains unmeasured, and generative AFC systems beyond Llama 3.1 were not evaluated.

4. Scaling up failure and annotation analysis. The error analysis covers only a subset of error cases with moderate annotator agreement, so the authors note it cannot be interpreted as a dataset-wide estimate of annotation noise.

Target Audience

Researchers and practitioners in NLP, information retrieval, and automated fact-checking; benchmark designers and dataset curators concerned with annotation quality and cross-domain generalisation; engineers evaluating whether to deploy a fine-tuned transformer, a zero-shot LLM, or a classical sparse-retrieval baseline for claim verification; and policy or platform teams who need to understand the reliability limits of automated veracity prediction before trusting it in deployment.

Authors’ abstract

Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a single benchmark and cannot be trusted to generalise across domains. No prior work cross-evaluates the full two-stage retrieve-then-verify pipeline across diverse datasets, complementing retrieval-only studies (Thakur et al., 2021) and single-stage benchmarking studies (Calamai et al., 2025). We benchmark nine models, ranging from random and sparse baselines to fine-tuned transformers, zero-shot LLMs, and the two highest-ranked systems from the AVeriTeC 2025 shared task, across four datasets spanning scientific, open-web, and climate domains. Three findings stand out: (1) on ClimateCheck claim-only and fine-tuned models outperform zero-shot LLM and top-performing AVeriTeC 2025 systems, highlighting that noisy evidence can degrade veracity prediction; (2) system rankings are strongly domain- and metric-dependent: the best model on SciFact (macro-F1 0.70) drops to 0.31 on ClimateCheck, while the AVeriTeC 2025 winner and runner-up swap rankings based on evaluation metrics and datasets; (3) replacing retrieved evidence with gold annotations improves veracity accuracy by 14-22 points across models, confirming retrieval remains primary bottleneck. We release code, pre-processed datasets, and all results to support reproducible AFC research.