CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluation

AI summary

Overview

Research area: Natural Language Processing / LLM evaluation methodology, specifically data contamination in benchmark datasets.

Technical level: Advanced. The paper assumes familiarity with LLM fine-tuning (LoRA), macro F1 evaluation, benchmark contamination theory, and knowledge-graph style entity-relation extraction.

Scope: The paper proposes a framework called CoreEval that automatically rewrites existing NLP benchmark datasets by injecting up-to-date real-world knowledge from the GDELT event database, aiming to produce datasets that reveal rather than hide contamination-driven performance inflation across 11 LLMs and 5 NLU tasks.

What This Paper Is About

Public benchmark datasets are often absorbed into the training corpora of newer LLMs, which artificially inflates their measured performance and makes leaderboard comparisons unreliable. Existing fixes modify old data (risking label drift) or generate brand-new data (risking loss of semantic complexity). CoreEval instead retrieves real-world, time-stamped knowledge and merges it into the original examples while preserving labels, producing a refreshed dataset that resists leaked memorization.

Key Contributions

1. CoreEval framework: An automatic contamination-resilient evaluation strategy that updates datasets by integrating real-world knowledge retrieved from the GDELT database rather than paraphrasing or generating data from scratch.
2. A structured workflow grounded in cognitive learning theory: The pipeline is organized around Bruner's three learning processes — acquisition, transformation, and evaluation — mapped to three modules: Real-World Knowledge Attainment, Knowledge Recontextualization, and Data Reflection.
3. A rigorous data reflection mechanism: An LLM agent iteratively checks generated text against two criteria (Incorrect Information and Label Alignment) and triggers regeneration when either fails, keeping updated labels consistent with original annotations.
4. Extensive empirical validation: Experiments over five datasets (Emotion, Irony, Stance, MRPC, RTE), eight open-source and three proprietary LLMs, with human quality verification and multiple contamination simulations.

Main Findings

• Zero-shot drops concentrate in subjective tasks: On the updated datasets, emotion recognition and stance detection performance declined substantially relative to the original datasets, while other tasks were more stable. The authors attribute this to likely pre-existing contamination in those tasks plus their inherent subjectivity and dependence on evolving social context.
• Proprietary models appear more contaminated: Proprietary LLMs showed a larger average performance drop of 5.42% on the updated data, compared with 3.62% for open-source models. The authors note that closed training data and parameters make contamination detection especially hard for proprietary systems.
• Qwen2.5 is competitive among open-source models: The paper reports that while proprietary models generally outperform most open-source models, the Qwen2.5 series achieves comparable or even superior performance among open-source models.
• Overestimation grows with model scale: In the contamination simulation on the original dataset, Qwen2.5-7B showed δ1 = 12.01 and δ2 = 4.74, while Qwen2.5-14B showed 17.45 and 7.19. These scale-induced gaps shrank markedly on the updated dataset.
• Cognitively complex tasks are more contamination-sensitive: Irony detection, stance detection, and RTE consistently produced higher δ values than simpler tasks such as emotion recognition and MRPC.
• CoreEval beats simple semantic rewriting: For Llama3-8B, average δ1/δ2 were 18.95/6.84 on the original dataset, 10.62/2.09 on the semantic (restated) dataset, and 4.38/1.29 on the CoreEval dataset. Similar ordering held across the eight open-source models in Table 3.
• Proportion effects behave as expected: As the exposed test-set proportion grows, δ1 rises (more memorization); as the training-set proportion grows, δ2 falls, because task understanding increasingly comes from training data rather than leaked test data.
• Text-only leakage does not inflate scores: When only test text (without labels) leaks into training, δ1 and δ2 were predominantly negative across the eight open-source models, consistent with prior findings that text-only contamination alone does not produce overestimation. The authors contrast this with the large gains seen when both test labels and prompts are exposed.
• Updated data is more stable across ratios: Variance of δ1 and δ2 across data proportions (20%, 40%, 60%, 80%, 100%) was consistently lower for the CoreEval dataset than for the original and semantic datasets.
• Human quality checks passed: Five trained computational linguistics researchers rated 50 randomly selected samples. Fluency and coherence (0-2 scale) scored between 2.55 and 2.99; factuality and accuracy (0-1) between 0.94 and 0.99; Fleiss' Kappa ranged from 0.73 (Emotion, Stance) to 0.86 (MRPC), which the authors classify as substantial agreement.

Methodology in Plain English

1. Pull entities out of the original text. An LLM reads each example and extracts the entities it mentions.
2. Look up what has happened since. Those entities are queried against GDELT, a CAMEO-coded event database containing over 200 million geolocated events with global coverage from 1979 to the present, accessed through Google BigQuery. The retrieval window starts from the release date of the latest open-source model, so the retrieved news cannot overlap with that model's training data. An LLM then summarizes the retrieved events.
3. Swap old relations for new ones. Relational triples (entity, relation, entity) are extracted from the original sentence and regenerated using the fresh knowledge. The original sentence is also rewritten semantically while preserving its triples, giving a style reference that keeps the updated text natural.
4. Rebuild the sentence, keep the label. The final updated text blends the original, the triple-substituted version, the new triples, and the semantic style reference — with the label copied unchanged from the original example to keep the classification task intact.
5. Reflect and regenerate. An LLM agent reviews each generated item step by step against two criteria — whether it misstates the retrieved facts, and whether it aligns with the ground-truth label — and sends failing items back for regeneration.
6. Test for contamination. Each open-source model is LoRA fine-tuned (rank 16, alpha 32, dropout 0.1, learning rate 1e-4, 3 epochs) under simulated leak scenarios, and two deltas are computed: δ1 = P_test − P_zero (test-set-only fine-tuning minus zero-shot) and δ2 = P_train+test − P_train (both sets minus training-set-only), where δ2 isolates memorization from genuine task learning. Macro F1 is the unified metric.

Why This Matters

Impact on research: Benchmark scores are the primary currency of LLM progress, and contamination silently corrupts them. CoreEval offers a repeatable, largely automated way to refresh an existing benchmark without discarding its task semantics or its labels, and it introduces δ2 as a sharper diagnostic than the usual single performance-gain number. The reported finding that proprietary models show a larger drop (5.42% vs 3.62%) raises accountability questions for closed models.

Real-world applications:

• Model procurement and vendor evaluation: Enterprises comparing proprietary LLM APIs need benchmarks that have not silently been trained on, since vendors do not disclose training data.
• Regulatory and compliance auditing: Auditors assessing whether a model's claimed capability is genuine versus memorized get a quantitative method (δ1, δ2) rather than a qualitative claim.
• News and social-media monitoring systems: Because updates draw on time-stamped GDELT events, the pipeline is directly usable for stance and emotion tasks where expressions shift as real events unfold.
• Continuous benchmark maintenance: Dataset maintainers can refresh a benchmark on a rolling schedule instead of commissioning costly manual re-annotation.

Industry relevance: The pipeline is cheap relative to human curation — it uses an LLM plus an API — so teams running internal eval suites can regenerate contamination-resistant test sets on the same cadence as model releases. It also gives model providers a defensible answer to contamination accusations.

Future Directions

• Extending beyond classification: The authors state CoreEval was applied only to classification tasks and plan to extend it to question answering and summarization.
• Reducing hallucination risk: The authors acknowledge the pipeline can generate a minimal amount of hallucinated data despite the reflection and iteration steps, and note its impact is judged negligible based on their manual quality scores.
• Scaling the reflection mechanism: Whether the LLM-as-judge reflection step stays reliable for longer outputs and open-ended tasks is left open.
• Broadening the contamination simulations: Only two simulation settings (test set only; train plus test) and a text-only variant are tested, leaving other leak patterns unexamined.
• More model families and languages: The study covers eight open-source and three proprietary English-language models; generalization to other languages and architectures is not reported.

Target Audience

Researchers working on LLM evaluation, benchmark construction, and data contamination; NLP practitioners who need trustworthy internal evaluation suites; model providers and auditors who must defend or verify benchmark claims; and graduate students interested in how external knowledge bases such as GDELT can be fused with LLM pipelines. The heavy appendix material on prompts and training configurations also makes it useful to engineers implementing similar data-refresh systems.

Authors’ abstract

Data contamination poses a significant challenge to the fairness of LLM evaluations in natural language processing tasks by inadvertently exposing models to test data during training. Current studies attempt to mitigate this issue by modifying existing datasets or generating new ones from freshly collected information. However, these methods fall short of ensuring contamination-resilient evaluation, as they fail to fully eliminate pre-existing knowledge from models or preserve the semantic complexity of the original datasets. To address these limitations, we propose \textbf{CoreEval}, a \textbf{Co}ntamination-\textbf{re}silient \textbf{Eval}uation strategy for automatically updating data with real-world knowledge. This approach begins by extracting entity relationships from the original data and leveraging the GDELT database to retrieve relevant, up-to-date knowledge. The retrieved knowledge is then recontextualized and integrated with the original data, which is refined and restructured to ensure semantic coherence and enhanced task relevance. Ultimately, a robust data reflection mechanism is employed to iteratively verify and refine labels, ensuring consistency between the updated and original datasets. Extensive experiments on updated datasets validate the robustness of CoreEval, demonstrating its effectiveness in mitigating performance overestimation caused by data contamination.