CORE: Comprehensive Ontological Relation Evaluation for Large Language Models

AI summary

Overview

Research area: Natural Language Processing — specifically benchmark design and evaluation of large language models' semantic and ontological reasoning, with implications for model safety and reliability.

Technical level: Intermediate. The abstract uses evaluation metrics such as Cohen's Kappa, Expected Calibration Error, and confidence calibration, but the core idea — can a model tell "related" from "unrelated" — is intuitive.

Scope (one sentence): The paper introduces CORE, a large multiple-choice dataset plus a small human-validated benchmark, and uses them to show that state-of-the-art LLMs are far better at spotting related concepts than at recognizing when two things are genuinely unrelated.

What This Paper Is About

Existing LLM benchmarks mostly test whether a model can find or apply a meaningful relationship between concepts, but rarely test whether the model knows when no relationship exists. That gap matters because a model that always produces some relation — even a spurious one — is unreliable for knowledge-heavy and safety-sensitive tasks. The paper builds a dataset and benchmark designed to test both sides equally, and measures how badly current models fail on the "unrelated" side.

Key Contributions

1. CORE dataset: 225K multiple-choice questions spanning 74 academic disciplines, built to probe semantic relation reasoning, including domain-specific content.
2. A validated open-domain benchmark: 203 rigorously validated questions (reported Cohen's Kappa = 1.0) covering 24 semantic relation types, with equal representation of unrelated pairs — a deliberate design choice to prevent models from succeeding by always guessing "related."
3. A human baseline: collected from over 1,000 participants, reaching 92.6% overall accuracy and 95.1% on unrelated pairs, giving a reference point for what good performance looks like.
4. A systematic evaluation of 29 state-of-the-art LLMs, measuring not just accuracy but also confidence and calibration, and reporting a "semantic collapse rate" that quantifies how often models generate spurious relations.

Main Findings

• Large gap between human and model performance on the open-domain benchmark: humans reached 92.6% overall (95.1% on unrelated pairs), while the 29 evaluated LLMs ranged from 48.25% to 70.9% overall — a substantial shortfall.
• The failure is concentrated almost entirely on unrelated pairs: models scored 86.5–100% on related pairs (near ceiling) but only 0–41.35% on unrelated pairs. The benchmark's equal representation of unrelated pairs is what exposes this.
• Models are confidently wrong: despite their poor accuracy on unrelated pairs, the models assigned similar confidence levels, roughly 92–94%, across conditions.
• Calibration degrades sharply: Expected Calibration Error increased by 2–4x on unrelated pairs compared with related ones, meaning the confidence scores became much less trustworthy exactly where the models erred.
• Systematic spurious relation generation: a mean semantic collapse rate of 37.6% indicates models frequently manufacture a relationship between concepts that are not related.
• Domain-specific questions are much harder: on the full 225K CORE multiple-choice set, accuracy fell to approximately 2%, which the authors describe as highlighting substantial challenges in domain-specific semantic reasoning.
• The authors frame unrelatedness reasoning as an under-evaluated frontier for LLM evaluation and safety.

Methodology in Plain English

The researchers built two evaluation resources. The first is a very large set of multiple-choice questions — 225K of them — drawn from 74 disciplines, so that models are tested on semantics across many domains rather than only general knowledge. The second is a smaller, carefully checked benchmark of 203 questions covering 24 kinds of semantic relations, where the design deliberately balances questions about genuinely related concept pairs with questions about unrelated pairs; the reported inter-annotator agreement (Cohen's Kappa = 1.0) is the evidence that this small set was validated rather than loosely assembled.

They then established a human reference point by collecting answers from more than 1,000 participants. Finally, they ran 29 current LLMs on both the small benchmark and the large dataset, recording accuracy on related versus unrelated items separately, and also recording how confident the models were — which lets them check whether confidence tracked correctness. The "semantic collapse rate" is their measure of how often a model invents a relation where none exists. The abstract reports these aggregate results without describing the specific models tested, the question-construction pipeline, or any statistical tests.

Why This Matters

Impact on research: The paper argues that current benchmarks are one-sided. A model can look strong on relation-detection tasks while being nearly unable to recognize unrelatedness, and standard accuracy averages can hide that because related-pair performance dominates. Separating related from unrelated accuracy — and pairing it with calibration measurement — offers a template for evaluating whether a model knows the limits of its own knowledge.

Real-world applications:

• Knowledge bases and search: systems that infer links between entities can silently inject false relationships into a graph or index; a model that cannot say "unrelated" makes this failure systematic rather than occasional.
• Retrieval and question answering: retrieving or citing material based on a fabricated connection produces confident, plausible-looking output that is nonetheless wrong.
• Medical, legal, and scientific decision support: in the 74 domain-specific disciplines covered, spurious relations between concepts carry higher stakes than in general chat.
• Safety and trust: high confidence alongside low accuracy on unrelated pairs (with calibration error rising 2–4x and a 37.6% semantic collapse rate) points to a failure mode where models mislead users rather than flagging uncertainty.

Industry relevance: teams that deploy LLMs for entity linking, recommendation, compliance checks, or domain assistants should treat unrelatedness as an explicit test case in their evaluation suites rather than assuming strong benchmark scores imply reliable relational judgment. The confidence/calibration finding is directly relevant to whether users can trust a model's stated certainty.

Future Directions

• Diagnose why unrelatedness fails: determine whether the near-ceiling related-pair versus near-floor unrelated-pair split reflects a training-data bias toward asserting relations, a prompting artifact, or an architectural limitation.
• Improve calibration on unrelated items: the 2–4x rise in Expected Calibration Error suggests confidence estimation needs work, or that models should be trained to abstain or flag uncertainty instead of answering.
• Extend and harden the resource: the abstract does not specify the annotation protocol or the models evaluated, so broader validation across more annotators, languages, and relation inventories is a natural next step.
• Address the domain-specific collapse: approximately 2% accuracy on the 225K domain-specific set raises the question of whether these disciplines require retrieval, domain fine-tuning, or entirely different evaluation formats.

Target Audience

Researchers and practitioners in LLM evaluation, benchmark design, and knowledge representation; NLP engineers building entity linking, retrieval, or knowledge-graph systems; and safety-focused teams who need to know where model confidence can be trusted. Readers looking for a gentle introduction will find the central idea — related versus unrelated — accessible, while the calibration and dataset-design details suit readers with some familiarity with evaluation methodology.

Authors’ abstract

Large Language Models (LLMs) perform well on many reasoning benchmarks, yet existing evaluations rarely assess their ability to distinguish between meaningful semantic relations and genuine unrelatedness. We introduce CORE (Comprehensive Ontological Relation Evaluation), a dataset of 225K multiple-choice questions spanning 74 disciplines, together with a general-domain open-source benchmark of 203 rigorously validated questions (Cohen's Kappa = 1.0) covering 24 semantic relation types with equal representation of unrelated pairs. A human baseline from 1,000+ participants achieves 92.6% accuracy (95.1% on unrelated pairs). In contrast, 29 state-of-the-art LLMs achieve 48.25-70.9% overall accuracy, with near-ceiling performance on related pairs (86.5-100%) but severe degradation on unrelated pairs (0-41.35%), despite assigning similar confidence (92-94%). Expected Calibration Error increases 2-4x on unrelated pairs, and a mean semantic collapse rate of 37.6% indicates systematic generation of spurious relations. On the CORE 225K MCQs dataset, accuracy further drops to approximately 2%, highlighting substantial challenges in domain-specific semantic reasoning. We identify unrelatedness reasoning as a critical, under-evaluated frontier for LLM evaluation and safety.