Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection

AI summary

Overview

Research area: Natural Language Processing — hallucination detection, retrieval-augmented generation (RAG), text-to-SQL, and neurosymbolic AI.

Technical level: Intermediate. Readers need basic familiarity with large language models, retrieval-augmented generation, and SQL, but no deep mathematical background.

Scope: The paper introduces TeQHallu, an unsupervised text-to-SQL pipeline that grounds hallucination detection in an SQL database built from reference documents, and evaluates it on the RAGTruth and DiaHalu benchmarks.

What This Paper Is About

Large language models frequently produce hallucinations — outputs that are factually incorrect or unsupported by the source context — and existing detectors mostly act as black boxes, returning a label without explaining why. The authors ask whether an LLM's pre-existing low-level competence in SQL can be repurposed to build a queryable database from reference documents, so that hallucination checks become explicit symbolic retrieval operations rather than opaque neural judgements. The goal is competitive hallucination detection with no domain-specific fine-tuning and with an inspectable reasoning trace.

Key Contributions

1. TeQHallu, an unsupervised text-to-SQL method that leverages low-level LLM competence to structure reference knowledge for hallucination detection.
2. Transparent, inspectable reasoning traces: grounding detection in an SQL database produces evidence that can be audited, in contrast to black-box detection approaches.
3. Competitive benchmark performance without fine-tuning: evaluation on the RAGTruth and DiaHalu benchmarks shows performance competing with state-of-the-art methods while requiring no domain-specific training.
4. A broader argument for neurosymbolic checkups: the results highlight lower-level, task-agnostic competences as a promising avenue for improving LLM reliability.

The code is released at https://github.com/renatovukovic/teqhallu.

Main Findings

• RAGTruth overall macro F1: TeQHallu with Gemini-2.5-Flash reaches precision 61.7, recall 88.9, and F1 71.3, outperforming all other unsupervised baselines and the supervisedly fine-tuned Llama-2-13B (70.3 / 74.1 / 71.8). It comes close to the fine-tuned LettuceDetect-large-v1 (73.5 / 72.5 / 72.8), while RAG-HAT (LLaMA-3-8B) remains the only configuration to outpace it at 82.4 / 74.4 / 78.0.

• Per-task RAGTruth results: TeQHallu scores F1 58.5 on question answering (42.8 precision / 92.5 recall), 88.4 on data-to-text (86.8 / 90.0), and 66.9 on summarisation (55.5 / 84.3).

• SQL beats other multi-step decompositions: The direct prediction baseline (Gemini-2.5-Flash) reaches F1 61.6, multi-step atomic fact checking (Min et al. 2023) reaches 65.7, and JSON key-value extraction reaches 64.9 — TeQHallu's 71.3 exceeds all three, driven by higher precision.

• Ablation isolates the mechanism: Direct prediction scores 61.6 F1, pure symbolic prediction over entity IDs drops to 39.7, a "SELECT only IDs" variant reaches 69.7, and full TeQHallu reaches 71.3. Using SQL entity IDs directly creates a symbolic coupling that is too strict, while feeding ID sets into the LLM prompt provides the optimal balance.

• Open-weight backbones also benefit: With Qwen3 235B/22B (Spider exact match 29.0), TeQHallu improves F1 from 64.9 to 67.2; with LLaMa 3.3 70B (Spider EM 35.4) from 64.3 to 65.4. The exception is GPT-OSS 20B/3.6B (Spider EM 22.4), where direct prediction's 68.7 F1 stays slightly above TeQHallu's 68.6. Strong text-to-SQL capability helps but exceptionally high parsing metrics are not required.

• DiaHalu TOD results: TeQHallu achieves the highest F1 for each model group: 57.33 with Gemini-1.5 PRO, 58.18 with GPT-4, and 58.11 with Gemini-2.5-Flash. A variant that builds one dataset-wide MultiWOZ database instead of per-dialogue databases degrades to 56.76 F1 because cross-dialogue constraints make verification overly strict.

• Fine-grained DiaHalu gains: TeQHallu with Gemini-2.5-Flash raises micro-F1 from 31.39 to 42.95, with large improvements on non-factual errors (27.63 to 48.65) and over-reliance (30.77 to 40.00). Incoherence stays stable, irrelevance drops from 23.53 to 10.53.

• Database construction fidelity: Manual evaluation of 50 sampled reference documents per task (150 total) yields SQL parsing accuracy of 80.00% for question answering, 94.00% for data-to-text, 82.00% for summarisation, and 85.33% overall. Of the 22 incomplete database cases (14.67%), 14 (63.6%) still produced correct predictions via the reference-text safety net; cascading errors accounted for 8 instances (5.33% total) and pure reasoning failures for 8 of 128 cases (6.25%).

• Cost profile: TeQHallu requires 5 sequential prompting steps per document-response pair, with roughly 4.5k–9.7k input tokens and about 600 output tokens per sample, versus 1 API call and roughly 0.8k–1.8k input tokens for direct prediction. It requires zero training FLOPs, unlike fine-tuned baselines.

• Qualitative example: In the gluten-containing foods RAGTruth example, direct prediction flags only the unsupported health "Note," while TeQHallu also catches hallucinated grain derivatives such as "Bran," "Germ," and "Barley sugar" that are absent from the reference.

Methodology in Plain English

The pipeline has three stages, all driven by prompting a single general-purpose LLM (Gemini 2.5 Flash via the Google Vertex AI API, with open-weight models tested separately).

First, the model builds a relational SQL database from the retrieved passages incrementally. It inspects the current schema with metadata queries such as PRAGMA table_info, compares the passages against what the database already holds, and generates operations to insert missing information or extend the schema, encoding entities, numeric ranges, and qualifiers as relational tuples with source provenance.

Second, given a sampled response, the system isolates the claims that need factual verification and prompts the LLM to write targeted SELECT queries against the database. Executing these queries returns symbolic evidence drawn directly from the source passages.

Third, a neurosymbolic checkup reconciles the two signals. The model first makes a neural-only judgement from the reference and response alone, then receives the SQL retrieval outputs, interpreted database summaries, and relevant context, and is instructed to compare its preliminary judgement with the retrieved evidence and justify a final decision. The response is classified as fully supported, partially supported, unsupported, or contradicted — the gluten example is labelled an "Evident Conflict." The output pairs the hallucination prediction with a structured explanation.

Why This Matters

Impact on research. The paper argues that explicit symbolic reasoning can compensate for the absence of parameter optimisation, delivering supervised-level accuracy without domain-specific fine-tuning. It also extends the TeQoDO line of work by giving a generated knowledge base a concrete downstream application, and it suggests that low-level, task-agnostic competences already present in LLMs are an under-explored resource for reliability.

Real-world applications (from the paper's framing):

• High-stakes domains such as medicine and law, where unsupported model outputs can undermine decision-making.
• Retrieval-augmented generation systems, where cross-referencing outputs against external knowledge is the primary safeguard and transparent evidence trails aid human oversight.
• Task-oriented dialogue systems, which the DiaHalu evaluation directly targets, including multi-turn conversations built on structured database intents.
• Automated fact-checking and digital safety pipelines that need auditable justifications rather than binary labels.

Industry relevance. The approach shifts cost from offline data curation and parameter optimisation to lightweight inference-time querying, and it works across both closed and open-weight models. The authors note it can run on lightweight models such as GPT-OSS with 3.6B active parameters without GPU cluster fine-tuning, though the token and latency overhead of five prompting steps is real. The paper also raises ethical concerns: reliance on biased reference documents would propagate those biases, automated "verified" statuses risk automation bias, the multi-step process increases carbon footprint, and transparent traces could be exploited to reverse-engineer sensitive reference data or craft adversarial responses.

Future Directions

1. Integrating knowledge graphs alongside SQL grounding to broaden the scope of verifiable facts.
2. Iterative database refinement using detection feedback loops, to dynamically improve grounding fidelity over time.
3. Improving performance under a strict neurosymbolic check-up, which the authors link to better interpretability.
4. Human-in-the-loop evaluations to validate whether TeQHallu's symbolic explanations are actually useful to end users, particularly in high-stakes legal and financial settings — explicitly stated as out of scope for this paper.

Open questions remain around the quality and completeness of reference documents, information loss during translation into SQL, sensitivity to prompt wording, and errors in SQL generation propagating to the final prediction.

Target Audience

Researchers and practitioners working on hallucination detection, retrieval-augmented generation, and neurosymbolic AI will find the core argument useful, as will engineers who want interpretable detection without labelling data or fine-tuning. The paper is also relevant to evaluators and auditors who need justifications for model-output labels, and to NLP researchers interested in how low-level competences such as SQL can be repurposed for high-level verification tasks.

Authors’ abstract

Hallucination-where a language model generates outputs that are factually incorrect or unsupported by the source-is a major challenge for both prompted and fine-tuned language models. Detecting hallucinations is difficult due to the opaque reasoning processes of LLMs, which often provide little insight into why a model's output may be inaccurate. In this work, we investigate whether an LLM can use an alternative, low level, symbolic competence such as SQL for unsupervised hallucination detection in some high level task. For this, we make an LLM build an SQL database from reference documents. This SQL database is then used for reasoning over the reference and the sampled response in a hallucination detection pipeline that is grounded in the database, thereby providing a neurosymbolic checkup. On RAGTruth and DiaHalu hallucination detection datasets, we find that our approach improves on direct prediction and competes with state-of-the-art hallucination detection methods, while not requiring domain-specific fine-tuning. Instead it relies on a low-level general competence already present in LLMs. This warrants further investigation of low-level LLM competences in neurosymbolic approaches.