A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

AI summary

Overview

Research area: Evaluation of synthetic data produced by large language models, with a focus on intrinsic data properties rather than downstream task scores.

Technical level: Intermediate. The paper is a survey, but it includes formal metric definitions (equations for validity, fidelity, diversity, faithfulness) and a structured audit protocol, so it assumes some familiarity with LLM training pipelines and evaluation terminology.

Scope: The paper proposes a unified, data-centric framework — the LLM Data Auditor — for organizing LLM-based data generation methods, categorizing intrinsic quality and trustworthiness metrics, auditing representative works across six modalities, and outlining how such data is used downstream.

What This Paper Is About

LLMs can generate training data across many formats, turning data from a scarce resource into a controllable one, but existing research concentrates on how to generate data while paying limited attention to whether the generated data is any good. Most evaluation that does exist is extrinsic — it measures downstream model performance rather than the inherent properties of the data itself — and most studies stay within a single modality. This survey addresses that gap by building a unified framework that characterizes generation methods across six modalities and then classifies intrinsic metrics into quality and trustworthiness pillars, using it to audit representative works and expose under-covered evaluation dimensions.

Key Contributions

1. Shifting to a data perspective for comprehensive evaluation. Rather than the model- or workflow-centric framing of prior surveys (such as those organized around the generation-and-curation workflow or the synthetic-data lifecycle), the paper adopts a data-centric structure it calls the LLM Data Auditor, moving from data generation to the evaluation system, then to gap analysis, then to downstream use.
2. A systematic metrics taxonomy. Metrics are classified by utility into two pillars — Quality (validity, fidelity, diversity, utility) and Trustworthiness (faithfulness, safety, robustness, fairness, privacy, provenance, and benchmark contamination) — with modality-specific instantiations developed in later sections.
3. Unified cross-modal coverage. The framework organizes six mainstream data modalities under one structure: text data; symbolic and logical reasoning data; tabular data; semi-structured graph, JSON, and log data; vision–language data; and agent data.
4. Evaluation gap analysis. The authors conduct a structured audit of representative works, annotating each with a Family and a Role, and identify evaluation dimensions that are less consistently reported in that representative sample, offering concrete recommendations for the community.

Main Findings

• Evaluation lags behind generation. Current research on LLM-based data generation centers on models or the generation process; prior surveys either mention evaluation only briefly or are confined to a single modality such as healthcare or tabular data. Intrinsic evaluation, which assesses data quality directly, remains comparatively underdeveloped relative to extrinsic downstream-performance evaluation.
• Low-quality synthetic data causes concrete harm. The paper cites theory and large-scale experiments on repeated training over generated corpora showing that uncontrolled reliance on synthetic data can distort scaling laws and lead to "model collapse," where models gradually lose skills and degenerate across generations.
• Synthetic-data pipelines raise privacy exposure risks. Work on undesirable memorization and privacy leakage warns that these pipelines may expose personally identifiable or proprietary content.
• Common evaluation methods carry model-specific bias. Many current evaluation methods rely heavily on LLMs for scoring or filtering, which the authors note introduces significant model-specific biases.
• Correctness criteria can matter as much as scale. The paper points to prior work showing that enforcing simple correctness criteria on synthetic examples can be as vital as increasing dataset size for downstream performance.
• A "generate-evaluate-filter-refine" paradigm exists but is not unified. Frameworks such as Dataflow formalize this loop, yet the literature still lacks a unified framework to audit synthetic data before it enters the training loop.
• The audit is deliberately a snapshot, not a census. The three-tier scheme marks a dimension with a checkmark if it is explicitly operationalized via dedicated metrics, benchmarks, or protocols; a triangle if it receives only partial or indirect coverage such as proxies or narrow ablations; and a cross if it is absent. The authors state the gap analysis should be read as a comparative snapshot of a representative sample, not a field-wide prevalence estimate from an exhaustive meta-analysis.
• Under-covered dimensions are highlighted per modality. Stage 4 of the framework highlights under-covered evaluation dimensions identified from the modality-specific audits; the section-by-section lists of which dimensions are under-covered are not reported in the truncated portion of the content supplied.

Methodology in Plain English

The authors do not run new experiments. Instead, they build an organizing framework and apply it as an audit.

1. Define the framework. The LLM Data Auditor is built on five core components: LLM-based data generation methods, quality metrics, trustworthy metrics, evaluation gaps, and data usage. Figure 1 lays out six stages, from generation methods through metric taxonomy, audit protocol and gap analysis, under-covered dimensions, data usage, and future directions.
2. Split metrics into two pillars. Quality metrics capture fundamental usability (validity, fidelity, diversity, utility), while trustworthiness metrics capture risk- and governance-related concerns (faithfulness, safety, robustness, fairness, privacy, provenance, benchmark contamination).
3. Cover six modalities. For each modality the authors describe generation methods (how the data is produced), quality metrics, trustworthiness metrics, the audit, and usage/applications — mirroring the same structure across sections.
4. Select representative works rather than every paper. For each modality they choose papers from the generation-method families introduced in that modality's section, prioritizing family coverage over near-duplicate methods.
5. Annotate each audited paper with two attributes. A Family (the modality-specific category of the generation method) and a Role — one of five descriptive categories: Generation (synthetic-data pipelines or strategies), Data Construction (corpus curation, filtering, or large-scale assembly), Benchmark (novel datasets or evaluation suites), Evaluation (verification methods or dimension-specific assessments), and Governance (trustworthiness controls such as privacy, safety, and data provenance). The authors state the Role is a descriptive taxonomy, not a quality judgment.
6. Map works to the taxonomy at three levels. Checkmark, triangle, or cross as described above, then read off the gaps.

For the text modality specifically, the authors also give the formal definitions of the metrics they recommend. Examples: GAR (Grammatical Acceptability Rate) uses a RoBERTa-large model fine-tuned on the CoLA benchmark and counts the proportion of sentences whose acceptability score exceeds a threshold such as 0.5; USL-H combines understandability, sensibleness, and likability scores in a hierarchical formula for dialogue; EDS compares mean ℓ2-normalized sentence embeddings of synthetic and real corpora by cosine similarity; strict prompt-level accuracy from IFEval requires all verifiable instructions to be satisfied; RUBER blends a referenced and an unreferenced score after min-max normalization; PMI-FAITH takes the log-probability difference that captures the information gain contributed by a document beyond the dialogue history; diversity combines Self-Cosine Similarity, Type Token Ratio, Distinct-N, and n-gram Response Entropy; and faithfulness uses evidence-selection precision/recall/F1, AIS attribution rates (human and automatic), and the RARR preservation metrics (intent preservation, Levenshtein-based edit minimality, their combined form, and the harmonic-mean F1 of attribution and preservation).

Why This Matters

Impact on research. The survey argues that to realize the full potential of LLM-generated data, the research focus must shift from generation techniques to evaluation methodologies, and that a more systematic organization of data evaluation methods is urgently needed. It provides a shared vocabulary (quality vs. trustworthiness, with named dimensions) that lets researchers report comparable results across modalities, and it flags specific dimensions that the representative literature under-covers, giving the community actionable targets.

Real-world applications (all drawn from the paper's cited use cases):

• Annotating and enriching existing data. LLM-generated topic annotations support metadata enrichment when data exists but lacks labels.
• Specialized and safety-critical domains. Generating clinical records for healthcare, and designing safety-critical scenarios for autonomous driving.
• Multi-format data augmentation. Generating text and code, tabular data, and graph data, as well as semi-structured JSON and log data for evaluation and anomaly-detection work.
• Agent, embodied, and control data. Environment and task data for driving simulation and embodied agents, control and decision data, and perception/telemetry data for simulation scenes.
• Privacy-sensitive substitution. Replacing real records in settings characterized by limited resources or privacy concerns, while metrics such as DCR, membership-inference attack AUC and advantage, and attribute-inference gain are used to check leakage.

Industry relevance. Dataset curation, decontamination, and licensing concerns map directly onto commercial training pipelines: transparent mixture design and reproducible tooling let practitioners adjust domain coverage and data budgets without conflating quantity with quality; pretraining filters aim to remove documents that enable harmful capabilities (including chemical, biological, radiological, and nuclear misuse support) while minimizing impact on standard tasks; and benchmark-aware decontamination via retrieval-based overlap analysis extends source control from quality filtering to preserving evaluation integrity.

Future Directions

The framework's Stage 6 outlines three directions, with additional open questions implied by the gap analysis:

1. Dynamic feedback-loop evaluation. Move beyond one-shot, static auditing of a synthetic dataset toward evaluation that responds as data flows back into training and generation.
2. Process verifiability. Assess not only whether the output is good but whether the generation process itself can be verified, which connects to the paper's discussion of tool-verified generation, trajectory harvesting, and post-hoc verification mechanisms such as Chain-of-Verification, RARR, and SelfCheckGPT.
3. Trust–utility trade-off analysis. Systematically characterize how gains in trustworthiness (safety, privacy, fairness) trade off against data utility, rather than treating the two pillars as separate checklists.
4. Closing the modality-specific evaluation gaps. The audits identify which dimensions are marked as partial or absent in the representative sample; turning those under-covered dimensions (such as provenance, fairness, robustness, or benchmark contamination in particular modalities) into routinely reported metrics is the natural next step.

Target Audience

Researchers and practitioners who build or consume synthetic datasets produced by LLMs and need to judge their quality: dataset-construction and pretraining teams, evaluation and benchmark designers, and researchers working on data-centric machine learning, trustworthy AI, or modality-specific synthesis (text, reasoning/code, tabular, graph/JSON/log, vision–language, agents). It is also useful for reviewers and standards authors who need a shared taxonomy and a checklist of which evaluation dimensions a paper using generated data should report. Readers seeking hands-on methodology will find the accompanying repository useful; readers seeking a single-modality deep dive will find the framework's cross-modal view a complement rather than a replacement for existing modality-specific surveys.

Authors’ abstract

Large Language Models (LLMs) have emerged as powerful tools for generating data across various modalities. By transforming data from a scarce resource into a controllable asset, LLMs mitigate the bottlenecks imposed by the acquisition costs of real-world data for model training, evaluation, and system iteration. However, ensuring the high quality of LLM-generated synthetic data remains a critical challenge. Existing research primarily focuses on generation methodologies, with limited direct attention to the quality of the resulting data. Furthermore, most studies are restricted to single modalities, lacking a unified perspective across different data types. To bridge this gap, we propose the \textbf{LLM Data Auditor framework}. In this framework, we first describe how LLMs are utilized to generate data across six distinct modalities. More importantly, we systematically categorize intrinsic metrics for evaluating synthetic data from two dimensions: quality and trustworthiness. This approach shifts the focus from extrinsic evaluation, which relies on downstream task performance, to the inherent properties of the data itself. Using this evaluation system, we analyze the experimental evaluations of representative generation methods for each modality and identify substantial deficiencies in current evaluation practices. Based on these findings, we offer concrete recommendations for the community to improve the evaluation of data generation. Finally, the framework outlines methodologies for the practical application of synthetic data across different modalities.