Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories

AI summary

Overview

Research area: Agentic AI for computational journalism — a multi-agent system that turns a raw dataset into a finished, evidence-traceable multimedia news article. Although listed under Computer Vision on arXiv, the work spans language agents, data analysis, interactive web design, and evaluation methodology.

Technical level: Intermediate. The system architecture is described conceptually rather than with deep implementation detail, but the evaluation protocols assume familiarity with LLM-as-judge setups, claim-level coverage metrics, and code-based verification.

Scope (one sentence): The paper introduces Data2Story, a seven-role "virtual newsroom" of agents that converts datasets into multimodal, verifiable web articles, and evaluates those articles against 18 human-written pieces from three publication sources using four complementary protocols.

What This Paper Is About

A single high-quality data-journalism feature can take a newsroom team weeks of work: hunting background context, running statistics, deciding on an editorial angle, designing visuals, and building the page. Existing AI agents handle individual steps well — data-science agents can close the analysis loop, and design agents can generate websites — but they do not take raw data all the way to a story a reader will finish and trust, and they rarely expose where any given number came from.

The paper's goal is to build an end-to-end agent that does both: produce a multimedia article and make nearly every claim in it traceable back to a specific line of code, a data source, or an external URL. The authors explicitly position the system as an assistant to journalists rather than a replacement.

Key Contributions

1. A seven-role virtual newsroom. Data2Story orchestrates a Detective (web context gathering), an Analyst (runs real code to compute statistics), an Editor (decides the angle and drafts the prose outline), a Designer (selects multimodal assets via tool calls), a Programmer (renders the final HTML, in assembly and revision modes), an Auditor (flags visual and structural defects such as overlapping elements, broken charts, missing assets, unresponsive interactions), and an Inspector (links final claims to evidence).

2. Claim-level verifiability through the Inspector. Every published fragment of the article — a sentence, chart, or interactive element — is bound to upstream evidence. The paper distinguishes two link types: code evidence, where a claim traces to the specific script and line that produced it, and reference evidence, where a contextual claim is grounded in an external URL. The paper frames this as establishing auditability and traceability rather than factual correctness.

3. Multimodally generative articles instead of plain text and static charts. Rather than defaulting to text, the Designer reasons about what the reader will want to see and selects media to fit both the topic and audience — an interactive map with zoom for a geography piece, an audio clip for a music piece, a playable demo for a card-game piece.

4. A four-axis evaluation protocol with 18 paired articles. Each dataset is paired with the originally published expert-written piece, and the agent is scored on human–agent angle coverage, a rubric study with human judges, computer-use agents as judges, and verifiability.

Main Findings

• Human-authored articles retain a clear edge on editorial angle, creative design, and informative presentation. This is stated as the paper's central contrast: Data2Story is competitive overall, but humans are judged stronger on these three fronts.

• The agent's strongest advantage is auditability. The authors report particularly strong performance on transparency and auditability dimensions, and argue that Data2Story makes the evidentiary basis of each claim explicit and measurable in a way that even carefully crafted human articles rarely provide natively.

• Writing volume is roughly comparable, but the sentence structure differs. Across the 18 paired articles, total writing volume is 1305 for Data2Story versus 1557 for humans. The agent uses 1.45× as many sentences, each shorter (0.77×), producing more granular statements.

• The agent cites the human reference article, but the density depends heavily on source. 14 of 18 articles cite the human reference at all. The density is highest for The Pudding (44.3%), where the human article essentially is the dataset (data-as-article), and far lower for The Economist (15.5%) and TidyTuesday (16.3%).

• Claim-level coverage points one way. About half of the human article's angle (50.4%) lands in the agent's article, while only about a third of the agent's angle (35.1%) maps back to the human piece.

• Coverage is widest exactly where grounding is lowest. Coverage peaks on The Economist's short briefings (Δ = 73.0% − 39.5% = 33.5%), whose narrow single-topic scope makes them easy for the agent to predict and cover from primary data without leaning on the original. Coverage stays uniformly lower on The Pudding, whose articles carry a single editorial thesis the agent does not fully reproduce, and on TidyTuesday, which spans diverse topics and external sources.

• The agent discovers original angles on data with no human reference. Applied to three 2026-era datasets:

  • FIFA 2026 schedule ("One Tournament, Sixteen Climates"): the agent fuses venue geography with climate data (Open-Meteo) and FIFPRO heat-risk flags, finding that roughly four in ten matches are booked at venues FIFPRO flags as "extremely high risk," and that humidity, not air temperature, drives the worst penalties. The interactive weather map is the article's centerpiece.
  • arXiv submissions 1991–2026: computer science is now 42.5% of everything posted, and in May 2025 it crossed half of all submissions in a single month for the first time. The chart traces output to a first-ever 30,000-submission month in March 2026, and ties the surge to arXiv's January 2026 change stopping treating an institutional email as enough to endorse a first-time submitter.
  • Time-use diaries (Multinational Time Use Study): women do more than twice men's unpaid work, and once paid and unpaid hours are summed, they work longer days overall. Read by decade, "screen time" rose while paid work and housework fell, and the gender gap narrowed because men slowly did more at home — the "work-time invariance" pattern, where total work society performs barely moves and only its division shifts.

• Rubric scores, overall reader preference, and verifiability pass rates are not reported in the available content. The paper states that human raters judge the articles favorably across multiple quality dimensions, but the specific numbers behind the 53-participant study and the verifiability test are beyond the truncated text provided.

Methodology in Plain English

The system takes any raw data source as input and emits a generative multimedia article, with no predefined angle or preference supplied — Data2Story decides for itself what is worth reporting.

The pipeline runs like a newsroom. The Detective augments the raw dataset through web search into an enriched corpus of context items, each tagged with a category and source URL, plus reference media the other agents can reuse. The Analyst deliberately prioritises completeness: it enumerates every analysis the dataset can support, profiles every column, and runs actual code rather than asking the model to estimate — and every result carries a pointer to the script that produced it. The Editor then treats the findings as raw material, ranking them by priority, selecting what to keep, cutting the rest, and drafting a paragraph-level prose outline where each paragraph is annotated with the findings and code it draws on.

The Designer looks at each planned finding and decides which medium a reader would most want, calling external generative tools (text-to-image, text-to-video, and so on) while storing every prompt and parameter. The Programmer renders the page as HTML rather than PDF, since static formats cannot natively coordinate multimedia — it generates no new facts or numbers, working in assembly mode (quote upstream artifacts into a complete interactive article) and revision mode (apply the Auditor's suggestions). The Auditor reviews the rendered page for visual and structural defects and sends it back for repair.

The Inspector then decomposes the audited page into self-contained HTML fragments — a sentence, a chart, an interactive element — and binds each to the evidence entries that ground it, producing a page whose truthfulness is evidence-traceable back to the original data file or source reference.

For evaluation, the authors curate 18 examples from three stylistically distinct sources: The Economist (concise analytical economics-style reporting), The Pudding (artist-first long-form interactive essays), and TidyTuesday (diverse community datasets with processing code). The set spans 2018–2026, multiple domains (science, media, sports, politics, health, culture, music, food, tech, climate), and several data modalities including time series, panel, geospatial, tabular, and text. Every example pairs the underlying data with the human-written reference piece.

They then measure four things. Angle coverage parses articles into factual claims, filters out non-article content with gpt-4o-mini, retrieves the top-3 nearest candidates using OpenAI's text-embedding-3-small by cosine similarity, and lets gpt-4o-mini decide whether a pair covers the same topic — yielding directional scores for how much of the human angle the agent covers and how much of the agent angle the human covers. Rubric evaluation recruited 53 reviewers via the Prolific platform; each reviewer is assigned one Data2Story–human pair (presentation order randomised and blind) and scores both along five dimensions on a 1–7 scale: visual design, narrative and pacing, data and method transparency, claim–data alignment, and insight value. Computer-use agents as judge replace the human reader with OpenAI's browser-use gpt-5.5-xhigh, which navigates the rendered page by clicking and scrolling rather than reading static screenshots. Verifiability uses an across-family verifier, OpenAI's Coder codex-GPT-5.4, which extracts factual statements, re-executes Python or R scripts against the raw dataset for computational claims, and re-fetches cited source URLs for reference-supported claims.

The Data2Story articles themselves were produced using Claude Code with claude-opus-4.7. The authors acknowledge potential training-data contamination, since well-known Economist and Pudding articles may appear in pretraining corpora and web-search scope is hard to control. They argue that under their framework, merely recalling a human article earns no credit: coverage is bidirectional, rewarding claims the human article omits; and human articles ship no code, so a memorised angle cannot pass verifiability, which is checked by re-running code against the data.

Why This Matters

Impact on research. The paper makes claim-level provenance a first-class output of a generative agent rather than an afterthought. Prior systems in the comparison table — search agents such as MindSearch, MMSearch, and DR Tulu; visualization agents such as MatplotAgent, LIDA, and CoDA; data-science agents such as DSGym, Data Interpreter, and AI Scientist; and the LLM-writer data-journalism baseline — are each missing at least one column that Data2Story marks present. In particular, Data2Story is the only system in that table marked as shipping runnable code and producing reader-interactive output. That reframes how agentic outputs might be evaluated in general: not just "is it good" but "can each claim be traced."

Real-world applications:

• Newsroom story development. Assisting reporters on labour-intensive computation and graphics while humans set perspective and editorial judgment.
• Specialised, data-rich coverage. Producing stories on topics that newsrooms do not have the bandwidth to cover, as demonstrated on the FIFA 2026 schedule, arXiv submission trends, and multinational time-use diaries.
• Audit and fact-checking workflows. Providing an explicit evidence chain from published claim back to code line or source URL, which editors could check directly.
• Scaling verified reporting at platforms already publishing AI-generated news. The paper notes companies such as CitizenPortal and Locunity already deploy AI agents to produce articles at scale, and identifies the shared lack of verification and traceability as the critical challenge.

Industry relevance. The work sits at the intersection of publishing and AI tooling, with affiliations spanning University of Oxford, Stanford University, and BBC R&D. The paper's framing is deliberately non-displacement: humans set perspective and editorial judgment, while agents handle computation and graphics design and open the door to coverage that would otherwise go unwritten.

Future Directions

• Evaluation with professional journalists. The authors state plainly that their study measures reader reception, not newsroom adoption, and that robust evaluation with practising journalists requires expert recruitment and realistic editorial settings — which they treat as future work.
• Closing the gap on editorial angle, creative design, and informative presentation, the three dimensions where human-authored articles retain a clear edge.
• Reducing contamination risk and disentangling memorisation from genuine analysis, given that web-search scope is difficult to control and well-known articles may reside in pretraining corpora.
• Extending the coverage-versus-grounding analysis. The paper shows per-source coverage is the inverse of grounding density and that the two views "separate two" — a thought left unfinished in the available text — suggesting a closer study of when leaning on a human reference helps versus when independent primary-data analysis does better.

Target Audience

Researchers working on multi-agent systems, tool-using LLM agents, and provenance or auditability in generative outputs will find the architecture and the Inspector mechanism most directly useful. Computational-journalism researchers and practitioners evaluating AI in newsroom workflows will find the three-source evaluation set and the four-protocol rubric design the most relevant part. Evaluation researchers interested in human-versus-agent judging, and in computer-use agents as a substitute for static screenshot judging, will also benefit. Readers looking for headline benchmark numbers should note that the rubric results, preference rates, and verifiability pass rates are not contained in the available content.

Authors’ abstract

Data tells stories that shape society; the data journalist's job is to turn raw information into stories non-experts can trust. A high-quality news feature takes a newsroom team weeks: hunting for context, running statistics, choosing an angle, and designing visuals. Recent agents handle individual steps well: data-science agents close the analysis loop, while design agents synthesize beautiful websites. But can an agent serve as a data journalist end to end? We introduce Data Journalist Agent (Data2Story), a multi-agent framework that orchestrates specialized roles into a single virtual newsroom. Data2Story contributes two innovations. (i) Claims are evidence-grounded: an Inspector links every number, angle, and asset back to data, code, or an external reference. (ii) Articles are multimodally generative: rather than defaulting to plain text and static charts, Data2Story reasons about what readers will want to see, then deploys multimodal tools, such as interactive maps for geography and audio for music. We evaluate Data2Story on 18 articles, each paired with the originally published expert piece, along four axes: (a) human-agent angle coverage; (b) rubric evaluation with 53 participants across five dimensions; (c) computer-use agents as judges, a cost-saving proxy for how readers navigate interactive articles; and (d) verifiability, where a coding verifier re-executes statements against the data and checks claims against references. Data2Story produces competitive, evidence-traceable multimedia stories, with particular strength in transparency and auditability. Human articles retain an edge in editorial angle, creative design, and presentation. We position Data2Story as a collaborator for journalists, enabling more evidence-based, transparent, and verifiable reporting. Code and demos are available at https://data2story.github.io.