The Denario project: Deep knowledge AI agents for scientific discovery

AI summary

Note on scope: The available paper text covers the Abstract and Sections 1 through 3.3. Sections 4 through 8 — running the system, the AI-generated papers themselves, the domain-expert evaluations, failure modes, ethics, and conclusions — are not present in the provided content, so their specific results and numbers are not reported here.

Overview

• Research area: AI for scientific discovery, specifically multi-agent systems built on large language models (LLMs) that act as research assistants.
• Technical level: Intermediate. The paper assumes familiarity with LLMs, prompting, and multi-agent orchestration concepts, but explains its architecture from the ground up.
• Scope (one sentence): The paper introduces and details Denario, a modular multi-agent system that can generate research ideas, check the literature, plan projects, write and execute code, make plots, and draft and review scientific papers.

What This Paper Is About

Scientific research is a slow, multi-stage process — forming a hypothesis, reviewing the literature, planning the analysis, writing and debugging code, producing figures, and writing the paper — and much of that time goes to activities that LLM-based AI agents can assist with. The authors describe Denario, a modular AI multi-agent system designed as a scientific research assistant that can perform any one of these tasks individually or run them sequentially for end-to-end research. The paper aims to document the system's architecture and modules, and (per the abstract) to illustrate its capabilities with AI-generated papers across many disciplines judged by domain experts.

Key Contributions

1. The Denario system itself: a modular AI multi-agent framework, built with AG2 and LangGraph, in which each module targets one scientific task and modules can be composed for end-to-end research.
2. Six-module architecture with defined interfaces: Idea, Literature, Methods, Analysis, Paper, and Review modules, each with specified inputs and outputs (for example, input.md to idea.md, and results.md plus plots to paper.pdf).
3. Model-agnostic LLM backend: Denario lets users pick among models including gpt-5, gpt-5-mini, gpt-4o, gpt-4.1, gpt-4.1-mini and o3-mini (OpenAI); gemini-2.5-pro, gemini-2.5-flash and gemini-2.0-flash (Google); and claude-3-7-sonnet, claude-4-Opus, claude-4.1-Opus and claude-4.5 (Anthropic), with new models able to be incorporated.
4. A Planning & Control orchestration strategy, inherited from Cmbagent, used as the deep-research backend and as an alternative implementation of idea generation, with tunable hyperparameters such as the number of review rounds, plan steps, and message rounds.

Main Findings

• A six-module pipeline with explicit data contracts: The Idea module generates a project idea; Literature judges whether the idea is new; Methods produces a research plan; Analysis implements the plan by writing and executing code and making plots; Paper drafts the manuscript; Review produces a referee report. Outputs are stored as idea.md, literature.md, methods.md, results.md plus a Plots folder, paper.pdf, and referee.md.
• Modularity means human input can be inserted anywhere: Denario can run end-to-end from an input text file, or a single module can be run alone — for instance, given a user-supplied idea and plan, the Analysis module can write code and make plots.
• Two implementations of idea generation with very different costs: The Fast idea path (LangGraph) produces a final idea after three iterations between idea_maker and idea_hater, followed by a further idea_maker iteration, taking around 15 seconds with gemini-2.0-flash. The Planning & Control path (Cmbagent) uses gpt-4o and claude-3-7-sonnet and takes about 1 minute and roughly $0.20 for Planning plus about 3 minutes and roughly $0.90 for Control — roughly $1 and roughly 4 minutes per idea.
• Planning & Control hyperparameters: The number of review rounds is generally set to one, because more rounds often produced overly complex and ineffective plans; plans typically contain between 3 and 8 steps; and the hard limit on agent-to-agent messages defaults to 500 rounds, after which the system fails.
• Control-stage failure handling: A record_status function tracks the current step, whether it is completed, failed or in progress, and whether new plots or code were produced or code execution failed; the session aborts if code-execution failures exceed a user-set maximum, and ends positively when the final step succeeds.
• Input text quality critically drives performance: The authors give explicit guidelines, including detailed column/row descriptions with units and ranges, absolute data paths, "Do not create dummy data" to prevent the system from inventing synthetic data, "Join plots that are similar" to avoid repetitive figures, resource statements such as "You have access to 1 GPU and 10 CPUs", and prohibitions such as "Do not use X library".
• Automatic paper ingestion from arXiv URLs: If arXiv URLs appear in the input text, Denario downloads the PDFs, OCRs them with mistral_ocr_lattest, stores the markdown locally, sends the full markdown to o3-mini for summarization, and appends the summary to the input text. The GUI exposes this through an "Enhance data description" button.
• Literature checking uses a query loop: In the Semantic Scholar implementation, a novelty agent can conclude the idea is new, is not new, or request more information; if it needs more, a semantic scholar agent issues an API query and returns titles and abstracts for re-evaluation, repeating until the relevant literature is sampled; a summary agent then writes the report. If no relevant papers are found after a number of iterations, the novelty agent concludes the idea is new.
• LLM selection was heuristic, not systematically benchmarked: The authors state that model choice so far came from trial and error rather than systematic evaluation; for coding their primary models are gemini-2.5-pro, gpt-4.1 and o3-mini, and for critiquing and reviewing their default is Claude models or reasoning models from OpenAI.
• Prior art is acknowledged: The paper positions Denario alongside earlier systems including the robot scientists Adam (functional genomics) and Eve (drug discovery), the Automated Statistician, Sakana AI-scientist, Google co-scientist, AI-cosmologist, Curie, Agent laboratory, AutoWEKA, ResearchAgent, AstroAgents, AgentRxiv, and self-driving or virtual labs for enzyme re-engineering and nanobody design.
• Motivating evidence cited for multi-agent idea generation: The authors cite prior work suggesting AI-generated ideas may be judged more novel than human-generated ones by human evaluators, reinforcement learning between agents for idea generation, and a finding that a large team of agents produced better ideas than a single agent.

Methodology in Plain English

The authors build Denario as a collection of cooperating agents rather than one monolithic program. Two orchestration frameworks handle the coordination: AG2, an agent framework organized around conversation patterns and recursive agent composition, and LangGraph, which routes computation through a graph of nodes connected by edges and passes a shared state object between them. Some modules exist in both frameworks (the idea module), while others exist in only one (the analysis module).

For the deep-research behavior, they adopt the Planning & Control strategy from Cmbagent. A plan_setter chooses which agents participate; a planner proposes a plan while a plan_reviewer critiques it; the agreed plan is a list of subtasks, each with the subtask description, the agent responsible, and bullet-pointed instructions. A control agent then walks through the subtasks, recording status after each one and deciding where to hand off next, ending through a terminator agent.

Idea generation is framed as a propose-critique loop: an idea_maker proposes, an idea_hater critiques, and the maker revises. The Semantic Scholar literature module similarly loops between a novelty judge and a search agent until enough evidence exists to rule on originality.

Everything is driven by a user-written input text, which the paper treats as the single most important lever on output quality — hence the detailed writing guidelines.

Why This Matters

Impact on research: The authors frame Denario not as an attempt to automate science but as a research assistant intended to accelerate discovery, letting researchers explore a wide range of possibilities at a more superficial level through AI while they go deeper into the most promising ones. The paper explicitly targets the tradeoff between depth and breadth under a fixed time budget.

Real-world applications (as described or illustrated in the paper):

• Accelerating literature review and automated novelty checking of a proposed research idea against existing publications.
• Generating candidate research directions from a dataset description, with a critique loop that filters out weak or infeasible ideas — the paper shows this with a Perovskite database containing 95 attributes and 42,400+ devices.
• Running analyses directly: writing, executing and debugging code, producing plots, and summarizing results from a user-supplied idea and plan.
• Drafting manuscripts and producing referee-style reports, and transferring methods across fields — the abstract highlights a paper applying quantum physics and machine learning methods to astrophysical data.

Industry relevance: Denario is released publicly with code and a web demo, and the authors state the full app will be deployed on the cloud. Its model-agnostic design links it to the commercial LLM ecosystem (OpenAI, Google, Anthropic), and its manuscript-writing and reviewing modules touch directly on scientific publishing and peer review workflows.

Future Directions

• Systematic model evaluation: The authors state model choices were made by trial and error; a systematic comparison of LLMs per task is an obvious open need.
• Improving robustness against failure modes: The paper defers a discussion of observed failure modes and limitations to Section 6, which is not included in the provided content; preventing dummy-data creation, unreadable-library use, and excessive similar plots are among the issues already surfaced by the input-text guidelines.
• Cost and latency reduction: The gap between the 15-second Fast idea path and the roughly 4-minute, roughly $1 Planning & Control path invites work on cheaper or better-targeted planning strategies.
• Evaluating the scientific value of AI-generated papers: The abstract reports domain-expert evaluations with both numerical scores and review-like feedback across many disciplines, but the actual evaluation outcomes are not reported in the available content.

Target Audience

This paper is most useful to AI researchers and engineers building multi-agent LLM systems, to computational scientists who want an assistant for literature review, coding and analysis, and to research groups, publishers and institutions weighing the role of AI in scientific production. Readers interested in the ethics and philosophy of AI-driven research are addressed in later sections of the paper that are outside the provided text.

Authors’ abstract

We present Denario, an AI multi-agent system designed to serve as a scientific research assistant. Denario can perform many different tasks, such as generating ideas, checking the literature, developing research plans, writing and executing code, making plots, and drafting and reviewing a scientific paper. The system has a modular architecture, allowing it to handle specific tasks, such as generating an idea, or carrying out end-to-end scientific analysis using Cmbagent as a deep-research backend. In this work, we describe in detail Denario and its modules, and illustrate its capabilities by presenting multiple AI-generated papers generated by it in many different scientific disciplines such as astrophysics, biology, biophysics, biomedical informatics, chemistry, material science, mathematical physics, medicine, neuroscience and planetary science. Denario also excels at combining ideas from different disciplines, and we illustrate this by showing a paper that applies methods from quantum physics and machine learning to astrophysical data. We report the evaluations performed on these papers by domain experts, who provided both numerical scores and review-like feedback. We then highlight the strengths, weaknesses, and limitations of the current system. Finally, we discuss the ethical implications of AI-driven research and reflect on how such technology relates to the philosophy of science. We publicly release the code at https://github.com/AstroPilot-AI/Denario. A Denario demo can also be run directly on the web at https://huggingface.co/spaces/astropilot-ai/Denario, and the full app will be deployed on the cloud.