Echo-CoPilot: A Multiple-Perspective Agentic Framework for Reliable Echocardiography Interpretation

AI summary

Overview

Research area: Medical AI / agentic large language model systems applied to echocardiography (cardiac ultrasound) interpretation.

Technical level: Intermediate. The paper assumes familiarity with large language models, tool-using (ReAct) agents, knowledge graphs, and echocardiography measurement terminology (ejection fraction, LVEDV, chamber dilation).

Scope: Echo-CoPilot is an end-to-end agentic pipeline that runs three independent ReAct-style reasoning agents over echocardiogram videos, constrains their measurement choices with a guideline-derived knowledge graph called EchoKG, and resolves disagreements through a self-contrast language model — evaluated zero-shot on the MIMICEchoQA benchmark.

What This Paper Is About

Echocardiography interpretation requires combining multi-view video evidence, numerical measurements, and guideline-based criteria, but current foundation-model pipelines solve isolated subtasks and break down when tool outputs are noisy or when values sit near clinical cutoffs. The authors build Echo-CoPilot, which treats interpretation as a multi-step procedure: three specialized agents reason independently, a knowledge graph dictates which measurements are required or should be avoided, and a contrast module arbitrates conflicts before producing a final answer. The goal is both higher diagnostic accuracy and more reproducible answers across repeated runs.

Key Contributions

1. EchoKG, an echocardiography knowledge graph distilled from consensus guidelines (e.g., ASE, EACVI), modeled as a directed graph with Structure nodes, Measurement nodes (such as LVEDV and EF), and Pattern nodes comprising 16 distinct clinical query templates. It encodes both "requires" and "avoid" edges, so it can negatively constrain measurement selection rather than only suggesting metrics.

2. A multiple-perspective agentic framework with three parallel ReAct agents — Structural, Pathological, and Quantitative — that reason over the same exam, share a deterministic tool cache for consistency, and are guided by EchoKG at inference time.

3. A self-contrast mechanism in which a dedicated contrast LLM compares the three perspectives, generates a structured discrepancy checklist, re-queries EchoKG for guideline thresholds, and weights perspectives that followed that guidance more heavily than a simple majority vote.

4. An empirical evaluation reporting state-of-the-art accuracy on MIMICEchoQA and a stochasticity stress test showing improved diagnostic stability and fewer answer flips versus single-trajectory baselines.

Main Findings

• Accuracy on MIMICEchoQA: Echo-CoPilot reaches 54.0 ± 0.7 accuracy, the best reported in Table 1. For comparison, GPT-o4-mini scores 43.9, Qwen2-VL-7B-Biomed scores 49.0, GPT-4o scores 41.6, and MedGemma-4B scores 33.4. Gains are described as largest on questions requiring quantitative reasoning or integration of multiple cues, such as grading systolic dysfunction from ejection fraction, assessing hypertrophy severity, and stratifying pericardial effusion.

• Stability under repeated runs: Over 10 runs on 50 questions (500 total inferences), Echo-CoPilot achieves a 72.0% stability rate with 1.22 average unique answers and 0.96 average answer changes. The ReAct Agent baseline reaches 70.0% stability with 1.30 unique answers and 1.14 changes; a plain LLM reaches 40.0% stability (1.86 unique answers, 2.10 changes); naive LLM+Tools is worst at 26.0% stability (2.00 unique answers, 2.58 changes).

• Component ablation: Accuracy rises steadily as components are added — LLM 46.9, LLM+Tools 49.7, ReAct Agent 52.3, and full Echo-CoPilot 54.0.

• Error analysis by region: Compared with Qwen2-VL-7B-Biomed, Echo-CoPilot cuts error on Doppler views from 83.0% to 59.3%, on PSAX great vessels from 71.0% to 59.6%, and on the left atrium from 68.4% to 60.5%.

• Error correction example: For the query "Is there aortic stenosis?" (ground truth: No), one perspective mistook a tool confidence value of 84% for a probability of disease presence and voted Yes, while the other two voted No based on the categorical output "No stenosis detected." The self-contrast layer flagged the conflict, used EchoKG to prioritize the categorical label, and recovered the correct answer.

• Bottleneck identified: In the 500-iteration stability test, pure LLM logic errors occurred in less than 1% of cases. Tool variability is described as the primary bottleneck — when perceptual tools return noisy or conflicting data, the framework can produce reasoning that is internally consistent but factually incorrect.

Methodology in Plain English

The authors frame echocardiography interpretation as a procedure rather than a single prediction. First, they build EchoKG by extracting logic from clinical guidelines: which anatomical structures exist, which standard measurements apply, and which measurements must be used or avoided for each of 16 clinical query patterns. Patterns are evaluated by priority (1 = most specific, 3 = most general) so specific questions are matched before general ones, with keyword and exclusion rules controlling matches.

For a given question and video, three agents with different system prompts each run their own ReAct loop — deciding which tool to call, observing the output, and reasoning toward an answer. All three share the same deterministic tool cache, so an identical tool call returns an identical result, and all three can query EchoKG for measurement selection rules and clinical thresholds. The available tools include EchoPrime for view classification and measurement prediction, PanEcho for disease and finding prediction, report-style synthesis, and EchoKG itself. Perception is delegated to vision-language tools; the agents themselves are LLM-driven.

After the three perspectives finish, a contrast LLM receives all perspective responses, the aggregated tool measurements, the EchoKG guidance and thresholds, and an instruction to produce an explicit discrepancy checklist. It then writes a final answer that resolves each checklist item, applies EchoKG thresholds for severity classification, and gives more weight to perspectives that adhered to EchoKG guidance. Experiments use gpt-oss-120b as the LLM with LangChain/LangGraph for structured function calls, run on a single NVIDIA A100 (80GB) GPU, with contrast temperature 0.3 and perspective temperature 0.7, and a maximum of 10 reasoning steps per ReAct loop. MIMICEchoQA is used strictly for zero-shot evaluation, so no training occurred on it.

Why This Matters

Impact on research: The paper argues for moving medical AI from black-box prediction toward guideline-constrained, auditable reasoning, and shows that reasoning architecture (multi-perspective + contrast) can be added on top of existing perceptual tools without retraining them. It also reframes the field's bottleneck: the authors report that LLM logic errors were rare (under 1%), while tool variability dominated the remaining failures — a diagnostic signal for where future medical agent research should invest.

Real-world applications:

• Decision support in echocardiography labs, where guideline-anchored measurement selection could reduce the chance of reporting the wrong metric for a given clinical question.
• Triage and screening workflows in settings with limited specialized readers, where a reproducible second opinion on questions like systolic function grading or pericardial effusion severity is valuable.
• Quality assurance and audit, since the discrepancy checklist and EchoKG constraints make the reasoning trace inspectable rather than opaque.
• Safety monitoring near guideline cutoffs, where small measurement variations can flip a severity classification — the exact scenario the stability test targets.

Industry relevance: The framework composes commercially and openly available components (EchoPrime, PanEcho, an open-source LLM, LangChain/LangGraph) rather than training a new model from scratch, which lowers the barrier to adoption by vendors building echo reporting or clinical decision support software. The 54.0 versus 49.0 comparison against the strongest biomedical baseline (Qwen2-VL-7B-Biomed) is described by the authors as a modest accuracy margin accepted as a trade-off for explainability and verifiable guideline adherence.

Future Directions

• Improving tool reliability: The authors state that future work in agentic echocardiography must prioritize the deterministic reliability of underlying measurement tools, since tool variability — not reasoning — is the reported bottleneck.
• Broadening benchmarks: Evaluation is limited to a single large-scale cohort; the authors call for curating diverse, multi-center benchmarks to test generalization, citing the scarcity of public video-based benchmarks.
• Prospective clinical validation: The conclusion names advancing toward prospective clinical validation as a future effort.
• Open question raised by the results: How to raise the overall performance ceiling given that the architecture resolves reasoning conflicts but cannot correct perceptual errors that all perspectives inherit — and whether the modest accuracy margin over the strongest baseline justifies the added pipeline complexity in deployment.

Target Audience

Researchers and practitioners working on multimodal medical AI, clinical decision support, and tool-augmented LLM agents, particularly those focused on cardiology or ultrasound. It is also relevant to clinical informatics teams and medical device developers evaluating whether multi-agent architectures are mature enough for guideline-constrained diagnostic assistance, and to benchmark designers interested in how zero-shot stability under stochastic decoding should be measured.

Authors’ abstract

Echocardiography interpretation requires integrating multi-view temporal evidence with quantitative measurements and guideline-grounded reasoning, yet existing foundation-model pipelines largely solve isolated subtasks and fail when tool outputs are noisy or values fall near clinical cutoffs. We propose Echo-CoPilot, an end-to-end agentic framework that combines a multi-perspective workflow with knowledge-graph-guided measurement selection. Echo-CoPilot runs three independent ReAct-style agents, structural, pathological, and quantitative, that invoke specialized echocardiography tools to extract parameters while querying EchoKG to determine which measurements are required for the clinical question and which should be avoided. A self-contrast language model then compares the evidence-grounded perspectives, generates a discrepancy checklist, and re-queries EchoKG to apply the appropriate guideline thresholds and resolve conflicts, reducing hallucinated measurement selection and borderline flip-flops. On MIMICEchoQA, Echo-CoPilot provides higher accuracy compared to SOTA baselines and, under a stochasticity stress test, achieves higher reliability through more consistent conclusions and fewer answer changes across repeated runs. Our code is publicly available at https://github.com/moeinheidari7829/Echo-CoPilot.