scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery

AI summary

Overview

Research area: Artificial intelligence for science — large language model (LLM) reasoning applied to single-cell RNA sequencing (scRNA-seq) analysis, spanning cell-type annotation, developmental-trajectory reconstruction, and gene-regulatory network (GRN) inference.

Technical level: Advanced. The paper assumes familiarity with scRNA-seq tooling (Scanpy, Seurat, Monocle, pySCENIC), standard evaluation metrics (AUROC, graph-edit distance, Jaccard), and LLM prompting paradigms.

Scope in one sentence: The paper introduces scPilot, a framework in which an off-the-shelf LLM reasons directly over raw single-cell data via natural-language hypotheses and on-demand bioinformatics tool calls, together with scBench, a nine-task benchmark for measuring that capability.

What This Paper Is About

Single-cell biology has moved from bulk averages to datasets of roughly 10^5–10^6 cells, yet the analytical pipeline still depends on implicit, human-only reasoning: analysts choose tools, set hyperparameters, and interpret outputs in ways that are rarely recorded or auditable. Existing computational approaches sit at two extremes — LLM "tool agents" that write and execute code but whose reasoning lives only in comments and chat text, and foundation models that embed cells into opaque high-dimensional vector spaces with no explanations.

The authors propose "omics-native reasoning" (ONR): an interactive loop in which an LLM receives a compact textual summary derived from the expression matrix, states biological hypotheses in natural language, invokes targeted bioinformatics operations on the raw data, evaluates the numerical evidence, and iteratively revises its conclusions. scPilot is the framework that operationalizes this idea, and scBench is the benchmark built to measure whether it actually works.

Key Contributions

1. An LLM-driven single-cell analysis framework. scPilot formulates the first omics-native reasoning, language-centric workflow that automates cell-type annotation, trajectory inference, and gene-regulatory network prediction while preserving scientific transparency. It combines a problem-to-text converter, an LLM planner/reasoner, and a curated bio-tool library.

2. A comprehensive benchmark suite. scBench comprises nine expertly curated single-cell tasks with task-specific metrics and expert-verified ground truth, covering three workflows across three datasets each, enabling objective comparison of LLMs on biologically meaningful problems rather than embedding-quality proxies.

3. Empirical validation across the benchmark. Comprehensive experiments across the nine benchmark datasets demonstrate effectiveness: the paper reports that iterative omics-native reasoning lifts average cell-type annotation accuracy by 11%, reduces trajectory graph-edit distance by 26%, and improves GRN prediction AUROC by 0.03 over direct prompting and conventional baselines. (The abstract states a 30% graph-edit distance reduction for Gemini-2.5-Pro versus one-shot prompting and the results section reports an average +0.098 AUROC improvement over direct prompting; these figures differ across sections of the paper.)

4. Biological interpretability and diagnostic reasoning. scPilot generates transparent reasoning traces that expose marker ambiguities, lineage inconsistencies, and tissue-specific regulatory logic, enabling biologically interpretable and diagnostically informative analyses.

Main Findings

Cell-type annotation improves, especially on Retina. Implementing the scPilot pipeline improved accuracy for 19 out of 24 model–dataset combinations. The Retina dataset showed the most significant median accuracy gain (+0.180), followed by PBMC3k (+0.042) and Liver (+0.024); the gain on Retina is attributed to iterative reasoning that differentiated rod photoreceptors, Müller glia, and bipolar cells via dotplot expression reasoning, which the one-step direct approach struggled with when limited to top marker genes.

Best cell-type annotation results. scPilot implementations using o1 and Gemini-2.0-Pro ranked highest (0.792 on PBMC3k; 0.728 and 0.763 on Retina; 0.518 and 0.509 on Liver). Under direct prompting, o1 achieved the highest overall accuracy (0.667 on PBMC3k, 0.560 on Liver, 0.474 on Retina). Against non-LLM and other LLM baselines in Table 2, scPilot (o1) reached 0.792 on PBMC and 0.728 on Retina versus CellTypist (0.563, 0.388), GPTCellType (0.613, 0.300), CellMarker 2.0 (0.250, 0.632), and Biomni with Gemini-2.5 Pro (0.646, 0.570); on Liver, Direct (o1) scored 0.560 versus scPilot (o1) at 0.518.

Trajectory reconstruction beats both Biomni and py-Monocle. In Table 6, scPilot with Gemini-2.5-Pro achieved Jaccard 1, GED-nx (10s) of 3.33 ± 2.31, and Spectral Distance 0.199 ± 0.033, compared with Biomni (Gemini-2.5 Pro) at Jaccard 1, GED-nx 8.33 ± 3.21, Spectral Distance 0.482 ± 0.379, and py-Monocle at Jaccard 1, GED-nx 20, Spectral Distance 0.469. Table 6 labels GED-nx with an upward arrow, whereas Table 4 labels GED-nx with a downward arrow; the paper presents the scPilot result as superior. When adopting the scPilot pipeline, structural errors were further reduced in 10 of 21 model–metric pairs (median improvements: GED −2.0, spectral distance −0.14), with Gemini-2.5-Pro consistently optimal, closely followed by Gemini-2.0-Pro. Under direct prompting, o1 obtained perfect Jaccard scores (1.000) and superior structural accuracy on Pancreas and Neocortex.

GRN prediction surpasses trained GNNs and LLM baselines. On the Stomach dataset, scPilot (o1) reached AUROC 0.873 ± 0.004 versus Direct (o1) at 0.827 ± 0.002, LLM4GRN at 0.727 ± 0.025, GCN at 0.723 ± 0.071, GraphSAGE at 0.713 ± 0.063, GAT at 0.683 ± 0.071, and BioGPT-Large-PubMedQA at 0.660. Averaged across the three tissues, scPilot demonstrated an average AUROC improvement of +0.098 versus direct prompting. GPT-4o exhibited the greatest relative improvement (+0.162 average AUROC). The o1 model under scPilot achieved the highest overall accuracy (AUROC 0.873 stomach, 0.760 liver,

Authors’ abstract

We present scPilot, the first systematic framework to practice omics-native reasoning: a large language model (LLM) converses in natural language while directly inspecting single-cell RNA-seq data and on-demand bioinformatics tools. scPilot converts core single-cell analyses, i.e., cell-type annotation, developmental-trajectory reconstruction, and transcription-factor targeting, into step-by-step reasoning problems that the model must solve, justify, and, when needed, revise with new evidence. To measure progress, we release scBench, a suite of 9 expertly curated datasets and graders that faithfully evaluate the omics-native reasoning capability of scPilot w.r.t various LLMs. Experiments with o1 show that iterative omics-native reasoning lifts average accuracy by 11% for cell-type annotation and Gemini-2.5-Pro cuts trajectory graph-edit distance by 30% versus one-shot prompting, while generating transparent reasoning traces explain marker gene ambiguity and regulatory logic. By grounding LLMs in raw omics data, scPilot enables auditable, interpretable, and diagnostically informative single-cell analyses. Code, data, and package are available at https://github.com/maitrix-org/scPilot