SutureBot: A Precision Framework & Benchmark For Autonomous End-to-End Suturing

AI summary

Overview

Research area: Surgical robotics and robot learning, specifically imitation learning (IL) and vision-language-action (VLA) models applied to autonomous robotic suturing on the da Vinci Research Kit (dVRK).

Technical level: Advanced. The paper assumes familiarity with imitation learning, hierarchical policies, VLA architectures (flow-matching action heads, LoRA, FiLM conditioning), and surgical robotics hardware.

Scope: The paper introduces a public dVRK suturing dataset and benchmark, a goal-conditioned control framework for precision needle placement, and an evaluation of four low-level imitation learning policies across needle pickup, needle throw, and knot tying.

What This Paper Is About

Robotic suturing requires a robot to grasp a needle, drive it through tissue at precise points, and tie a secure knot, all in sequence with no human help. The authors report that while individual suturing sub-skills have been demonstrated before, a fully autonomous suturing pipeline had not been demonstrated on physical hardware, and the field lacked both sufficient real-world demonstration data and a standard benchmark. SutureBot addresses these gaps by releasing a large real-world suturing dataset, defining a benchmark with precision metrics, and evaluating state-of-the-art imitation learning models on it.

Key Contributions

1. A new dexterous benchmark featuring a long-horizon suture task for evaluating imitation learning policies in a surgical environment on the dVRK.
2. The largest public real-world suturing dataset, comprising 1,890 high-fidelity dVRK demonstrations (including 454 recovery demonstrations) for reproducible research.
3. A goal-conditioned imitation learning framework that lets learned policies achieve precision-targeted needle insertion outcomes, reported as a 59%-74% improvement over a task-only baseline.
4. A comprehensive evaluation of state-of-the-art VLA models (π₀, GR00T N1, OpenVLA-OFT, and multitask ACT) on the benchmark, establishing a performance baseline.

Main Findings

• Goal representation matters: Point labels (opaque blue/green pixels overlaid on the endoscope image at the insertion and exit points) achieved the lowest average insertion error, at 1.3 mm for ACT and 1.0 mm for π₀. No-goal baselines reached 3.2 mm (ACT) and 3.9 mm (π₀).
• Statistical support for point labels: Using Mann-Whitney U tests for accuracy and Brown-Forsythe tests for precision with a Bonferroni-corrected threshold (α = 0.0167), point labels were significantly more accurate than distance maps (p = 0.010) and masks (p = 0.009) and more precise than masks (p = 0.009) and no goal (p = 0.013) for ACT. For π₀, point labels were significantly more accurate than distance maps (p = 0.010) and no goal (p = 0.002), and more precise than no goal (p = 0.007).
• ACT was the strongest low-level policy: Multitask ACT achieved 9/10 needle pickup, 8/10 throw, 4/10 pull through, and 9/10 knot tie, with a mean insertion error of 1.5 ± 0.8 mm and total procedure time of 182 ± 58 s. It was the only policy to complete end-to-end sutures: 3/10 with no manual intervention between tasks.
• Other policies trailed: π₀ achieved 7/10 pickup, 7/10 throw, 3/10 pull through, 4/10 knot tie, 1.9 ± 1.0 mm insertion error, 348 ± 45 s, and 0/10 end-to-end. GR00T N1 achieved 1/10 on each of pickup, throw, pull through, and knot tie (2.3 ± 1.2 mm insertion error, 388 ± 67 s, 0/10 end-to-end). OpenVLA-OFT achieved 0/10 on all four and 0/10 end-to-end, with insertion and exit error reported as NA.
• Significant model differences: A 4x2 Chi-squared test showed a highly significant overall difference (p = 8.4e-14). Post-hoc one-tailed Fisher's Exact tests with α = 0.0167 showed ACT significantly outperformed GR00T N1 (p = 8.8e-9) and OpenVLA-OFT (p = 2.1e-12), but not π₀ (p = 0.018).
• The high-level policy was not the bottleneck: The task-prediction policy achieved an F1 score of 0.92 and accuracy of 88.73% on offline validation, and 100% F1 and accuracy at detecting task transitions. Oracle comparisons where a human supplied subtask labels produced comparable success rates, timing, and precision to the high-level policy.
• Pretraining gave limited in-domain benefit: π₀ fine-tuned from its public checkpoint, π₀ post-trained on 20,000 predominantly cholecystectomy trajectories from SRT-H (π₀ Chole), and π₀ initialized from scratch with a PaliGemma backbone (π₀ Scratch) did not clearly separate on task success. π₀ Scratch showed only a small performance drop compared to π₀ from checkpoint. Pretraining did reduce time to convergence, with π₀ Chole converging fastest and π₀ Scratch requiring more training time than the baseline.
• Generalization is limited: π₀ performed comparably on unseen wound types (wounds two through six) versus the trained wound, while ACT dropped noticeably (from 9/10 to 5/10 pickup, 8/10 to 6/10 throw, 4/10 to 2/10 pull through, 9/10 to 5/10 knot tie). Both policies declined further under alternate lighting and alternate tools. No policy achieved end-to-end success in any generalization condition.
• Exit error stayed high: The authors report that exit error remained comparable across non-baseline goal-conditioning methods and attribute this to models lacking historical context when the needle is obscured within tissue.
• Behavioral difference by goal type: Models trained with point labels showed deliberate, hesitant motions near the target, while distance-map and mask-conditioned models completed tasks faster but less accurately.

Methodology in Plain English

The team used a da Vinci Research Kit (dVRK) Si with a Soft Tissue Suture Pad and 3-0 Ethibond green braided polyester suture. A 3D-printed trocar cage held fixed remote centers of motion, a DeBakey forceps was placed on the left arm and a large needle driver on the right, and 5.5 mm borescope wrist cameras were mounted 35 mm from each wrist. Images were recorded at 30 Hz alongside robot kinematics.

They split suturing into three tasks: needle pickup, needle throw, and knot tying. Demonstrations were collected through standard dVRK teleoperation, plus recovery demonstrations in the spirit of the DAgger framework, starting from failure states and showing how to complete the task. Across 1,890 demonstrations they collected 628 needle pickups (148 recoveries), 310 needle throws (96 recoveries), and 952 knot ties (210 recoveries). Each needle throw was annotated with insertion and exit points on the final frame using a GUI, stored as x and y image coordinates in CSV files.

Control is hierarchical. A high-level policy built on a Swin Transformer reads images and outputs a language instruction naming the current subtask. A low-level policy takes that language instruction, wrist and endoscope images, and the goal condition, and outputs a chunk of relative robot actions. The team fine-tuned π₀, GR00T N1, and OpenVLA-OFT, and trained a language-conditioned multitask ACT from scratch as a non-VLA reference point. They also attempted a language-conditioned Diffusion Policy but excluded it after failing to get reasonable performance.

Goal conditions were tested in four forms: point labels drawn directly on the endoscope image, binary masks passed as a separate three-channel input, distance maps passed as a separate three-channel input, and no goal at all.

Precision was measured physically: targets were marked with invisible ultraviolet (UV) ink, illuminated during target selection, then re-illuminated after execution so ImageJ could measure the Euclidean distance between the intended UV marks and the actual insertion and exit points. Each policy ran ten full suture procedures in a fixed robot configuration with consistent pad placement and varying needle position, evaluated on a dual NVIDIA RTX 4090 workstation. Training used an NVIDIA DGX A100 system with 8x A100 80 GB GPUs.

Why This Matters

Impact on research: The paper provides a shared, reproducible benchmark and the largest public real-world suturing dataset (1,890 demonstrations), addressing what the authors identify as a data bottleneck: prior combined end-to-end suturing datasets totaled less than 200 trajectories, while generalist policies are trained on 1 million+ trajectories. It also introduces precision metrics (insertion and exit error) beyond coarse task-completion rates.

Real-world applications:

• Autonomous or semi-autonomous surgical systems that can extend the capacity of a strained surgical workforce.
• Surgical robot data collection standards, giving other labs a reproducible dVRK setup, including released STL files for the trocar cage and camera fixtures.
• Pretraining or fine-tuning data for generalist robotic foundation models on dual-arm platforms.
• Benchmarks for evaluating imitation learning on long-horizon, deformable-object manipulation beyond surgery.

Industry relevance: The work targets the da Vinci ecosystem, uses commercially available components (3-D Med, Ethicon, Intuitive Surgical lineage hardware), and was supported in part by NVIDIA computational resources. The authors cite estimates that 67% of the world's population lacks access to surgical care and project a shortage of 10,100 to 19,900 surgical specialists in the United States by 2036, framing autonomy as a capacity-expansion lever. They also caution that this is exploratory research with performance well below expert surgeons.

Future Directions

• Scale and diversify the dataset. The authors note that generalization degrades under lighting and tool changes and would likely degrade further with different pad materials, phantom blood, or real tissue, and that expanding scale and diversity will help determine whether the bottleneck is dataset size or policy architecture.
• Automate target-point selection. The current pipeline relies on manual target-point selection via a GUI, which must be automated for full autonomy.
• Evaluate the high-level policy online. High-level policy evaluation was limited to offline and Oracle settings; the authors call for more discrete online evaluations.
• Explore alternative architectures and transfer learning. The authors note that clinical post-training did not significantly improve performance here but remains promising, and that alternative architectures, pretraining, and larger datasets are needed to push toward reliable end-to-end suturing.
• Incorporate clinical metrics. Future work should consider bite depth, tissue trauma, and suture tension rather than only success and error metrics.
• Address trial size. Each experiment used only 10 trials due to time constraints, which the authors state may be insufficient to resolve subtle effects such as those in the pretraining evaluation.

Target Audience

Researchers and engineers working on robot learning, imitation learning, and vision-language-action models who need a hard, long-horizon dexterous manipulation benchmark. It is also relevant to surgical robotics groups using the dVRK, to teams seeking real-world dual-arm demonstration data for policy training, and to clinical translation researchers interested in precision metrics for autonomy. Readers without background in imitation learning or surgical robotics will find the architecture and evaluation sections demanding, though the task decomposition and precision measurement approach are broadly accessible.

Authors’ abstract

Robotic suturing is a prototypical long-horizon dexterous manipulation task, requiring coordinated needle grasping, precise tissue penetration, and secure knot tying. Despite numerous efforts toward end-to-end autonomy, a fully autonomous suturing pipeline has yet to be demonstrated on physical hardware. We introduce SutureBot: an autonomous suturing benchmark on the da Vinci Research Kit (dVRK), spanning needle pickup, tissue insertion, and knot tying. To ensure repeatability, we release a high-fidelity dataset comprising 1,890 suturing demonstrations. Furthermore, we propose a goal-conditioned framework that explicitly optimizes insertion-point precision, improving targeting accuracy by 59\%-74\% over a task-only baseline. To establish this task as a benchmark for dexterous imitation learning, we evaluate state-of-the-art vision-language-action (VLA) models, including $π_0$, GR00T N1, OpenVLA-OFT, and multitask ACT, each augmented with a high-level task-prediction policy. Autonomous suturing is a key milestone toward achieving robotic autonomy in surgery. These contributions support reproducible evaluation and development of precision-focused, long-horizon dexterous manipulation policies necessary for end-to-end suturing. Dataset is available at: https://huggingface.co/datasets/jchen396/suturebot