Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment

AI summary

Overview

• Research area: Computer vision for surgical video understanding, specifically fine-grained surgical skill assessment from minimally invasive surgery recordings.
• Technical level: Intermediate. The paper assumes familiarity with video backbone architectures, self-supervised visual features, point tracking, and cross-validation protocols, but its core idea (modeling motion via sparse trajectories) is intuitive and explained accessibly.
• Scope: The paper introduces tissue tension recognition as a new surgical video understanding task, releases the SurgTension benchmark of clinically annotated robot-assisted rectal cancer surgery clips, and proposes TensionTRAC, a lightweight sparse-point-trajectory framework that competes with frozen video backbones while using far less input data and compute.

What This Paper Is About

Surgical performance in minimally invasive surgery is judged largely by expert manual video review, which is slow, subjective, and hard to scale. Existing computer vision tasks for surgical video (instrument segmentation, phase recognition, action recognition) describe what is happening in the scene but do not capture how tissue is being handled — a key indicator of surgical quality. The paper formulates tissue tension (the visually apparent mechanical state produced by traction and counter-traction before or during dissection) as an event-level recognition task, builds a clinically annotated dataset for it, and proposes a trajectory-based model that reads tension from how local points in the tissue move rather than from dense global appearance.

Key Contributions

1. A new task: tissue tension recognition. The authors introduce the recognition of visually apparent tissue tension as a clinically motivated surgical video understanding task, targeting fine-grained tissue handling assessment beyond conventional phase-, action-, and instrument-centric analysis, with a focus on total mesorectal excision in robotic rectal cancer surgery.
2. The SurgTension dataset. The first clinically annotated and expert-reviewed tissue tension dataset, built from seven robot-assisted rectal cancer resection videos (11 hours, 7 minutes, 44 seconds at 30 fps), containing 3,593 valid approximately 1-second clips across five tension levels, with event-level start and end timestamps.
3. The TensionTRAC framework. A lightweight trajectory-based framework that fuses intra-trajectory motion (histogram-of-displacements descriptors), inter-trajectory motion (pairwise relative positions), and point-aligned DINOv2 semantic features through a one-block spatio-temporal transformer, shifting representation from dense video-level appearance modeling to sparse trajectory modeling.
4. Empirical validation of the trajectory hypothesis. A statistical analysis showing that 10 of 14 trajectory-derived metrics differ significantly between tension and non-tension clips at α = 0.05, plus benchmark results comparing TensionTRAC against four frozen video backbones on binary and cascade tension recognition.

Main Findings

• Trajectory statistics distinguish tension from non-tension. Of 14 trajectory metrics tested with a two-sided statistical test, 10 differed significantly at α = 0.05. Among these, velocity mean rose from 1.78 (non-tension) to 2.29 (tension) with p = 1.11 × 10⁻⁵, distance standard deviation over time rose from 13.13 to 18.43 with p = 8.69 × 10⁻⁷, max stretch rose from 2.87 to 3.49 with p = 2.34 × 10⁻⁴, and mean strain energy rose from 0.0225 to 0.0372 with p = 1.78 × 10⁻⁵. The authors interpret this as tension clips exhibiting stronger motion and deformation patterns.
• Dense appearance backbones are also strong on binary tension. Under stratified clip-level validation of binary tension classification, all methods performed similarly, with macro-F1 scores between 82.2% and 85.3%. Video Swin-3D with the MLP reached 85.3% macro-F1, and TensionTRAC with the MLP matched it at 85.3% (accuracy 88.9%, balanced accuracy 85.2%).
• TensionTRAC leads on the harder cross-video binary setting with k-NN. Under video-grouped evaluation, TensionTRAC with k-NN achieved the best accuracy, balanced accuracy, and macro-F1 among the compared methods at 82.4%, 76.4%, and 75.6% respectively. With the MLP it reached the highest balanced accuracy of 76.8% (tied with Video Swin-3D) and a macro-F1 of 75.8%.
• Cascade recognition is much harder than binary detection. End-to-end two-stage cascade results drop sharply because detection errors propagate into level grading. Under stratified evaluation, Video Swin-3D was strongest (macro-F1 62.7% with k-NN and 61.2% with MLP), while TensionTRAC obtained 59.4% (k-NN) and 58.9% (MLP), outperforming SlowFast under the stratified protocol. Under video-grouped evaluation, TensionTRAC reached 31.7% (k-NN) and 35.8% (MLP) macro-F1, with the MLP result approaching TimeSformer by a 0.4-point gap, while Video Swin-3D again achieved the strongest overall cascade performance.
• TensionTRAC is substantially more compact. Its trajectory encoder has 18M trainable parameters, processes 8 × 1 frames, and requires 35 GFLOPs, pretrained on a 10K-video subset of Kinetics-400 (K400-10K). For comparison, SlowFast uses 34M parameters, 32 × 1 frames, 36 GFLOPs, and Kinetics-400 with 240K videos; Video Swin-3D uses 88M parameters, 282 GFLOPs, and 14M images plus 240K videos; TimeSformer uses 122M parameters, 362 GFLOPs, and 14M images plus 392K videos; SurgMotion uses 304M parameters, 5,785 GFLOPs, and 15M images. TensionTRAC is thus comparable to SlowFast in GFLOPs while using roughly half the trainable parameters and one quarter of the input frames.
• Dataset label distribution is highly imbalanced. Of 3,867 annotated clips, the final benchmark retains 3,593 clips (93%): 874 non-tension (level 0, 22.6%), 545 low (14.1%), 1,349 moderate (34.9%), 749 adequate (19.4%), and 76 excessive (2.0%). Excluded were 208 reverse-tension clips (5.4%) and 66 unclear clips (1.7%). Because of this imbalance, macro-F1 is used as the primary metric, with accuracy and balanced accuracy reported alongside.
• Failure modes are identified qualitatively. Correctly classified tension clips show structured local tissue deformation, whereas non-tension clips mainly contain limited or tension-unrelated motion. Failures arise from occlusion and visual disturbances such as smoke or fluid degrading trajectory extraction, from camera motion, instrument motion, and unrelated tissue deformation producing valid but tension-irrelevant trajectories, from subtle or highly localized deformation (false negatives), and from confusion between adjacent tension levels such as low misclassified as moderate.
• Inter-rater reliability was not quantified. Annotations were produced by a sixth-year medical student under direct supervision of a gastrointestinal surgeon with 15 years of clinical experience, with ambiguous cases resolved by consensus, but the annotations were not independently performed by multiple experts.

Methodology in Plain English

The authors start from a clinical observation: when a surgeon pulls tissue with an instrument, the tissue deforms, and this deformation is visible as coordinated local movement rather than as a change in how the tissue looks. Static appearance is therefore a weak signal, and dense whole-frame video features may be dominated by instruments, background, and camera motion.

To test this idea, they uniformly sample points on a grid in the first frame of a short clip and track those points through the whole clip with a frozen point tracker, giving a set of trajectories. They then compute statistics from these trajectories (such as visibility, velocity, acceleration, displacement change, stretch, strain energy, motion coherence, and spatial displacement) and run a statistical test comparing tension-present and non-tension clips. Finding that most of these metrics differ significantly, they build a model around trajectories.

The model has three parallel streams per tracked point. The first encodes how each individual trajectory moves over time using a histogram-of-displacements descriptor. The second encodes how trajectories relate to each other by computing pairwise relative positions among all points at each timestep. The third samples semantic patch features from a frozen DINOv2 encoder at the tracked point locations in each frame, so each point carries both motion and appearance information. The three streams are projected to a shared dimension and added together, then passed through a one-block spatio-temporal transformer that first applies temporal attention along the frames of each trajectory and then spatial attention across trajectories at each frame. The resulting point tokens are pooled with visibility weighting to produce one embedding per clip.

For recognition, the authors keep the encoder frozen and evaluate only lightweight classifiers on top: a k-nearest-neighbor classifier using cosine similarity with similarity-weighted voting, and a two-layer MLP. This design isolates the quality of the representation rather than the capacity of a trained head. They evaluate binary tension detection (non-tension versus any tension) and a two-stage cascade that first detects tension and then grades it into four levels, using both stratified five-fold cross-validation over all clips and video-grouped four-fold cross-validation where all clips from a source video stay in the same fold. The dataset was constructed from de-identified da Vinci surgical platform recordings collected during routine clinical care under an opt-out policy, with black borders and lower system overlays removed before analysis.

Why This Matters

Impact on research. The paper argues that existing surgical video tasks do not correspond to how surgeons judge operative quality, because phase labels, tool categories, and global action representations do not describe tissue handling, traction, counter-traction, or safe dissection. By defining tissue tension as an event-level recognition task with clinically defined states and providing an annotated benchmark, the work opens a direction that is adjacent to but distinct from prior vision-based surgical force estimation, which estimates continuous force-related signals rather than recognizing clinically defined tension events. The trajectory-based formulation also offers a general recipe for modeling fine-grained tissue dynamics at low computational cost.

Real-world applications:

• Post-operative technical performance review, where automatically recognized tension events can support structured feedback alongside existing competency assessment tools.
• Surgical training, where trainees receive objective, timestamped feedback about whether their traction was insufficient, adequate, or excessive during dissection.
• Surgical quality improvement programs, where tension distributions across many procedures can be aggregated without manual review of full-length recordings.
• Scalable screening of operative video, since the method requires only 8 frames and 35 GFLOPs after offline extraction, making it more practical for long, high-resolution procedures than dense spatio-temporal encoders.

Industry relevance. Surgical robotics platforms such as the da Vinci system routinely record the operative field, which makes automated analysis of fine-grained tissue handling feasible at scale. A lightweight representation that runs on frozen foundation models and a compact 18M-parameter encoder is attractive to device manufacturers, hospital systems, and surgical education providers who need to process long procedure libraries without the cost of dense video transformer inference, although the paper reports benchmark results only and does not describe a deployed product.

Future Directions

• Independent multi-expert annotation. The authors state that because annotations were not independently performed by multiple experts, inter-rater reliability could not be quantified, and that independent multi-expert validation remains an important direction for future work.
• Better fine-grained tension grading. Cascade results show confusion between adjacent tension levels such as low misclassified as moderate, indicating that distinguishing levels 1 through 4 remains an open problem, particularly across videos where performance dropped substantially.
• More robust trajectory extraction and motion disentanglement. Failure cases involve occlusion, smoke, fluid, camera motion, and instrument motion generating valid but tension-irrelevant trajectories. The authors note that visibility masking and visibility-weighted aggregation help but do not fully solve severe visual degradation, and that false negatives occur when deformation is subtle or highly localized.
• Generalization across procedures and larger cohorts. The benchmark contains seven source videos and 3,593 clips from one procedure type, total mesorectal excision in rectal cancer surgery, and the paper frames this as an initial step toward clinically meaningful assessment rather than a finished system.

Target Audience

This paper is most useful to computer vision researchers working on video understanding and representation learning, especially those interested in point tracking, sparse trajectory modeling, and efficient alternatives to dense video transformers. It is also relevant to surgical data science and medical imaging researchers building benchmarks for surgical video analysis, and to clinicians, surgical educators, and quality-improvement specialists interested in objective measurement of tissue handling. Readers with a background in machine learning but no surgical domain knowledge should be able to follow the technical contribution, while the clinical framing of tension states will be most meaningful to those familiar with laparoscopic and robot-assisted rectal surgery.

Authors’ abstract

Surgical performance assessment in minimally invasive surgery largely relies on manual expert review, making it time-consuming, subjective, and difficult to scale. While existing surgical video understanding methods address tasks such as instrument segmentation, surgical phase recognition, and action recognition, they do not explicitly capture fine-grained tissue handling, a key indicator of surgical quality. To address this gap, we introduce tissue tension recognition, a new clinically motivated video understanding task for laparoscopic and robot-assisted rectal cancer surgery. To support this task, we construct SurgTension, the first expert-annotated tissue tension dataset, providing a benchmark for objective tissue tension recognition. We further propose TensionTRAC, a lightweight trajectory-based framework that models tissue tension from sparse point trajectories. Using a compact trajectory encoder, TensionTRAC achieves competitive performance against strong pretrained video backbones.