The Pulse
Stanford’s TRACER Turns ICU Video Into Hourly Patient Records
Stanford researchers introduced TRACER, an AI system that combines ICU video with electronic health records to summarize patient activity over time. In a study spanning 280 patients and about 60,000 camera-hours, the team found an associati
AI.info Team · Written with AI ·
Stanford researchers introduced TRACER in a paper posted October 15, describing a system that turns bedside ICU video and electronic health records into a searchable timeline of patient behavior. The tool records activity minute by minute, compiles hourly summaries alongside medication events, and lets a language-model agent answer questions by retrieving records and inspecting relevant video. A Stanford Artificial Intelligence Laboratory blog post about the project followed on October 16.
The researchers frame TRACER as a way to help clinicians review changes across a hospital stay, not as an automated diagnostic system. They report finding an association between lower levels of purposeful activity and positive delirium assessments in a cohort of 280 patients. The study does not establish that the system can diagnose delirium or that changes in activity cause it.
From ICU video to minute-by-minute records
TRACER was built around video collected at Stanford Hospital from August 2021 through June 2025. Sixteen fixed cameras covered eight rooms in the cardiovascular and medical intensive care units, recording two views per room; the footage was linked to structured electronic health records, including medication administrations and nursing assessments. The researchers say the study received approval from Stanford’s institutional review board.
For each minute, a vision-language model examines sampled video frames and labels 15 clinician-informed actions, such as patient movement, communication, contact with clinicians and interaction with objects. It also produces captions describing the room. A language model combines those records with medication events to form hourly summaries, while a separate agent can search the timeline, query the medical record and inspect selected video clips.
The workflow addresses a practical scale problem: the median stay in the study cohort lasted 3.3 days, and some stays reached two weeks. Rather than ask a model to process an entire stay as one enormous video, TRACER builds a hierarchy of brief records and summaries, then retrieves evidence relevant to a clinician’s question.
Two benchmarks test detection and recall
The team tested action detection on 397 annotated one-minute clips drawn from 26 patient stays. Across the 15 action categories, Gemini 3.5 Flash achieved 84.8% accuracy and a micro-F1 score of 68.6. The paper says the less expensive Gemini 3.1 Flash Lite reached 82.3% accuracy and 62.2 micro-F1; the researchers used that model for the larger cohort analysis.
A second benchmark, TRAJECTORY-144, tested questions about 264 hours of video and medical records from two ICU stays. With Gemini 3.1 Flash Lite as the reasoning model, TRACER scored 61.0% accuracy, compared with 49.3% for the strongest baseline using the same model. The best TRACER result, 76.9%, came from Gemini 3.5 Flash. The authors note that open-weight models performed relatively close to proprietary models on reasoning, while proprietary models did better at recognizing actions in video.
“Our results show that combining continuous video with clinical records can support understanding of patient behavior over time and across patient cohorts without training on protected health data.” Zane Durante et al., authors of “TRACER: A Multimodal Harness for Patient Care in Hospitals”
A delirium association, not a diagnosis
Across the 280-patient cohort, TRACER processed about 60,000 camera-hours. The researchers found significantly fewer predicted purposeful actions among patients with at least one positive delirium assessment during their stay than among those without one; the paper reports a p-value below 0.0001. Of the cohort, 53 patients had a positive assessment and 227 did not.
That result is a group-level association, not evidence that TRACER can identify delirium in an individual patient or replace bedside assessment. The authors present the behavioral pattern as a possible starting point for studying digital measures of activity. Early clinicians and nurses who saw the interface offered encouraging feedback, according to the Stanford project post, but the publication does not report a clinical trial showing improved outcomes.
Patient recordings stay private
The paper says the video and linked medical records cannot be released publicly. Stanford’s team instead designed the method so its general components—including the action taxonomy, timeline structure, tools and prompts—can be shared without publishing protected patient records or model parameters fitted to them.
That distinction sets the immediate boundary of TRACER: the paper demonstrates a research system for organizing and querying sensitive ICU data, with a measured signal associated with delirium assessments. It does not establish a deployed hospital product or a validated delirium screening tool.