TRACER 結合 ICU 影片與病歷,協助及早識別譫妄徵兆
AI-Enabled Video System Monitors ICU Patients for Early Signs of Delirium
研究團隊開發 TRACER,結合 ICU 牀邊影片與電子健康紀錄追蹤患者活動變化,協助醫護及早識別譫妄徵兆。系統每分鐘抽樣15類「原子動作」,再整理成逐小時報告;測試中動作偵測準確率為85%,醫師查詢答對率為61%。在280名 ICU 患者中,譫妄篩查呈陽性者的「目的性活動」較少;研究團隊表示,TRACER 未使用患者資料訓練。
AI-Enabled Video System Monitors ICU Patients for Early Signs of Delirium
Date
November 20, 2026
Topics

Computer vision experts have developed a 24/7 video monitoring system that tracks patient activity and helps doctors identify subtle signs of delirium.
Patients in intensive care units (ICUs) are surrounded by life-saving machinery, yet many experience a frequently overlooked state of confusion doctors call delirium. It’s among the most common complications for ICU patients, especially those on ventilators, and, left unchecked, it is linked to cognitive impairments and an increased risk of death. Yet because doctors on the front lines typically check patients only a few times a day, early signs of delirium go undetected. Ironically, the most severe form of the condition is actually a lack of activity, making it nearly impossible to spot using traditional question-and-answer methods.
Against that backdrop, a team of computer vision specialists and physicians set out to develop an AI-enabled ICU video monitoring system. Known as TRACER (Trajectory Reasoning Across Clinical Event Records), the system maintains a 24-hour vigil in the ICU. By integrating a patient’s electronic health records with continuous visual analysis, TRACER tracks their movements and evaluates how their activity changes over time. The team’s research was funded in part by the Stanford Institute for Human-Centered AI (HAI).
“TRACER helps doctors and nurses to better understand and ask questions about how patient behavior has changed over the course of the hospital stay,” said co-first author Zane Durante, a doctoral student advised by Fei-Fei Li, professor of computer science and a computer vision expert, and Ehsan Adeli, assistant professor of psychiatry and behavioral sciences. “Integrated with knowledge from the patient’s medical records, TRACER can help understand a patient’s physical state and map it to their mental states and human behavior over time to flag the possibility of delirium faster and more decisively.”
“This was a problem-driven solution, not a hammer in search of a nail,” said co-first author Paul Tang, a physician and adjunct professor at Stanford’s Clinical Excellence Research Center. “Most patients on a ventilator develop delirium during their ICU stay, and for many, cognitive impairment may last for years. By detecting signs of delirium early, TRACER could enable interventions that make a life-changing difference for patients.”
24/7 Observation
TRACER includes a queryable bedside video viewer designed to assist clinicians during rounds. The interface allows medical teams to ask questions, watch patient activities on a timeline, review drug and treatment data from the patient’s medical records, review footage that TRACER has flagged for human evaluation, and quickly navigate to timestamped video clips.
A new AI system called TRACER watches and analyzes days of ICU bedside video to help clinicians answer detailed questions about a patient's care — tracking movements, medications, and behaviors hour by hour to build a fuller picture of what happened and when.
“Our lab has long been interested in teaching machines to understand human behavior in long-form videos to help patients and clinicians,” said senior author Li. “Bringing that capability into the ICU is exactly the kind of high-impact setting where AI and computer vision can benefit humanity.”
The primary challenge to detecting delirium is the constraint on time and attention. During routine rounds – which typically last only a few minutes per patient – clinicians evaluate cognitive status by asking basic questions, such as What day is it? and Who’s the president? It can be difficult to detect delirium even when the patient is able to respond, but it becomes nearly impossible when they are incapacitated. Still, the TRACER team says vital indicators of a patient’s mental state are hidden in their movements – and, notably, in their inactivity – and critical to the system’s analytical approach.
Round-the-clock, AI-enabled video monitors that can “see” and interpret human motion while synthesizing written and numerical medical records offer a compelling alternative. The challenge, however, is wrestling with the sheer volume of data – both in terms of storage and the compute needed to analyze it. A TRACER system includes two cameras, trained on an ICU bed, that generate 15 frames per second of high-resolution video, 24 hours a day. Given that the average ICU stay lasts three-and-a-half days, and two cameras are needed to prevent the medical staff or patient from blocking the view, the resulting data volume is substantial.
The researchers mitigated these demands by reducing the amount of footage for processing and the sampling frequency. TRACER reviews a smaller subset of 15 “atomic actions” – defined as the smallest meaningful behavioral observations the system can make, such as hand-to-face movements, talking, or lying still – sampled at one-minute intervals. These observations are then compressed into hourly reports and visualized across the entire hospital stay, supported by the AI agent’s analysis of the patient’s health records. At the bedside, the doctor can query the system and review videos to assess patient activity and make a diagnosis.
“TRACER continuously analyzes video and clinical records to understand patient behavior over multiple days. Clinicians can then query patient activity through TRACER’s multimodal interface to support decision-making,” said co-first author Keshigeyan Chandrasegaran, a doctoral student advised by Fei-Fei Li and Juan Carlos Niebles.
TRACER includes reporting, timeline, video, and EHR tools to aid the medical team. In the paper, the authors provide an example of the voice-activated query tool in which the physician asks TRACER: “How did the patient’s behavior change across the past three days?” To which TRACER responds: “The patient’s activity peaked last night (8 p.m.–4 a.m.) with more sporadic activity earlier.”
Promising Prototype
In a test environment, TRACER achieved 85% action-detection accuracy and answered physicians’ queries correctly 61% of the time. An impressive result, the team acknowledges, that leaves them with room to optimize a promising prototype. Notably, in a cohort of 280 ICU patients, TRACER showed that those who screened positive for delirium exhibited markedly lower levels of “purposeful activity,” such as looking at others or communicating. That insight can be leveraged to refine the system in future iterations.
Importantly, the researchers stress that TRACER is “training free” – a plug-and-play model that understands human activity across patient populations at any ICU. It can also be easily updated as new models evolve. No patient data was used to train TRACER, allowing it to be shared with other ICUs without raising patient privacy concerns.
“This is an important distinction,” says physician Tang. “Building TRACER on a foundation model is intended to reduce the risk of one-off solutions that do not generalize well to other institutions – a persistent challenge for clinical AI.”
The research team is now refining the platform, and TRACER points toward a future where computer vision and AI improve clinical capabilities and extend observation around the clock, transforming how critically ill patients are evaluated, diagnosed, and cared for.
Read the preprint study, “TRACER: A Multimodal Harness for Patient Care in Hospitals.”
Share
Link copied to clipboard!
Contributor(s)
Andrew Myers
Related News
Measuring What Medical AI Can Actually Do
Nigam Shah, Jonathan Chen
Oct 19, 2026
News

Over three years, we’ve built progressively harder, more realistic benchmarks for medical AI. Here’s what we learned.
News

Measuring What Medical AI Can Actually Do
Nigam Shah, Jonathan Chen
Foundation ModelsHealthcareOct 19
Over three years, we’ve built progressively harder, more realistic benchmarks for medical AI. Here’s what we learned.
AI Companions May Worsen Loneliness for Vulnerable Users, Stanford Study Finds
Alexander Gelfand
Aug 04, 2026
News

Stanford research finds that users with limited social networks who seek emotional support from AI companions experience lower well-being.
News

AI Companions May Worsen Loneliness for Vulnerable Users, Stanford Study Finds
Alexander Gelfand
HealthcareGenerative AIAug 04
Stanford research finds that users with limited social networks who seek emotional support from AI companions experience lower well-being.
The Complexities of Governing Mental Health AI
Caroline Yee, Caroline Meinhardt, Michelle Mello, Jane Paik Kim
Jul 24, 2026
News

Policymakers, academics, healthcare providers, AI developers, and patient advocates convened by Stanford HAI identify critical gaps in how we regulate AI tools used for therapy and emotional support.
News

The Complexities of Governing Mental Health AI
Caroline Yee, Caroline Meinhardt, Michelle Mello, Jane Paik Kim
HealthcarePrivacy, Safety, SecurityGenerative AIRegulation, Policy, GovernanceJul 24
Policymakers, academics, healthcare providers, AI developers, and patient advocates convened by Stanford HAI identify critical gaps in how we regulate AI tools used for therapy and emotional support.
Navigate
Participate
Stay Up To Date
Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.
© Stanford University. Stanford, California 94305.
Pause Media
AI-Enabled Video System Monitors ICU Patients for Early Signs of Delirium | Stanford HAI
來源:Stanford HAI News · hai.stanford.edu