跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-06AI 評分44

Sensor-Language-Action Models

Sensor-Language-Action Models

AI 導讀

研究提出感測器—語言—行動(SLA)建模框架,將多模態感測器觀察、自然語言與行動整合至同一模型。研究建立涵蓋逾 116,000 人、79 種感測器模態及 60 個行動組別的基準,並推出用於階層式行動預測、狀態理解及行動解釋的 OpenSLA。OpenSLA 在臨牀預測、手術室及代謝健康等真實任務中表現優於最先進方法。

正文

Published on Oct 6

Authors:

,

Abstract

Sensors are useful not only for understanding the world but also for deciding what to do next. Existing sensor models however largely stop at perception: they recognize states or predict outcomes, leaving actions modeled separately through task-specific and often closed label spaces. We introduce Sensor-Language-Action (SLA) modeling, a framework that connects multimodal sensor observations, natural language, and actions within a unified model. SLA uses language as a semantic interface between sensing and acting, allowing heterogeneous actions to be represented, predicted, and explained while remaining grounded in the underlying sensor evidence. We build a large-scale SLA benchmark consisting of datasets that span more than 116,000 individuals, 79 sensor modalities, and 60 action groups, together with a multi-faceted captioning pipeline that aligns user context, sensor dynamics, and action evidence. Building on this framework, we present OpenSLA, a unified SLA model for hierarchical action prediction, state understanding, and action explanation. Extensive experiments on real-world tasks in clinical prediction, operating rooms, and metabolic health verify its superior performance over the state-of-the-art. OpenSLA also demonstrates intriguing capabilities including language-guided evidence grounding and zero-shot generalization to unseen actions and cohorts.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.08244

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

yang-ai-lab/OpenSLA

Question Answering •

Updated 6 minutes ago

•

1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.08244 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.08244 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co