JoHan:影片條件式生成的 2D-3D 聯合手部動作復原
Video-Conditioned Generative Joint 2D-3D Hand Motion Recovery
JoHan 是一種可直接從影片序列復原手部動作的生成框架,會聯合生成對齊的 2D 和 3D 局部手部姿態序列,毋須依賴逐幀姿態預測。基準測試顯示,該方法提升局部手部姿態和相機空間重建的準確度與速度,生成的動作亦較以往方法平滑,同時保持較高的逐幀姿態準確度。
Published on Oct 7
·
Submitted by
on Oct 12
Authors:
,
,
,
,
Abstract
Recovering faithful 3D hand motion from video remains challenging due to frequent occlusions and incomplete visual observations, which make frame-wise pose estimates unreliable and temporally inconsistent. To address this problem, we propose JoHan, a unified generative framework that recovers hand motion directly from video sequences without relying on intermediate per-frame pose predictions. Trained from scratch, our model jointly generates aligned 2D and 3D local hand pose sequences by learning their temporal dynamics and cross-representation correspondence. The generated 2D trajectories exploit direct spatial and temporal cues from the 2D images to guide the following generative 3D motion reconstruction, while the learned motion prior promotes temporal consistency. Their learned 2D-3D correspondence further enables recovery of the hand's global position and orientation relative to the camera. Extensive experiments on challenging benchmarks demonstrate significantly improved accuracy and speed in local hand-pose and camera-space reconstruction. Notably, our method captures much better hand-motion dynamics, producing significantly smoother motion than previous methods while maintaining high per-frame pose accuracy.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.10512
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.10512 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.10512 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.10512 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co