VTuber Studio Pro：構建開源網絡攝影機驅動 VRM 工作室的單目追蹤經驗
Building an open-source webcam-to-VRM studio: lessons from monocular tracking
VTuber Studio Pro 開發者打造免費開源的網絡攝影機驅動 VRM 虛擬形象動畫工具，並分享單目追蹤的實作取捨。Windows 便攜版 v1.2.5 支援 VRM 0.x 和 1.0，追蹤在本機執行，亦可提供透明虛擬形象畫面供 OBS 工作流程使用。單目影像難以準確判斷深度，降低深度動作幅度雖可減少跳動，卻會犧牲反應速度。
I have been working on VTuber Studio Pro, a free, open-source app for animating a VRM avatar from an ordinary webcam. The current Windows portable release is v1.2.5. It supports VRM 0.x and 1.0 models and can provide a transparent avatar view for an OBS workflow.
The interesting part has been making a single-camera setup usable without pretending it can measure everything a depth camera can.
What the camera can observe
A webcam gives us a 2D image. Face, shoulder, arm, hand and finger landmarks can be estimated from that image, then mapped onto the avatar rig. The application runs the tracking locally. For someone trying to get started with a VRM model, this keeps the hardware requirement low: a webcam and a computer.
Where the guesswork begins
Depth is ambiguous in a monocular image. A hand moving toward the lens can look similar to a change in scale or body pose. I damp depth motion to reduce distracting jumps, but this necessarily trades responsiveness for stability. When an arm or shoulder leaves the frame, the pose also needs a fallback rather than trying to invent an exact position. Lighting and face occlusion affect expression tracking too.
These are current limitations, not features I would claim to have solved. If you test the app, I would especially like to hear about models with unusual rigs, hands near the camera, and tracking in imperfect lighting.
Try it and report a concrete failure
- Download the portable Windows v1.2.5 release: https://github.com/ydjdydjfbfuduxbxiehsjd-source/vtuber-studio/releases/tag/v1.2.5
- Load your own VRM 0.x or 1.0 avatar and enable your webcam.
- Try face movement, both arms and fingers, then a transparent OBS scene.
- In a GitHub issue, include your VRM version, camera/lighting setup, the motion that failed and what you expected.
The source and issue tracker are here: https://github.com/ydjdydjfbfuduxbxiehsjd-source/vtuber-studio
I am sharing this as an early release and would value reproducible feedback. If you have built webcam-driven avatar tools, which stability tradeoff has been hardest for you?
來源：DEV Community · dev.to