Idlewild:將呼吸轉化為動畫與聲音
Idlewild: Turn Your Breath into Animation and Sound
Idlewild 是一款供動畫師和遊戲製作者使用的桌面表演工作室,採用 Presage SmartSpectra SDK,將用戶的呼吸、動作及生理訊號帶入角色動畫與場景。用戶可錄製一次表演,套用至不同角色或場景,並匯出影片、程序化聲音、帶有呼吸及脈搏曲線的 GLB,以及供動畫工具使用的 CSV 曲線。攝影機須經應用程式內同意才會啟用;畫面不會儲存或上載,追蹤在裝置上進行。
This is a submission for the Presage SmartSpectra SDK Challenge
What I Built
Idlewild is a desktop performance studio for animators and game makers. You breathe, move and perform in front of a webcam; SmartSpectra measures your chest breathing and, when your chest and face are well framed, breathing rate and pulse rate; on-device tracking follows your face and body, and a character performs with you. Record once, replay the same performance on any character, then export video, procedural sound, a baked GLB with real breathing curves and CSV curves for your animation tools. Idle loops and cutscenes get breathing captured from a real person instead of a sine wave.
Demo
Code
Idlewild
A desktop performance studio for animators and game makers, built on the Presage SmartSpectra SDK with Electron, TypeScript and Three.js. Breathe, move and perform in front of your camera: SmartSpectra measures your chest breathing, breathing rate and pulse rate while on-device tracking follows your face and body. The character breathes with you. Record the performance once, replay it on any character or scene, and export video with a live signal panel, procedural sound, a baked GLB with breathing and pulse curves, and CSV curves for your animation tools.
How SmartSpectra drives the studio:
| SmartSpectra signal | What it does |
|---|---|
| Chest breathing trace | Characters' chests, shoulders and heads breathe; bubbles rise on inhale and release on exhale; the star glows; the sound's breath layer swells. |
| Breathing rate | Slow posture and scene pacing within comfortable bounds. Faster breathing is never rewarded. |
| Pulse rate | Slow color warmth and glow; shown with its trend. |
…
How It Responds to Human Signals
- Chest breathing trace drives chest, shoulder and head motion on Rex, Sprite and the Rain runner; bubbles rise on inhale and release on exhale; the star glows; the sound's breath layer swells. The trace has no declared units, so Idlewild centers and scales it against its own recent range.
- Breathing rate sets slow, bounded scene pacing. Faster breathing is never rewarded, and the runner's pace is chosen, not breath-driven.
- Pulse rate, when SmartSpectra reports it, warms colors and glow and is shown with its trend. It is never turned into fake heartbeats.
- Pulse waveform (uncalibrated arterial trace), when reported, adds a fine shimmer and a subtle tremolo.
- Blink and talking move eyelids and jaw when no face is tracked locally. HRV and expression estimates are opt-in.
- Every signal carries its quality and confidence onto an on-stage panel that is also composited into exported video. Missing or stale input fades out and is never invented. SmartSpectra frames and local MediaPipe tracking from the same camera are merged onto one timeline.
Prize Categories
Most Creative or Delightful Use; Best Thoughtful Real-World Use.
Consent and Data Handling
- Nothing starts at launch. The camera opens only after in-app consent; SmartSpectra needs your own key and a separate metered-measurement consent (18+).
- Keys stay in main-process memory for the session and are never written to disk. Camera images are never saved, uploaded by Idlewild or exported. No microphone is used; all audio is synthesized.
- SmartSpectra sends camera-derived data to Presage for processing. Idlewild disables SDK telemetry and never requests insights, but the SDK may generate automatic vitals insights that upload data and Idlewild cannot disable them; vendor retention is governed by Presage's terms and is not verified by Idlewild.
- Takes live in memory until you export them. Exports that include measured breathing and pulse are personal data, and you choose where they go. Face and body tracking runs entirely on the device.
- Creative animation, not medical measurement: no diagnosis, no health claims, no covert monitoring.
來源:DEV Community · dev.to