跳到正文
DEV Community· BinaryGarge.dev·· 15 小時前AI 評分45

RoadMate:能察覺駕駛者疲倦的免提副駕駛

RoadMate: a hands-free co-pilot that notices when you're getting drowsy

AI 導讀

RoadMate 是一款 Android 駕駛監測應用程式,利用手機前置鏡頭及 Presage SmartSpectra SDK 察覺駕駛者疲倦、分心和壓力,並以免提方式發出警示。眼睛閉合約 1.5 秒會觸發警告,PERCLOS 追蹤逐漸累積的疲勞,並即時量度脈搏、呼吸及 HRV。警示會由提示音升級至語音及警報,並可在 Google Maps 導航時以子母畫面持續監測。

正文

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

RoadMate turns the phone already sitting on your dashboard into a co-pilot that keeps an eye on you.

Drowsiness creeps up on drivers, and the person behind the wheel is usually the last to notice. RoadMate watches through the front camera for the moments that matter, eyes that stay closed, a gaze that drifts off the road, stress building up, and speaks up before they turn into a problem. It's built around one rule: the driver never has to look at or touch the screen.

What it does:

  • Microsleep alerts: eyes closed for ~1.5 s trigger a warning within about two seconds.
  • Fatigue tracking: PERCLOS, the share of time your eyes were closed over the last minute, catches gradual tiredness before a microsleep happens.
  • Eyes on the road: notices when your face leaves its normal position for a few seconds.
  • Live vitals from the camera: pulse and breathing with live waveforms, plus heart-rate variability (HRV), all from Presage SmartSpectra. No wearable needed.
  • Escalating, hands-free alerts: a soft chime first, then a spoken warning, then (for drowsiness only) a loud alarm that keeps going until you're alert again. Alerts duck your music and navigation and work in silent mode.
  • Break coaching: after two hours of driving, sustained eye closure, or a second drowsy episode, RoadMate says so out loud and shows a one-tap Find a rest stop button that opens Maps.
  • Trip summary: when you stop, you get trip length, when a break would have been wise, fatigue and stress trends, average vitals, and a timeline of every alert and how far it escalated.
  • Works next to navigation: switch to Google Maps and RoadMate shrinks into a picture-in-picture window with the live camera and status, and keeps monitoring.
  • Privacy blur: one switch blurs your face in the on-screen preview, which is handy with passengers or when recording. The measurements still run on the sharp camera frames.

It's an Android app (React Native / Expo) with a native Kotlin module around the SmartSpectra Android SDK.

Demo

The demo uses the real SDK on a Pixel 7. My face is blurred in the recording using RoadMate's own Blur camera preview setting. Only the on-screen picture is blurred; SmartSpectra still measures from the sharp camera frames, so the pulse, breathing and HRV you see are live.

The app also has a Simulated drive switch (Settings → Demo) that runs a scripted drive with an animated stand-in driver, so you can see every alert without an API key or a camera.

Code

RoadMate

A hands-free driver wellness co-pilot. RoadMate turns a phone on the dashboard into a driver monitor: it watches for closed eyes, growing fatigue, distraction and stress through the front camera, reads pulse, breathing and HRV with the Presage SmartSpectra SDK, and warns the driver by voice, so they never need to look at or touch the screen.

Android, React Native (Expo SDK 58), with a local Expo native module for SmartSpectra.

Features

Microsleep detection Eyes closed for 1.5 s (on-device ML Kit eye state) raise a drowsiness alert within ~2 s.
Fatigue (PERCLOS) Share of time the eyes were closed over the last 60 s, the standard driver-fatigue measure; rising PERCLOS raises drowsiness.
Eyes off the road Face missing, turned away or out of position for 2.5 s.
Live vitals Pulse and breathing with live waveforms, HRV (RMSSD), from SmartSpectra's camera-based measurement.
Escalating alerts Chime → spoken warning →

…

Try it with your own Presage API key

RoadMate doesn't ship with a key, so you can build it with yours and see the real measurements:

  1. Get a free key at the Presage Developer Portal.
  2. Install Node.js 20+, JDK 17 and the Android SDK, and connect an Android 9+ phone with USB or Wireless debugging.
  3. Clone the repo, add your key, and run:
npm install
cp .env.example .env    # then put your key in .env: EXPO_PUBLIC_PRESAGE_API_KEY=...
npm run android         # builds and installs RoadMate on your phone
  1. Open RoadMate, go through the short onboarding, allow the camera, and tap Start monitoring. Pulse appears after ~30 s and HRV after ~60 s.

The full guide, with troubleshooting, is in the README. There's no login.

No key? Turn on Settings → Demo → Simulated drive to see every alert with simulated signals and an animated stand-in driver.

npm test   # unit tests for the alert escalation, trip summary and break coaching

How It Responds to Human Signals

RoadMate uses these live SmartSpectra signals:

Signal What RoadMate does with it
Pulse rate and pulse waveform Shown live with a waveform, averaged in the trip summary, and an unusually high pulse contributes to the stress score.
Breathing rate and breathing waveform Live waveform and rate; slow, shallow breathing contributes to the drowsiness score.
HRV (RMSSD) Shown live; a very low HRV contributes to the stress score.
Facial expressions Angry, contempt and disgust confidence feed the stress score; sustained stress gets a calm spoken suggestion to ease off and breathe.
Blinking Feeds the drowsiness score.
Face validation status When the face is missing, turned away or out of position for 2.5 s: "Please look at the road."
Camera frames Drawn natively as the live preview, and checked on the phone for closed eyes (see below).

How the experience responds in real time. A native module scores the driver's state from these signals: alert, distracted, drowsy, stressed or looking away. It re-sends that state every 500 ms. A small state machine in TypeScript requires each state to persist before acting on it, so a single blink or a glance at the mirror doesn't trigger anything. It then escalates within the episode: chime → voice → alarm. When the driver recovers, the alarm stops and RoadMate says "Thank you. Back to normal."

Integrating the SDK. The SmartSpectra Android SDK lives in a local Expo module (modules/smartspectra), so it's autolinked and survives expo prebuild --clean:

sdk.config.apiKey = apiKey
sdk.config.imageOutputEnabled = true
sdk.config.enableTelemetry = false
sdk.config.requestedMetrics = SmartSpectraConfig.breathingMetrics +
  SmartSpectraConfig.cardioMetrics +
  listOf(MetricTypesProto.MetricType.EXPRESSIONS, MetricTypesProto.MetricType.BLINKING)
sdk.useCamera(CameraSelection.Front)
sdk.metrics.observeForever(metricsObserver)            // pulse, breathing, HRV, expressions, traces
sdk.validationStatus.observeForever(validationObserver) // face position -> "looking away"
sdk.imageOutput.observeForever(frameObserver)          // preview + eye-state detection
sdk.start()

Things I learned getting it road-ready:

  • Partial updates. The SDK sends partial metric updates: most carry only face data, and cardio and breathing arrive now and then. RoadMate keeps the last real value instead of letting the UI flicker back to zero.
  • Smooth preview. The live preview is a native view that draws imageOutput frames directly. My first version passed base64 JPEGs through JavaScript and managed about one frame per second.
  • Closed eyes. SmartSpectra's blinking metric reports short blinks, not how long the eyes stay shut, and updates pause when the eyes close. So for microsleeps I run Google ML Kit's on-device face detector on the SDK's own frames, about 6 times a second, to get an eye-open probability. A watchdog keeps evaluating state even when the SDK goes quiet.
  • Camera recovery. If Android takes the camera away mid-trip, a watchdog notices that frames have stopped and restarts the session.
  • Picture-in-picture. React Native stops drawing while the activity is paused, so the PiP view is drawn natively.
  • Privacy blur that doesn't cost accuracy. The blur is a GPU RenderEffect on the native preview view. It changes what's drawn on screen, not the frames, so SmartSpectra and the eye detector still analyze the sharp image.

Prize Categories

Best Thoughtful Real-World Use

  • The need: fatigue and distraction are a real risk on long drives, and the people most exposed (commuters, delivery and rideshare drivers, anyone on a long highway trip) rarely notice their own tiredness in time. Most phones already sit on a dashboard mount; RoadMate puts that camera to work without new hardware.
  • Designed for someone who's driving:
    • No touching required: everything important is spoken. Alerts clear themselves; there's nothing to dismiss.
    • Proportionate: a gentle chime first, voice next, and the loud alarm only if the driver stays asleep.
    • Works with navigation through picture-in-picture, because nobody drives with a wellness app full-screen.
    • Short, kind, actionable messages: "Ease off the accelerator and take a slow breath", plus a one-tap rest-stop search.
    • Honest trip summary: it tells you when you should have stopped, which is the habit that actually prevents the next tired drive.
    • Privacy in a shared car: with passengers, or a rideshare customer in the back, the driver may not want their face on a screen everyone can see. One switch blurs the preview, and monitoring keeps working.
  • Responsible by default: a three-step onboarding explains what RoadMate does, how to mount the phone safely, and how data is handled before the camera is ever switched on. Monitoring only runs while the driver has explicitly started it, with a visible LIVE badge.

Consent and Data Handling

Camera consent. On first launch, RoadMate shows a three-step onboarding:

  1. What the app does.
  2. How to mount the phone.
  3. Privacy and safety: it's a wellness aid, video is never stored, and setup happens before driving.

Only after that does it ask for Android's camera permission. A Blur camera preview setting blurs the face on screen and in picture-in-picture, for passengers and screen recordings. The camera is used only while the driver has tapped Start monitoring, shown by a LIVE badge (and in picture-in-picture, the live camera itself). Tapping Stop turns it off. Demo mode doesn't use the camera at all.

What happens to the data:

  • Camera frames are passed to the Presage SmartSpectra SDK during a session to measure pulse, breathing, HRV and expressions. RoadMate never writes video or images to storage and never uploads them itself.
  • Privacy blur (optional). Settings → Privacy → Blur camera preview blurs the face in the on-screen preview and the picture-in-picture window, for passengers or screen recordings. It changes only what's displayed: nothing about the analysis or the data flow changes. Before monitoring starts, the regular camera view is hidden instead, because it can't be blurred.
  • Closed-eye detection uses Google ML Kit face detection running on the phone. Frames aren't sent to Google. ML Kit may send anonymous usage and performance statistics to Google, as described in Google's ML Kit documentation.
  • SmartSpectra telemetry is turned off. The SDK has telemetry enabled by default. RoadMate sets enableTelemetry = false, so no session summaries or device-readiness reports are sent. Per Presage's Telemetry and Privacy docs, telemetry never includes raw frames or measured physiological values in any case. The SDK still authenticates with my Presage API key to run.
  • Storage and retention:
    • Measurements and the trip summary live in memory for the current trip only. They're discarded when you tap Done or close the app.
    • The only things written to the phone are two settings: "onboarding completed" and "blur camera preview".
    • Uninstalling the app, or clearing its storage, deletes them.
  • No sharing, no accounts, no analytics. There's no RoadMate server, no sign-in, no ads or tracking, and no AI or LLM services.
  • Alerts are spoken by the phone's built-in text-to-speech engine. Only fixed alert phrases are spoken; they contain no personal data.
  • Find a rest stop opens your maps app with a generic "rest area" search. RoadMate doesn't send your location anywhere.

Not a medical device. RoadMate is a wellness and situational-awareness aid. It doesn't diagnose or treat anything and doesn't make medical decisions. Pulse, breathing and HRV are shown for awareness only. It can't replace rest, and it's designed to be set up before driving, never while driving.

Agent Session

I built RoadMate with Claude Code as a pair programmer, including:

  • debugging on a real phone over Wi‑Fi
  • reading logcat to work out why closed eyes weren't detected
  • moving the native code into an Expo module
  • making the alerts hands-free

來源:DEV Community · dev.to