跳到正文
DEV Community· Nate Rupard·· 7 天前AI 評分44

打造 Signal2Speech:以 EEG 頭戴裝置輔助回答問題(Hack the Nest 2026 更新 1)

Building Signal2Speech: answering questions with an EEG headset at Hack the Nest 2026 (Update 1)

AI 導讀

Order of the Golden Eagles 團隊正於 Hack the Nest 2026 開發 Signal2Speech,讓使用者透過 EEG 頭戴裝置選擇、預覽並確認要説出的回覆。AI Answering 使用 Claude 按輸入問題建議回覆;只有輸入問題會傳送至 AI,不會傳送 EEG 數據。目前完整流程可透過鍵盤和 g.Pype 模擬腦訊號運作,尚未接上實體頭戴裝置。

正文

Note: This is only the first of many updates. We will also include a final post which will summarize the whole event.

We are Order of the Golden Eagles, a team of four at Hack the Nest 2026 (Eastern Michigan University), and we are building Signal2Speech. It is an accessibility prototype for people who have difficulty speaking or typing. A conversation partner asks a question, the app offers a few short replies, and the person wearing an EEG headset (a cap that reads electrical activity from the brain) picks, previews, and confirms the words they want spoken. Our main rule is that the app suggests what to say, but the wearer decides what actually gets said.

We are about five and a half hours in. This post covers where the project stands and a few things we ran into along the way.

How it works

The wearer sees two choice tiles that flash. To pick one, we use a P300 chooser. The P300 is a brain response that shows up about 300 milliseconds after the thing you’re watching flashes, so the app averages several flashes and classifies them with Linear Discriminant Analysis (LDA). The chosen reply then shows up on a “Say this?” preview, and nothing is spoken yet. To confirm, the wearer closes their eyes for about two seconds. Closing your eyes raises the brain’s alpha rhythm, which our alpha switch turns into exactly one Confirm. Only then does the app speak.

The same controls run two features. AI Answering suggests replies to a typed question using Claude, and it falls back to saved phrases like “Yes.” and “No.” if the AI is slow or offline. OpenOrbit lets the wearer explore Earth data. Our demo question is which Michigan county’s Air Quality Index (AQI) rose more when wildfire smoke arrived in June 2023. The answer comes from NASA satellite images and the EPA’s daily county AQI: Washtenaw rose 186 points, and Wayne rose 168.

The pivot

We started out planning to build in Unity with g.tec’s starter. About an hour and a half in, we switched the whole project to Python, g.Pype (g.tec’s Python toolkit for EEG), and PySide6. The main reason was that g.Pype lets judges actually see the signal pipeline, including OSCAR artifact removal (cleaning out noise from blinks and muscle movement). The selection rules we had already written carried over with no changes because they were written as a shared scenario file that every input has to pass, not built into one engine.

What works right now

The full choose, preview, and confirm flow runs end to end on keyboard input and on g.Pype’s simulated brain. OpenOrbit has its real data bundled in the repo, so the demo works with Wi-Fi off. We also added a NASA FIRMS map showing where the smoke came from. One thing we learned from that data is that southeast Michigan had no wildfire detections on either day. The fires were actually in Quebec and Ontario, and only the smoke reached Michigan.

If you want to try it without a headset, it runs on the keyboard:

git clone https://github.com/Order-of-the-Golden-Eagles/Signal2Speech.git
cd Signal2Speech
pip install -r requirements-lock.txt
python -m signal2speech.app --input keyboard

Press 1 or 2 to pick a tile, 1 to confirm, and 2 or Esc to cancel. A strip along the bottom always shows whether the input is Keyboard, Simulated, Recorded, or Live EEG, so simulated input is never mistaken for the headset.

A bug worth sharing

The P300 chooser and the alpha switch each made their own live EEG adapter, and both numbered their events e1, e2, and so on. Because of this, when the alpha switch sent its Confirm, the app saw the same ID as the P300 preview and rejected it as a repeat. Rejecting repeats is on purpose, since a repeated signal should never make the app speak twice, but here it meant the live flow could never actually confirm. The fix was one counter shared by every adapter, so IDs are unique for the whole run. A new test now has two live adapters preview and then confirm.

Keeping it safe

Since the app speaks for someone, the worst failure would be a late or repeated signal saying something the wearer never chose. Only one place in the code can speak or change the map, a decision left over from an old round is thrown away, and an unanswered preview cancels itself after 15 seconds. The preview now shows that countdown, so the wearer and the people around them can see it. This follows the fail-safe defaults principle: when something is unclear, the app does nothing instead of guessing.

On the privacy side, only the typed question is ever sent to the AI, never EEG data. Shared git hooks block API keys and raw EEG recordings from being committed, and every pull request has to pass CI on Windows, macOS, and Linux before it can merge. EEG is personal health data, so there is absolutely no reason for it to end up in a public repo.

What’s next

The P300 chooser and the alpha switch both work on their own, but they aren’t wired into the app with the real headset yet. So far, everything has been tested on simulated or synthetic brain data. Next, we will connect the g.tec Unicorn Hybrid Black, calibrate it on an actual wearer, and run the full flow live. After that, we will record the demo video.

Questions

Please leave any questions you have in the comments! We would love to discuss our project!

Repo

https://github.com/Order-of-the-Golden-Eagles/Signal2Speech

來源:DEV Community · dev.to