GitHub - joeblau/subvox
Subvox is a macOS prototype that checks whether AirPods Pro head-motion data is rich enough to recognize silently mouthed words. It skips language models entirely, using a small-vocabulary template matcher with dynamic time warping and a permutation test to show whether any signal is real.
- Matching is 1-nearest-neighbour with dynamic time warping over user acceleration, rotation rate, and attitude change, and a segment is rejected if the match is weak or ambiguous.
- Significance comes from leave-one-out accuracy compared against a 500-run label-shuffle test, with p < 0.05 counted as better than chance.
- The motion stream is low-rate (about 25 Hz on AirPods) and doesn't expose the speech-detecting accelerometer or inward mic, so it captures jaw and head movement rather than vocal vibration.
Subvox is a macOS prototype asking a narrow question: do AirPods Pro head-motion readings (via CMHeadphoneMotionManager) carry enough signal to tell which silently mouthed word someone is forming? It's not a speech-to-text system in the usual sense. There's no language model. Instead, the app is a small-vocabulary template recognizer. You record several takes of each word in the Train screen, and the Decode screen matches new motion segments against those templates using dynamic time warping with a Sakoe–Chiba band and 1-nearest-neighbour classification, rejecting matches that are too distant or too ambiguous. Features are per-frame user acceleration, rotation rate, and attitude change since the segment start, mean-centered and z-scaled. Requirements are macOS 14+ and AirPods Pro or another head-tracking headphone set as the Mac's audio output.
The part I find most useful is the built-in significance test. Evaluate runs leave-one-out classification, then repeats the procedure 500 times with shuffled labels to build a chance distribution. If the real accuracy beats that at p < 0.05, the motion stream is distinguishing your words above chance. The feature checkboxes let you ablate acceleration, rotation, and attitude separately, so you can see which channel is actually doing the work. The recommended experiment is the same five-word script spoken normally, whispered, and silently mouthed, ten reps each, then compared across feature sets. Sessions are logged to ~/Documents/Subvox/Sessions/ as motion.csv, audio.caf, events.csv, and meta.json, with an audio offset so you can align the streams after the fact.
The caveats matter. Headphone motion arrives at roughly 25 Hz, and the speech-detecting accelerometer and inward mic aren't exposed, so you're measuring jaw and head movement rather than vocal vibration. Templates are tied to a specific user and seating, so re-inserting the AirPods can degrade results. And selecting AirPods as the mic input forces the Bluetooth HFP profile, which drops audio quality for everything else on the machine. The repo is best read as a falsifiable experiment with working code, not a usable dictation tool.