Proof. Core to Kern.
LiveCaption
Live captions on the Even Realities G2 glasses that show who is speaking. Built pro bono for a deaf user.
IndustryAccessibility / Smart glasses
The challenge
For someone who is deaf, conversations with several people are the hardest. Captions on a phone mean looking down all the time, away from the people you are talking to. And even then you do not know who said what.
The Even Realities G2 are glasses with microphones and a small green display in the lens. We built an app for them pro bono, for a deaf user we do not name: what is said in the room appears live on the lens, with the name of who said it.
What we built
The display is small: 576 by 288 pixels, one fixed font and about three updates a second. Captions that keep jumping are unreadable on it. So words that are settled never move again. Only the last part, still being recognised, changes.
Deepgram’s speech recognition is good, but telling speakers apart in a live conversation proved unreliable. Comparing voices with a voice profile recorded in advance worked much better in tests with real recordings.
A wrong name is worse than no name. Unknown voices get a letter, and a name only appears when the voice clearly matches.
The app must never fail silently. Someone who is deaf does not hear the connection drop. So the screen always shows whether the app is listening, reconnecting or receiving no sound.
The blueprint
Live transcription
The glasses’ microphones send sound through the phone to a server. It turns the sound into text with Deepgram Nova-3 and sends it back to the lens.
Calm captions
A custom caption engine that adds text word by word, rolls on at sentence boundaries and never refreshes the display faster than the Bluetooth link can take.
Speaker recognition
Record a few seconds of someone’s voice, and the app then puts that person’s name next to what they say. Voice profiles are stored on the phone.
Controls on the glasses
A double tap pauses or resumes listening. A status indicator is always visible.
Tested with real voices
Decisions were checked against real recordings, not synthetic speech. Nearly 300 automated tests guard the code.
Tech & tools
- Even Hub SDK
- Deepgram Nova-3
- Speaker recognition
- WebSockets
- Node.js
- TypeScript
Want something like this?
Tell us what you have in mind. You get a blueprint with phases and one fixed price.