JOURNAL

Earshot, the full write-up: from Google’s Gemini Live translation tutorial to a product for 100 users

The long version: Google's Gemini Live translation tutorial, the seven gaps between the demo and production, market and competitor research, the Earshot product concept, production design, cost model, naming, test results and open risks.

Read with AI

Choose content to copy and paste into your AI assistant. Nothing is sent automatically. CMS content is converted to Markdown; original Markdown is used when available.

Google's tutorial turns live speech translation into a commodity that costs about $2.21 per hour per language, so the product worth building is everything around the engine. This is the full write-up behind my shorter engineering post: it walks through the tutorial, shows where it stops short of production, and adds the market research, product concept, cost model and naming behind Earshot, a working rebuild aimed at about 100 regular users.

  • Product: translation for the room, not the meeting. One speaker, listeners scan a QR code and hear their own language with captions. Google Meet already bundles translation for online meetings, but not for room audio.
  • Wedge: Vietnamese-Japanese workplace briefings first, client observation of research sessions second. Both recur on a calendar, which is what keeps 100 users coming back.
  • Cost: about $740 to $760 a month for 100 broadcast hours in three languages. Gemini is about 90% of it; audience size does not change the bill.
  • Engineering: the demo's in-memory singleton is replaced by a reconcile loop that rebuilds itself from LiveKit state, Gemini sessions are treated as disposable, and spend is capped.
  • Status: the full audio path passes an automated end-to-end test locally with a loopback engine. The Gemini engine passed the same test against the live API on 6 Nov 2026 with a synthesized English speech sample, translating to Vietnamese and Japanese with the first audio about 3.3 s after speech began.

Two things remain unproven and should be tested before selling anything: translation quality for Vietnamese, Korean and Thai, and whether anyone pays per site.

What the original tutorial builds

The tutorial builds a one-to-many translation broadcast: a host speaks, and each listener hears a live spoken translation in the language they pick. It is part 6 of Google's Gemini Live series on DEV, written by Thor Schaeff of Google DeepMind (article, source repo).

The whole app is one Next.js 16 project. The flow has five steps:

  1. The host opens the Broadcast page and publishes microphone audio into a LiveKit room.
  2. A listener opens the Watch page, joins the same room and picks a language.
  3. The Next.js backend starts a Translation Bridge for that language, if one is not already running. The bridge joins the room as a bot named translator-<lang>.
  4. The bridge subscribes to the host's track, reads 100 ms PCM frames and streams them over a WebSocket to the Gemini Live API model gemini-3.5-live-translate-preview.
  5. Gemini returns translated 24 kHz audio. The bridge publishes it as its own track, and the listener subscribes to that one track only.

Three details in the source matter more than the article suggests:

  • One bridge per language, not per listener. A TranslationSessionManager singleton counts subscribers per language and stops the bridge when the count reaches zero. Fifty Spanish listeners cost one Gemini session.
  • Selective subscription. Listeners connect with autoSubscribe: false and call setSubscribed() on the one translator track they want, so nobody downloads the other languages.
  • Session survival. The bridge stores the session resumption handle and reconnects on a goAway message or an unexpected socket close, while keeping the LiveKit side alive.

The translation request itself is small. The setup message asks for audio output and a target language:

{
  "setup": {
    "model": "models/gemini-3.5-live-translate-preview",
    "outputAudioTranscription": {},
    "generationConfig": {
      "responseModalities": ["AUDIO"],
      "translationConfig": { "targetLanguageCode": "ja", "echoTargetLanguage": true }
    },
    "realtimeInputConfig": { "automaticActivityDetection": { "disabled": false } },
    "sessionResumption": {}
  }
}

The output transcription doubles as live captions: the bridge sends it over the LiveKit data channel to listeners of that language only.

Deployment is a single Cloud Run service built from a two-stage node:22-slim Dockerfile, with the Gemini and LiveKit keys in Secret Manager. The deploy command pins --max-instances 1, --timeout 3600 and --no-cpu-throttling.

Architecture

Everything meets in one LiveKit room; the only thing the tutorial adds to a normal audio room is a bot per language that pipes the host's audio through Gemini and publishes the result.

The host's microphone audio enters a LiveKit room; one bridge per language sends it to Gemini Live Translate and publishes the translated track and captions back; listeners receive one translated track each; a supervisor reads the room and starts or stops bridges.
Earshot architecture: one bridge and one Gemini session per language, however many people listen. The dashed supervisor is the part the rebuild adds to the tutorial.

The host's audio reaches Gemini once per language, not once per listener, and each phone downloads a single translated track. In the tutorial the server starts bridges when the listener page calls an API; in the rebuild a supervisor reads each listener's chosen language from LiveKit itself.

The transport split is the key idea. WebRTC carries audio between people and the room, where latency and packet loss matter. A plain WebSocket carries audio between the server and Gemini, where the network is reliable. Reading the host track in 100 ms frames cuts the message rate to Gemini from 50 to 10 per second, at a cost of about 80 ms of delay.

Where the tutorial stops short of production

The tutorial is a demo, and its author says so in the comments: about 15 to 20 languages and a few hundred listeners on one instance. Seven gaps matter for real users.

GapWhat happens in the demoWhy it hurts
In-memory singletonAll bridges live in one global map, with subscriber counts kept by API callsA restart or redeploy forgets every bridge; listeners hear silence until they re-pick a language
One instance only--max-instances 1, kept warm by a 3-second browser pollA second instance would add duplicate translator bots to every room
Cloud Run service limits60-minute request cap, 10-second shutdown noticeLong events are cut; deploys end live sessions
Open listener pageAnyone with the link can start any languageEach language is a paid Gemini session, so the link is a spending button
No spend capNo limit per room, in total, or per dayOne busy or abusive room sets the bill
Leaked subscriber countsTab close calls /api/translate/unsubscribe, a route the repo does not defineCounts never reach zero, so bridges keep running and billing
Session handlingReconnects with a resumption handleOne production report measured translation pass rate falling from 98% to 65% with resumption, plus silent stalls of 8 to 51 seconds (Google AI forum)

Two smaller mismatches are worth knowing. The demo sends 48 kHz audio although the Live Translate docs specify 16 kHz, and it passes a system instruction although the docs say the model accepts none.

Product research: market, competitors and the gap

The gap is in-room, recurring, small-group translation in Asian language pairs at a flat monthly price. Online meetings are lost to the platforms: Google Meet has included speech translation in 63 languages in ordinary Workspace plans since 22 October 2026, but it excludes live streams and room audio (Google Workspace Updates).

Prices sit in three bands, seen on 6 November 2026:

BandExamplesPriceWhat hurts small recurring buyers
Sales-led event vendorsWordly, KUDO, InterprefyAbout $1,500 per 10-hour packageMinimum purchase, quotes, expiring hours
Self-serve subscriptionsPalabra, LiveVoice, Maestra$60 per month for 3 hours; EUR 0.42 per minute per languagePer-language multipliers, listener caps
Indie and niche toolsGlossa, LiveWord, Pocketalk for Tour$5 per language-hour; $39 to $499 per month; JPY 3,300 per monthMostly English-Spanish or church-only

Three findings shaped the product:

  • A hundred paying customers is a real outcome here. Glossa.live, a church-only tool founded in June 2025, shows 100 active subscriptions and $14,404 in monthly recurring revenue (TrustMRR). A near-identical product is for sale with five customers, so distribution decides the result, not technology.
  • Daily briefings for migrant workers are the least contested recurring use. Lotte E&C runs AI translation in 20 languages at about 40 Korean construction sites, with workers following on phones through a QR code; toolbox meetings dropped from about 40 minutes to 10 to 20 (Korea Herald via Asia News Network). Those tools are built in-house by conglomerates. Japan had 2,571,037 foreign workers in October 2025, 23.6% of them Vietnamese (Nippon.com).
  • The incumbents skip the languages. Teams Interpreter covers 10 voice languages and Zoom's beta covers 5; neither includes Vietnamese or Thai (InterMIND, Slator).

Demand from Japanese factories is inferred from the Korean precedent and workforce numbers, not observed. The research found no willingness-to-pay data for any segment, and could not read Reddit, Hacker News or G2.

Product concept

Earshot is translation for the room, not the meeting: one person speaks, and everyone within earshot follows in their own language on their own phone.

Who it is for. The primary user is a site supervisor or trainer who briefs a mixed-language group every day, starting with Vietnamese workers at Japanese sites. The secondary user is a research agency whose foreign clients observe interviews. The buyer is the organisation; listeners never pay and never sign up.

The three screens.

  • Listener: scan the QR code, tap a language, tap Start. Translated audio plays with large captions, and the speaker's original words show underneath so a supervisor can check what was said. No install, no account.
  • Host: one link per room. Start broadcast, watch a level meter, see how many people listen in each language, and show the QR code.
  • Organiser console: create rooms, choose the languages each room offers, set an optional access code, watch live bridges, and set spending limits.

What is deliberately left out of the first version: two-way questions from listeners, recordings and transcripts, glossaries, and per-organisation accounts and billing. Recording and glossaries are the first two requests to expect, but both bring consent and accuracy obligations that should follow a quality test, not precede it.

Pricing to test. Listeners free and unlimited. Pay-as-you-go at about $5 per language-hour, and site plans at $39 to $99 a month with a language-hour allowance (JPY 3,300 to 10,000 per site in Japan). A daily 15-minute briefing uses five to six hours a month, about $12 of model cost per language, so a per-site price leaves room. An uncapped flat plan does not: cost follows language-hours. These price points are a synthesis of competitor prices, not tested demand.

Production design for about 100 regular users

One always-on instance is enough for this scale, provided it can lose its memory at any moment and recover. The design follows from that.

Capacity. A hundred listeners in a handful of rooms means roughly 5 to 15 live languages at peak. Each bridge uses a few tens of megabytes and a fraction of a core. LiveKit handles thousands of audio subscribers on one server (benchmark). The real ceiling is Gemini: Tier 1's spend cap of $10 per rolling 10 minutes equals about 27 concurrent streams, so a paid Tier 2 project is the safe floor (rate limits).

Reliability.

  • State lives in LiveKit, not in the process. Each listener sets a language attribute on their own participant. A supervisor loop reads rooms and attributes every 1.5 seconds and starts or stops bridges to match. After a crash or deploy it rebuilds the same picture from scratch; there are no subscriber counts to leak.
  • Gemini sessions are disposable. The bridge opens a fresh socket every 8 minutes, before the API's 10-minute close, and also on goAway, on an unexpected close, and when 20 seconds of speech produce no translated audio. It never uses session resumption, because translation needs no history and the one production report shows resumption hurting quality.
  • Bridges heal. A bridge that gets no host audio for 10 seconds resubscribes, and after three tries it stops so the supervisor replaces it.
  • Short gaps do not cause churn. A language keeps running for 20 seconds after its last listener leaves, which covers page reloads and language switching.

Security and spend.

  • Listener tokens can subscribe and set their own attribute, nothing else. Host tokens can publish a microphone only, and require the room's secret host key.
  • A language starts only if the room offers it and a host is present.
  • Three limits are editable in the console: languages per room, languages running at once, and language-minutes per day. When the daily budget is used up no new language starts.
  • Token and sign-in endpoints are rate-limited per address, and rooms can require an access code.

Hosting. The same container image runs on a laptop, a homelab or Cloud Run. The console switches the LiveKit backend and the translation engine at runtime; where the app itself runs is a deploy-time choice. The research favours LiveKit Cloud's $50 Ship plan over self-hosting, which saves at most that fee and needs a public IP, open UDP ports and TURN. Moving the bridge to a per-room LiveKit Agents worker is the right step once more than one instance is needed; at this scale the reconcile loop gives the same restart safety with one moving part fewer.

Cost model

A month of 100 broadcast hours in three languages for 100 listeners costs about $740 to $760, and about 90% of that is Gemini.

Google prices the model at $3.50 per million audio input tokens and $21.00 per million output tokens, at 25 tokens per second, which comes to $0.0368 per minute or $2.21 per hour for one target language (Gemini API pricing, seen 6 November 2026). Each language is its own session, so cost grows with languages and broadcast hours and not with audience size.

Monthly line (USD)CostBasis
Gemini Live Translate, 300 language-hoursAbout 6622.21 per language-hour, upper bound; pauses lower it
LiveKit Cloud Ship plan50Includes 150,000 WebRTC minutes; this load uses 84,000 to 144,000 (LiveKit pricing)
App and bridge, one always-on small instanceAbout 30 to 50Cloud Run 1 vCPU / 1 GiB; rates from secondary sources
TotalAbout 740 to 760Roughly 7.50 per broadcast hour

Three consequences follow:

  • Languages are the cost lever. The per-room language limit in the console is a pricing control as much as a safety one.
  • Self-hosting saves little. Running LiveKit and the app on a homelab removes about $80 to $100 a month and leaves the Gemini line untouched.
  • Alternatives do not change the picture. OpenAI's gpt-realtime-translate costs $0.034 per minute but reportedly has 13 output languages, with Vietnamese and Thai unconfirmed (OpenAI). A captions-only pipeline is far cheaper than any speech-to-speech model and is the fallback if audio quality disappoints.

Name and domain on platformforge.dev

The product is named Earshot and lives at earshot.platformforge.dev, chosen on 6 Nov 2026 from the four candidates below. The category's English roots are used up: Word-, Live-, Vox/Voice-, Lingo-, Interpret- and Gloss- names are all taken (Wordly, LiveVoice, VoxLive, Boostlingo, Interprefy, Glossa).

NameSubdomainWhy it fitsWatch out for
Kikukiku.platformforge.devJapanese for "to listen" (聞く); two syllables that Vietnamese and English speakers say the same wayA common word, so other products likely use it
Earshotearshot.platformforge.devSays the promise in one English word: everyone within earshot understands. It is the working name in the codeEnglish-only; less natural for the Japan wedge
Mimimimi.platformforge.devJapanese for "ear" (耳); friendly and easy to sayReads as a personal name; weaker for a workplace tool
Nghenghe.platformforge.devVietnamese for "listen"; speaks directly to Vietnamese listenersHard to pronounce and type for Japanese buyers

Earshot was chosen because it keeps the product language-neutral. Kiku remains the fallback if the Vietnamese-Japanese wedge later needs a local name.

The subdomain layout is the same whichever name wins:

  • earshot.platformforge.dev for the app. Listener links become earshot.platformforge.dev/r/<code>, short enough to read aloud.
  • rtc.earshot.platformforge.dev for LiveKit signalling, needed only when self-hosting LiveKit. LiveKit Cloud supplies its own address.

No trademark search (USPTO, EUIPO, J-PlatPat, Vietnam) has been done for any of these names.

Build and deploy walkthrough

The working product is a single Next.js 16 app with the bridge supervisor running inside the server process. It keeps the tutorial's stack (LiveKit, @livekit/rtc-node, a raw WebSocket to Gemini) and changes how bridges are started, kept alive and paid for.

The pieces.

PartFileWhat it does
Plannersrc/bridge/plan.tsA pure function: rooms and listener languages in, bridges to start and stop out. Holds every limit
Supervisorsrc/bridge/supervisor.tsReads LiveKit every 1.5 seconds and applies the plan
Bridgesrc/bridge/bridge.tsJoins as translator-<lang>, pipes host audio to the engine, publishes audio and captions
Enginessrc/bridge/providers/gemini.ts with rotation and a stall watchdog; echo.ts, a free loopback for testing
Tokenssrc/lib/livekit.tsLeast-privilege grants per role
Storesrc/lib/store.tsSQLite file for rooms, settings and usage

Run it locally.

pnpm install
cp .env.example .env.local      # set APP_SECRET and ADMIN_PASSWORD
LIVEKIT_BIN=/path/to/livekit-server ./scripts/dev-livekit.sh
pnpm dev                        # http://localhost:3100

Sign in at /admin, create a room, and open its host link and listener link in two tabs. Without a Gemini key the echo engine loops the host back, which exercises everything except the translation itself.

One lesson from testing. On a machine with several network interfaces LiveKit advertised a VPN address, and bridges took up to 10 seconds to join or never received audio. Pinning --node-ip fixed it: joins dropped to 0.3 to 1.5 seconds. The address LiveKit advertises must be one every client can reach.

What was verified.

  • 11 unit tests on the planning rules and engine helpers pass, and the type check is clean.
  • An end-to-end script drives a scripted host and listener over real WebRTC. It passed four runs in a row, against both the dev server and the production build. It checks that the translator joins, that audio and captions arrive, that a second language starts on a switch, that an unlisted language is refused, and that bridges stop about 18 seconds after the last listener leaves.

What was not verified. The Gemini engine has run only on a short synthesized English sample: sessions longer than a few minutes, real microphones and real accents are untested. The Docker image has not been built, and the Cloud Run and homelab deploy files have not been run.

Deploy.

  • Cloud Run with LiveKit Cloud: deploy/cloudrun.sh deploys one always-on instance with CPU always allocated and the SQLite file on a Cloud Storage volume. Unlike the tutorial's command it sets --min-instances 1, so a broadcast does not depend on a browser poll to stay alive.
  • Homelab: deploy/docker-compose.homelab.yml runs the app and a LiveKit server. LiveKit needs 7881/tcp and a UDP range reachable from phones, plus TLS on both hostnames; an HTTP-only tunnel cannot carry the media.
  • Switching: list several LiveKit backends in LK_PROFILES and pick the active one in the console. Switching ends live rooms on the old backend.

Risks, open questions and next steps

The largest risk is the model, not the code: gemini-3.5-live-translate-preview is still a preview, and no independent test covers Vietnamese, Korean or Thai.

RiskWhat is knownMitigation
Translation quality in the target pairsOne competitor-run benchmark on English audio scored 4.93 of 5, with first audio after about 2.9 seconds (LiveLingo)Native-speaker test on real briefing audio before any sale
Preview model changes or retiresPreview status, global region only on VertexModel ID is configuration; engines sit behind one interface
Silent stalls and 10-minute connectionsOne production report; documented connection lifetime (session management)Scheduled rotation and stall watchdog, both untested against the live API
No glossary or instructionsDocumented limit of the modelSource captions beside the translation; a captions pipeline with glossary later
Single instanceBy design at this scaleRestart-safe supervisor; move bridges to a per-room worker before scaling out
SQLite on Cloud RunA Cloud Storage volume is not a databaseFine for one writer; move to Postgres with a second instance
Trust and liabilityProfessional bodies say AI output needs human review (ATA)On-screen notice; exclude medical, legal and disciplinary use; no audio retention

Open questions:

  • Is silent input billed in full, or only speech?
  • Does a worker hosted outside LiveKit Cloud count against agent session minutes?
  • Which data-transfer rules apply to worker speech in Japan, Vietnam and Korea?

Next steps, in order:

  • Done: Create a paid Gemini API project and run the end-to-end script with the Gemini engine
  • Record 30 minutes of real Vietnamese and Japanese briefing audio and have native speakers rate the output
  • Talk to five to ten supervising organisations, dispatch agencies or research agencies about paying per site
  • Run a trademark check on Earshot and point earshot.platformforge.dev at the deployment
  • Open a LiveKit Cloud project, build the image and deploy one instance
  • Add a soak test: a two-hour broadcast across at least twelve session rotations

Sources

Discussion

Comments are reviewed before publication. Your email is kept private.

← Back to allĐọc tiếng Việt