I wanted to type in Kannada without switching keyboards. Speaking felt more natural than typing, but every speech-to-text tool either required a server, needed an app install, or didn’t support Kannada well.

So I built ಲಿಪಿ (Lipi) — a voice editor that runs entirely in the browser. No backend. No framework. Five files.

The problem with one engine

Web Speech API streams text in real time. Words appear as you speak. But it’s cloud-dependent and not available on Firefox.

Whisper (via Transformers.js) runs locally. Accurate, private, offline. But it can’t stream — you record first, transcribe after.

Neither alone is enough. So I used both.

Three tiers

User clicks record
├── Tier 1: On-device Speech API (processLocally)
│   └── Chrome 130+ with language pack installed
├── Tier 2: Cloud Speech API (webkitSpeechRecognition)
│   └── Chrome/Edge, cloud-based, real-time streaming
└── Tier 3: Whisper ONNX chunks (Transformers.js worker)
    └── Firefox, Safari, or if Speech API fails

The app detects what’s available and picks the best tier automatically.

Tier 1: On-device speech

Chrome recently added processLocally to the Web Speech API. Here’s how Lipi uses it:

if ('processLocally' in recognition) {
  const availability = await SpeechRecognition.available({
    langs: ['kn-IN'],
    processLocally: true
  });

  if (availability === 'available') {
    recognition.processLocally = true;
  } else if (availability === 'downloadable') {
    await SpeechRecognition.install({ langs: ['kn-IN'], processLocally: true });
    recognition.processLocally = true;
  }
}

Real-time streaming like cloud, but no data leaves the device. Best of both worlds.

Tier 2: Cloud Speech API

The familiar webkitSpeechRecognition. Lipi configures it for Kannada with interim results:

recognition = new webkitSpeechRecognition();
recognition.lang = 'kn-IN';
recognition.continuous = true;
recognition.interimResults = true;

Interim text appears as a dimmed span that gets replaced when the final result arrives. It feels like the words are streaming out of your mouth onto the screen.

Tier 3: Whisper fallback

For browsers without Speech API, the app falls back to Whisper running in a Web Worker via Transformers.js:

// Record raw PCM at 16kHz
scriptProc.onaudioprocess = (e) => {
  pcmChunks.push(new Float32Array(e.inputBuffer.getChannelData(0)));
};

// Every 5 seconds, flush to Whisper
chunkTimer = setInterval(() => flushChunk(false), 5000);

Not as smooth as streaming, but it works everywhere. The model downloads once and caches in the browser’s Cache API.

The Polish button

Here’s where it gets interesting. While the Web Speech API streams text live, the app silently records raw PCM audio in parallel. After you stop, a “Polish” button appears.

Click it, and Whisper re-transcribes your audio from the raw recording. The result replaces only the recorded text — typed text before and after is untouched.

"typed text" [start-mark] {speech text → replaced by Whisper} [end-mark] "more typed text"

Whisper gets a Kannada initial prompt to condition its decoder:

opts.initial_prompt = 'ಕನ್ನಡ ಭಾಷೆಯಲ್ಲಿ ಸ್ಪಷ್ಟವಾಗಿ ಬರೆಯಿರಿ. ಸರಿಯಾದ ವಿರಾಮ ಚಿಹ್ನೆಗಳನ್ನು ಬಳಸಿ.';

This biases the decoder toward clean Kannada script with proper punctuation.

Phonetic typing

Don’t want to talk? Type namaskara and it becomes ನಮಸ್ಕಾರ. The transliteration engine intercepts keystrokes, buffers Latin characters, and converts on space.

On mobile, virtual keyboards don’t fire keydown reliably. Lipi uses beforeinput as a fallback:

ed.addEventListener('beforeinput', e => {
  if (e.inputType === 'insertText' && /^[a-zA-Z]$/.test(e.data)) {
    e.preventDefault();
    buf += e.data;
    showBuf();
  }
});

Desktop uses keydown (fires first, blocks beforeinput). Mobile falls through to beforeinput. Both work.

The stack

Five files. No build step. No node_modules.

index.html    — landing + editor (single page, hash routing)
index.css     — design system with oklch colors
app.js        — speech, recording, transliteration, UI
worker.js     — Whisper inference in a Web Worker
transliterate.js — phonetic-to-Kannada mapping

The Whisper model preloads in the background via requestIdleCallback when the editor opens. By the time you need it, it’s ready.

Trade-offs

• Web Speech API accuracy varies by accent. The Polish button exists precisely for this.
• Whisper Small is ~240MB. It caches after first download, but that first load takes time.
• On-device Speech API is Chrome 130+ only. The fallback chain handles everything else.

That’s it. No server. No API keys. Just the browser doing what browsers should do.

Try it