← All models

automatic-speech-recognition · audio · WebGPU (WASM fallback) · ~120 MB

Whisper — turn speech into text, on your device

Whisper listens to audio and writes down what was said — in your browser, with no server and no upload. Record from your microphone, drop in an audio file, or use a sample clip, and get a transcript with word-level timestamps you can see line up with the waveform.

At a glance

Modelonnx-community/whisper-base_timestamped (whisper-base, exported for word timestamps)
Task / pipelineautomatic-speech-recognition
ArchitectureWhisper encoder-decoder transformer (74M params), 80-bin log-Mel input → text tokens; cross-attentions exported so word timings align
Quantizationq4 on WebGPU · q8 (8-bit) on WASM — int8 on WebGPU hits a known numerical bug that garbles output, so the worker picks dtype per backend
BackendWebGPU when available, honest WebAssembly fallback
Download~120 MB, cached after first load
LicenseMIT (OpenAI Whisper)
Web APIsWebGPU / WebAssembly, Web Workers, Web Audio (decode + resample), MediaRecorder (mic)

Run it

Load the model, then choose audio: record from your mic, upload a file, or pick a sample. The page decodes it to 16 kHz mono (what Whisper expects), draws the waveform, and transcribes it in a Web Worker so the UI never janks.

What’s happening

  1. Audio inputPreparing the sample audio…
  2. On-device modelChecking the local validation record…
  3. TranscriptionWaiting for audio and model
  4. TranscriptWaiting for transcription

Preparing the sample audio and checking the model…

or a sample:

Bundled JFK inaugural excerpt (20 January 1961), spoken by John F. Kennedy — public domain, U.S. government work · JFK audio source and attribution record. LibriSpeech: reader Paul-Gabriel Wiener, OpenSLR SLR12, CC BY 4.0.

JFK — inaugural (11s)

Transcript

Transcribe a clip to see the text here.

See inside

Whisper doesn't emit a single blob of text — it decodes a stream of tokens and, with word timestamps on, attaches a start/end time to each run of speech. Below is that raw timestamped segment stream: the model's own view of who said what, when. The tok/s and backend readout above are measured on this device, not claimed.

Run a transcription to see the timestamped segment stream.

Why it matters

Speech-to-text used to mean shipping audio to a cloud API — a bill, a privacy problem, and a network dependency. Whisper-base runs on the visitor's own device, so transcription becomes a free, private, offline capability of your page.

Use cases

Basics

One clip in, a timestamped transcript out — the core move.

Practical

Live captions / dictation: transcribe as you speak, segment by segment.

Wild

Karaoke — subtitle the sample and highlight each word as it's spoken.

Multi-model

Voice loop: speak → Whisper → edit → Kokoro speaks it back.

How the API works

Transcription is one pipeline call. Word timestamps are a single option:

import { pipeline } from "@huggingface/transformers";

const transcribe = await pipeline(
  "automatic-speech-recognition",
  "onnx-community/whisper-base_timestamped", // whisper-base, exported for word timestamps
  { device: "webgpu", dtype: "q4" }, // WASM (q8) fallback when WebGPU is absent or garbles
);

// audio = a 16 kHz mono Float32Array (decode with Web Audio, then resample)
const out = await transcribe(audio, {
  return_timestamps: "word",   // "word" | true (segment) | false
  chunk_length_s: 30,          // long-audio chunking
  stride_length_s: 5,
});
// out.text  → the full transcript
// out.chunks → [{ text: " country", timestamp: [9.8, 10.2] }, …]

This page decodes the mic/file/sample to 16 kHz mono with the Web Audio API on the main thread, then transfers just the Float32 samples to a Web Worker where the pipeline runs — so the model never blocks the interface. Word chunks are grouped into the sentence-level segments shown under "See inside".

References