← All models

image-to-text · vision → text (OCR) · WASM · ~185 MB

TrOCR — read the text out of a picture

TrOCR is a transformer that does OCR the modern way: no hand-built pipeline, just a Vision Transformer that looks at a cropped line of text and a text decoder that writes it out, one token at a time — GRAND TOTAL: $42.99 — entirely on your device.

At a glance

ModelXenova/trocr-small-printed
Task / pipelineimage-to-text
ArchitectureViT-style image encoder → RoBERTa-style text decoder (VisionEncoderDecoder, cross-attention)
InputA single line of printed text (crop to the line you want)
OutputThe transcribed characters, generated token by token
Quantizationq8 (8-bit) ONNX — separate encoder + decoder graphs
BackendWebAssembly (runs anywhere; no WebGPU needed)
Download~185 MB, cached after first load
LicenseMIT (microsoft/trocr-small-printed)
Web APIsWebAssembly, Web Workers, Canvas, FileReader — all Baseline widely available

Run it

Pick or drop an image, then drag a box around one line of text and hit Read. TrOCR is a single-line reader, so the crop is the point — it transcribes what's inside the box.

Drop an image here, or click to choose
A receipt line reading Grand Total $42.99 A printed line: The quick brown fox jumps A sign reading Platform 9 A label reading SN 4B7X-9021-KD A small café receipt with several lines

Drag on the image to select a line.

See inside — the decode, token by token

OCR here is autoregressive generation, not template matching. The encoder reads the cropped line into patch embeddings once; then the decoder emits the transcription one token at a time, each conditioned on the image and everything decoded so far — which is why TrOCR can fix its own spelling from context. The trace below shows every token as it arrived, with the elapsed time. The first token is the slow one (it includes the image encode); the rest come faster.

Read a line to see the token-by-token trace.

Why it matters

Text trapped in an image is invisible to everything that isn't a person looking at it — you can't search it, copy it, translate it, or total it. OCR turns those pixels back into characters, and doing it locally makes it free, private, and offline-capable.

Use cases

Basics

Crop a line — a sign or a receipt total — and read it, token by token.

Practical

Digitise a whole receipt: read every line into an editable, copyable list.

Wild

Live webcam OCR — point your camera at text and read it on the fly.

Multi-model

OCR → translation: read foreign text, then translate it locally.

How the API works

The base call is two lines with Transformers.js:

import { pipeline } from "@huggingface/transformers";

const reader = await pipeline(
  "image-to-text",
  "Xenova/trocr-small-printed",
); // downloads + caches ~185 MB once

const out = await reader(lineCropURL, { max_new_tokens: 48 });
// → [{ generated_text: "GRAND TOTAL: $42.99" }]

The important bit is the input: TrOCR expects one line of text, so this page crops the region you box before handing it over. For "See inside", it passes a TextStreamer into the same call — the streamer fires once per decoded token, so the page renders the transcription as it forms and records each token's arrival time, all off the main thread in a Web Worker. Note: this SROIE-tuned checkpoint tends to normalise to UPPERCASE — that's the model, not a bug.

References