← All models

fill-mask · text · WASM · ~110 MB

BERT — predict the missing word

This is the original BERT, the model that taught machines to read in both directions at once. Its one job here is masked language modelling: hide a word behind [MASK] and it ranks what belongs there. Type a sentence with a mask and see its guesses — plus the exact logits it computed. Everything runs on your device.

At a glance

ModelXenova/bert-base-uncased (ONNX build of google-bert/bert-base-uncased)
Task / pipelinefill-mask — masked language modelling, loaded via AutoModelForMaskedLM for raw logits
OutputA probability over the whole 30,522-token vocabulary at each [MASK]
ArchitectureBERT-base (12 layers, 110M params), uncased WordPiece tokenizer
Quantizationq8 (8-bit) ONNX
BackendWebAssembly (runs anywhere; no WebGPU needed)
Download~110 MB, cached after first load
LicenseApache-2.0
Web APIsWebAssembly, Web Workers — all Baseline widely available

Run it

Write a sentence containing [MASK] (use the button if you're not sure). BERT predicts the missing word. You can use more than one mask.

See inside — logits, not just a guess

A language model doesn't output words — it outputs a logit (a raw, unbounded score) for every one of its 30,522 vocabulary tokens at the masked position. We softmax those into probabilities. The bars above show the probability; the logit beside each is the actual number the network produced. Watch how a confident prediction ("paris") sits far above the pack in logit space, while an ambiguous slot spreads the mass out. The – below is the entropy of the top-k — low means BERT is sure, high means it's hedging.

Why it matters

Masked language modelling is how BERT learned language in the first place — and the same trick is a surprisingly useful tool on its own. Because it scores every word in context, you can ask it what fits, how strongly, and compare alternatives — all locally, with no prompt engineering and no API bill.

Use cases

Basics

One mask, ranked predictions with probabilities and the raw logits behind them.

Practical

Word-by-word autocomplete and factual knowledge probing over ready-made templates.

Wild

Bias probe — run the same sentence with one word swapped and compare what BERT expects.

Multi-model

Fill the mask, then rank the candidates by meaning with MiniLM embeddings.

How the API works

The one-liner uses the fill-mask pipeline:

import { pipeline } from "@huggingface/transformers";

const unmask = await pipeline("fill-mask", "Xenova/bert-base-uncased");
const out = await unmask("The capital of France is [MASK].");
// → [{ token_str: "paris", score: 0.325, sequence: "…" }, { token_str: "lille", … }]

For the "see inside" surface we go one level lower — AutoModelForMaskedLM — so we can read the exact logits tensor ([1, sequence, 30522]), slice the row at the [MASK] position, and softmax it ourselves. That's the only way to show the real logit values, not just the pipeline's rounded scores. All of it runs in a Web Worker so the page never janks.

References