text-generation · text · WASM · ~280 MB

GPT-2 — the decoder-only model that started it all

GPT-2 is the original decoder-only transformer: at every step it predicts the next token from everything before it, samples one, feeds it back, and repeats. It isn't chat-tuned — it's a raw text completer, which makes it the clearest possible window into how autoregressive generation actually works. Give it a prompt and watch it write, token by token — and watch the real probability distribution it picks from at each step.

At a glance

Model	Xenova/gpt2 (ONNX of openai-community/gpt2)
Task / pipeline	`text-generation` — loaded via `AutoModelForCausalLM` for a custom decode loop with per-step logits
Architecture	Decoder-only transformer, 124M params, 12 layers, byte-level BPE, 1024-token context
Output	A probability over the whole 50,257-token vocabulary at each step
Quantization	int8 (8-bit) ONNX
Backend	WebAssembly (runs anywhere; no WebGPU needed)
Download	~280 MB, cached after first load
License	MIT
Web APIs	WebAssembly, Web Workers — all Baseline widely available

Run it

Type a prompt and press Generate. GPT-2 continues it one token at a time (streamed live from a Web Worker). Switch between greedy (always the single most likely token) and sampling (temperature + top-k) to feel the difference between safe and surprising.

Prompt

tokens 40

See inside — the next-token distribution

GPT-2 never outputs a word directly — it outputs a probability for every one of its 50,257 tokens, then one is chosen. As it generates, the panel below shows the top candidates it was choosing between at the latest step (the green one is the token it actually emitted). Underneath, the confidence trace colours each generated token by how probable the model thought it was — pale means it was a confident, obvious choice; dark means it took a gamble.

Why it matters

Every modern LLM — chat models included — is doing exactly this under the hood: predict a distribution, sample, repeat. GPT-2 is small and un-tuned enough to make that loop legible, and it runs on plain WASM with no GPU. It's the honest teaching model, and still a capable little writer.

Understand sampling — see first-hand what temperature and top-k do to text, with the real distribution in view.
Offline autocomplete / ideation — draft continuations, prompts, and variations with nothing leaving the tab.
A teaching tool — show students how "AI writing" is next-token prediction, not magic.
A generation stage in a pipeline — feed its output into a classifier or summariser (see multi-model).

Use cases

Basics

Prompt in, streamed completion out, with live token count and tok/s.

Practical

Autocomplete: see the ranked next-token suggestions for whatever you've typed.

Wild

Branching text — you pick the next token from GPT-2's top candidates and steer the story.

Multi-model

GPT-2 writes, then a sentiment model reads the mood of what it wrote.

How the API works

The friendly one-liner is the text-generation pipeline:

import { pipeline } from "@huggingface/transformers";

const gen = await pipeline("text-generation", "Xenova/gpt2", { dtype: "int8" });
const out = await gen("The meaning of life is", { max_new_tokens: 40, temperature: 0.9, do_sample: true, top_k: 40 });
// out[0].generated_text → the prompt plus GPT-2's continuation

For the "see inside" surface we run our own loop with AutoModelForCausalLM: each step is a forward pass whose logits row ([1, seq, 50257]) we softmax ourselves, so we can show the exact probability of every candidate and of the token actually chosen. It all runs in a Web Worker so streaming never blocks the page.