← All models

language identification (text-classification) · text · WASM · ~28 MB

mBERT language detection — what language is this, across 201?

Paste any text and this model tells you which of 201 languages it's written in, with a confidence score. It's a fine-tune of a multilingual BERT encoder with a classification head over the FLORES-200 label set (codes like fra_Latn, zho_Hans), so it reads the actual linguistic signal — script, morphology, function words — rather than matching a keyword list. A distinct model and a much wider net than the 20-language XLM-RoBERTa demo: different architecture, ten times the languages. Nothing leaves your device.

At a glance

Modelonnx-community/language_detection-ONNX (ONNX of alexneakameni/language_detection)
Task / pipelinetext-classification — one of 201 language labels + softmax scores; logits shape [1, 201]
Languages201, labelled with FLORES-200 codes <iso639-3>_<Script> (e.g. eng_Latn, fra_Latn, arb_Arab, zho_Hans, jpn_Jpan)
ArchitectureMultilingual BERT (BertForSequenceClassification) with a 201-way classification head
Quantizationq8 (8-bit) ONNX (model_quantized.onnx) — verified non-degenerate
BackendWebAssembly (runs anywhere; no WebGPU needed)
Download~28 MB (q8), cached after first load — small enough for mobile
LicenseMIT (base model)
Web APIsWebAssembly, Web Workers — all Baseline widely available

Run it

Type or paste text (or try a sample). Detection runs as you stop typing.

See inside — the probability distribution

The model doesn't just pick a language; the classification head emits a real logit per class ([1, 201]), softmaxed into a probability over all 201. A confident detection puts almost all the mass on one bar. On short or mixed text — or between closely related languages that share a script — the mass spreads; that spread is the model telling you it's unsure, and it's genuinely useful signal. Below are the top languages for the current text. Try the Code-switch sample and watch two languages share the mass.

Enter some text above to see the distribution.

Why it matters

Language identification is the quiet first step in almost every multilingual text pipeline. Before you can translate, moderate, route, or index text, you have to know what language it's in — and you often have to do it on a single sentence, a search query, or a chat message. Running it on-device means you can branch instantly with zero latency and without sending user text to a server.

Use cases

Basics

Detect the language of a sentence, with the full probability distribution.

Practical

Route incoming messages to the right per-language workflow, live.

Wild

Short, ambiguous, and code-switched snippets — watch the confidence honestly wobble.

Multi-model

Detect the language, then route to the matching Marian translator into English.

How the API works

It's a plain text-classification pipeline; ask for all labels to get the full distribution:

import { pipeline } from "@huggingface/transformers";

const detect = await pipeline(
  "text-classification",
  "onnx-community/language_detection-ONNX",
  { dtype: "q8" },
);

const out = await detect("Bonjour, comment allez-vous ?", { top_k: 201 });
// → [{ label: "fra_Latn", score: 0.998 }, { label: "bam_Latn", score: 0.0003 }, … ]
// out[0].label is the detected FLORES-200 code; the rest is the confidence distribution.

The model outputs a fixed label set (the 201 FLORES-200 codes). Passing top_k: 201 returns every language's probability, sorted — that's what powers the "see inside" bars. All inference runs in a Web Worker so detection never janks the page.

References