The tokenization tax: the hidden cost of Arabic in English-first LLMs

Procurement teams compare models on price per million tokens. Engineering teams budget context windows in tokens. Latency dashboards count tokens per second. All three assume the token is a stable unit of content. For Arabic, it is not: Arabic tokenization on English-first models turns the same content into more tokens than its English equivalent — a tokenization tax — and the difference can be large enough to change vendor choices and unit economics.

The metric that exposes this is fertility: the average number of tokens a tokenizer produces per word, a measure used in multilingual tokenizer studies such as Rust et al. (ACL 2021). For English, OpenAI's rule of thumb that 100 tokens are about 75 words works out to roughly 1.3 tokens per word. Arabic has fared far worse: when the Jais team measured GPT-2's tokenizer, it produced 4.17 tokens per Arabic word against about 1.1 per English word — nearly four times as many. Newer vocabularies have narrowed the gap, but nothing in your invoice will label what is left; it simply arrives as a higher bill for the same work.

Where the Arabic tokenization tax comes from

Three properties of Arabic collide with the way subword tokenizers are built:

• Morphology packs a sentence into a word. Arabic attaches conjunctions, prepositions, the definite article and object pronouns directly to the stem. A single written word like وسيكتبونها (wa-sa-yaktubūnahā) carries what English spreads across five words: "and they will write it." Frequency-based tokenizers rarely keep such inflected forms whole, so they shred them.
• Vocabulary is allocated by frequency — and the training mix is mostly English. A subword vocabulary is a fixed budget of a few tens or hundreds of thousands of entries, awarded to the character sequences most frequent in the tokenizer's training corpus. When that corpus is dominated by English and Latin script, Arabic gets a thin slice of the budget. The Jais paper notes that GPT-2's tokenizer splits even a common word like لماذا (limādhā, "why") into individual characters.
• Byte fallback doubles the floor. When a sequence isn't in the vocabulary at all, byte-level tokenizers fall back to raw bytes — and every letter in the basic Arabic Unicode block takes two bytes in UTF-8. The worst case for a rare Arabic word is not one token per letter; it is two. The ALLaM authors describe exactly this failure in existing models: Arabic split down to the character level or pushed into byte fallback. Orthographic variation (with and without diacritics, hamza and alef variants) multiplies the rare-sequence problem further.

Measuring the tax

Fertility is easy to measure: tokenize a representative sample of your own Arabic traffic and divide tokens by words. Published fertility studies show the same pattern across tokenizer generations — dramatic improvement over time, but a persistent premium on vocabularies that were not built with Arabic in mind:

• Multilingual BERT era. Rust et al. compared mBERT's shared multilingual vocabulary with dedicated monolingual tokenizers, including AraBERT's for Arabic, and found the shared vocabulary split Arabic into more pieces per word.
• English-first byte-level BPE. In the Jais technical report, GPT-2's 50,257-entry vocabulary scored 4.17 on Arabic. BLOOM's 250,000-entry multilingual vocabulary scored 1.20, and the bilingual 84,992-entry Jais tokenizer, trained on equal parts English and Arabic, scored 1.05.
• Arabic-adapted open models. The Arabic Stable LM team compared tokenizers on Arabic web text (OSCAR) and a treebank (PADT): the 32K-entry Llama 2 vocabulary behind AceGPT fragmented Arabic the most, Jais the least, and Stable LM 2's Arcade100k vocabulary had the second-highest fertility of the models compared.

One caution when reading these numbers: studies count "words" and normalize text differently, so compare ratios within a study rather than absolute scores across studies. The Jais report puts GPT-2's English at about 1.1 tokens per word, below OpenAI's 1.3 rule of thumb — a reminder that absolute numbers depend on method. The figure below turns the pattern into illustrative midpoints you can reason with.

What the tax actually costs you

Fertility is not an academic number. Petrov et al. (NeurIPS 2023) found that the same text translated into different languages can differ in tokenized length by up to 15 times, with direct consequences for cost, latency and how much fits in context. The ALLaM authors list the Arabic version plainly: inflated pretraining corpora, more tokens to generate per word, and shorter effective context. The ratio multiplies through every token-denominated resource in your stack:

• Spend. Illustratively, at fertility 2.4 vs 1.3, the same support conversation costs ~1.8× more tokens in Arabic than English — before any difference in verbosity. Across millions of daily interactions, tokenizer choice is a line item.
• Latency. Decoding is per-token. An answer that fragments into 1.8× the tokens takes longer to stream, and prompt changes cannot remove that per-word overhead.
• Context. A 128K-token window is not 128K tokens of content. High fertility shrinks how many Arabic words fit — squeezing RAG budgets, few-shot examples and long documents.

Put in procurement terms, the same workload prices very differently depending on the tokenizer underneath (illustrative, using the same midpoints):

This is why price per million tokens is a misleading basis for comparing providers on Arabic workloads. Illustratively, a model that is 30% cheaper per token but fragments Arabic 1.6× more ends up about 12% more expensive per word (0.7 × 1.6 = 1.12). The only fair unit is cost per word — measured on your own traffic.

The quality connection

The tax is not only economic. In a nine-language study that included Arabic, Rust et al. found that swapping mBERT's multilingual tokenizer for a dedicated monolingual one improved downstream performance for almost every task and language. Ali et al. found that training multilingual LLMs on English-centric tokenizers caused severe downstream degradation and up to 68% extra training cost — while cautioning that fertility on its own is not always predictive of downstream scores. And a 2025 study of 16 African languages found that higher fertility consistently predicted lower accuracy. The plausible mechanism: when a model must reassemble meaning from sub-morphemic shards, it spends capacity on spelling that better-tokenized languages spend on semantics. Treat fertility as a warning light, not a verdict.

Fragmentation also degrades the plumbing around the model — embedding quality for retrieval, token-budget chunking that splits Arabic sentences mid-word, deduplication and caching that treat diacritized and undiacritized variants of the same text as unrelated strings, and exact-match evaluation metrics that punish orthographic variation the writer never controlled.

And the tax may be highest exactly where it hurts. The published fertility figures above were measured mostly on edited, largely standard Arabic (web text, treebanks, curated validation sets), while production traffic is dialectal, code-switched and informal, the kind of text least represented in tokenizer training corpora. Treat your cleanest MSA benchmark as a floor, not a forecast, and check it against the dialects your users actually write.

What to do about it

1. Measure fertility on your corpus, not a vendor's demo. Tokenize a real sample of your traffic — including dialect and code-switching — and compute tokens per word for every model on your shortlist.
2. Negotiate and forecast in cost per word. Convert every per-token price into per-word cost using your measured fertility before comparing vendors or projecting spend.
3. Budget context in words. Set RAG chunk sizes, few-shot counts and output limits from word budgets divided by measured fertility — not from token numbers borrowed from English deployments.
4. Prefer Arabic-aware vocabularies where you control the stack. If you are self-hosting or fine-tuning, tokenizer choice is an architecture decision that affects cost, latency and quality. A vocabulary learns from its corpus, so the fix starts with genuine, dialect-diverse Arabic training data — not a corpus padded with machine-generated text, which carries risks of its own. If you are choosing that corpus, our comparison of Arabic text corpora for LLM training sets the open web sets against licensed text.
5. Normalize deliberately. Decide how you handle diacritics, hamza and alef variants at ingestion — consistently across training data, retrieval indexes and evaluation — so surface variation stops multiplying tokens and splitting caches.
6. Include fertility in evaluation. When you benchmark Arabic models — on public suites such as ArabicMMLU and the Open Arabic LLM Leaderboard or in task-level evaluations built on your own data — report tokens per word alongside quality scores. Two models with equal accuracy and unequal fertility are not equal products.

This is how we approach Arabic LLM evaluation: a custom eval built on a sample of your own traffic, split by dialect, with tokens per word reported next to quality, so the shortlist is priced in the unit that actually reaches your invoice.

If you pretrain or extend a tokenizer, the corpus is the other lever. Bayanat Knowledge licenses 2,700 Arabic books and professionally edited MSA covering material through 2026; dialectal text at volume is a custom collection. Any token count for these collections names the tokenizer and separates raw from cleaned text, so you can check it against your own fertility measurements.