Token Trees
An interactive viewer for how a language model's answers branch and converge, token by token. The example: 2,000 GPT-5.4-mini completions of two prompts that differ by one word, "History, written by AI" and "History, rewritten by AI".
What it shows
Ask a language model to complete the same thought a thousand times and you get a thousand answers that are almost, but not quite, the same. Token Trees makes that variation visible. The visualization tool merges every completion into a single tree, token by token, so you can see where the model's answers branch, which paths it takes most often, which it takes only once, and where different openings converge on the same ending.
The example
The viewer opens on 2,000 completions from GPT-5.4-mini, 1,000 for each of two prompts that differ by a single word:
- Complete the following thought: "History, written by Artificial Intelligence"
- Complete the following thought: "History, rewritten by Artificial Intelligence"
It offers three trees:
- History, written and rewritten by Artificial Intelligence: both prompts together, keeping only the branches that reach a shared ending, so you can see where "written" and "rewritten" end up saying the same thing.
- History, written by Artificial Intelligence and History, rewritten by Artificial Intelligence: the complete branching tree for each prompt, from the opening words to the most common endings.
Color marks how common each branch is, from rare (2% or less) to common (60% or more). You can search for a token across trees, step through a tree branch by branch, and export a view as a PDF.
How it works
- Frequency Counts. Each completion is re-tokenized with the model's own tokenizer and the observed branchings are counted. That's exact for OpenAI and open-weight models and approximate for Claude and Gemini, whose tokenizers aren't public. So it works with any model and any API.
- Root-alignment. An n-gram matcher finds phrases that many completions share and uses them as roots, so trees start at genuine branch points.
- Beginnings and endings. A beginning tree (how answers open) and an ending tree (how they close) are stitched into one.
The earlier version
The first Token Decision Tree was created to illustrate the differences between how Historians create "critical fabulations" and how LLMs fabulate. For a publication in Digital Memory Dialogues regarding AI and Holocaust representation, the earlier version was used to visualize 1,000 GPT-4o completions of a request for the testimony of a Holocaust survivor. The project clustered sentences from the testimony using embeddings and K-Means. The visualization then leveraged the model's logprobs to show the alternatives it considered but didn't choose. The new version drops both dependencies, so the same analysis runs on any model's output. The original visualization is still online.