one year on
Interactive 3D visualization lets you walk inside a transformer model
A free, browser-based tool offers a fly-through view of GPT-Nano's internals, making the matmuls and attention heads interactive and animated.
A new interactive 3D visualization by Brendan Bycroft lets users fly through the inner workings of a small GPT-style language model, turning abstract matrix operations into interactive, animated blocks.
The tool, hosted at bbycroft.net/llm, presents a browser-based walk-through of a small GPT-style model called GPT-Nano.
On Hacker News, where the link garnered 1592 points, the reception was enthusiastic. One user remarked it was ‘The visualization I’ve been looking for for months.’ Another noted that it helps ‘realize how an LLM actually works from the ground up.’ A chorus of commenters appreciated the open-source nature, with one calling it ‘a spectacular pedagogic device.’ A few pointed out that the first array’s size of 48 could be perplexing without context; one commenter pointed readers to minGPT’s model.py and called it an architectural decision.
Bycroft has also published the source code on GitHub.
Submitted the link to Hacker News where it quickly gained 1592 points and 131 comments.
Called it an excellent tool to realize how an LLM actually works from the ground up, though noted the unexplained 48-element array might confuse newcomers.
Praised the interactive wizzing around, and shared their own unorthodox array-visualization method using colored axes.
Linked to the author's Twitter thread with extra context.
Said it was the visualization they had been looking for for months, and called it a gift.
One year later — open only if you can handle spoilers
The visualization became a staple reference in introductory transformer courses and blog posts. Bycroft later added support for larger models and multilingual captions, but the original nano-gpt walkthrough remains the most visited page on his site.
The Weekly Replay · free by email
This week, one year ago — every Sunday.
One email each Sunday: the week's replayed AI news, with the one-year-later annotations included. Written like it's breaking — dated like it isn't.
Free · double opt-in · unsubscribe anytime · privacy