one year on
Andrej Karpathy releases tutorial video 'Let's build GPT: from scratch, in code, spelled out'
Andrej Karpathy posts a tutorial video walking through the implementation of a GPT-like model from the ground up, and the post draws 1,110 points and 104 comments on Hacker News.
January 17, 2023 — Andrej Karpathy released a new tutorial video today titled “Let’s build GPT: from scratch, in code, spelled out.” The tutorial is part of his “Zero to Hero” series, available on his personal website.
The video quickly rocketed to the top of Hacker News, gathering 1,110 points and over 100 comments in its first day. Commenters praised Karpathy’s ability to explain complex concepts from the ground up. Karpathy himself joined the thread, outlining the practical steps needed to replicate a full GPT system, from scraping massive datasets and securing thousands of GPUs to navigating the “training hell” of distributed systems.
In the thread, users debated the nature of intelligence and the difference between memorization and true understanding. Interest in large language models is high, and the video offers a rare, hands-on look at the mechanics behind the models that are reshaping the AI landscape.
The record
Praises Karpathy as an excellent 'thing explainer' and points to his active Discord community for assignments.
In a comment, breaks down the four rough steps to replicate a GPT, including data collection, procuring thousands of GPUs, pretraining with a fork of MegatronLM+DeepSpeed, and finetuning via the ChatGPT recipe.
Shares his experience being inspired by Karpathy's earlier char-rnn work and describes his own minGPT-inspired project that holds a world record for training to 94% on CIFAR10 in 600 lines of code.
One year later — open only if you can handle spoilers
The tutorial became one of the most-watched AI educational resources of 2023. Karpathy later returned to OpenAI in February 2023, and the video remains a canonical reference for newcomers to transformer models.
The Weekly Replay · free by email
This week, one year ago — every Sunday.
One email each Sunday: the week's replayed AI news, with the one-year-later annotations included. Written like it's breaking — dated like it isn't.
Free · double opt-in · unsubscribe anytime · privacy