one year on
Sebastian Raschka publishes three-hour workshop on building LLMs from scratch
A detailed coding workshop covering LLM architecture, pretraining, and finetuning draws reader comments on Substack.
Sebastian Raschka, author of ‘Build a Large Language Model (From Scratch),’ released a three-hour coding workshop video covering the full pipeline of building, training, and using LLMs. The workshop walks through tokenizer implementation, GPT-2 and Llama 2 architecture, pretraining, loading pretrained weights, instruction finetuning, and benchmark evaluation. Raschka also provides companion code and links to his book and the LitGPT repository.
The workshop received 445 likes and 28 comments on Substack. Readers praised the hands-on format and depth, with one commenter writing, ‘THANK YOU so much for doing this! I have bought the early release of your book via MEAP and it is fantastic.’ Another reader asked permission to add the workshop to a machine learning road map.
Raschka says the last time he did this a few months ago, it was so well received that he thought it might be nice to do another one. The video is freely available, and Raschka continues to offer his book and paid Substack subscription as optional support for his work.
The post received 445 likes and 28 comments. Many readers expressed gratitude for the workshop and discussed incorporating it into learning roadmaps.
One year later — open only if you can handle spoilers
The workshop remains a widely referenced resource for developers learning LLM internals. Raschka's book went on to become a bestseller in its category, and his LitGPT project evolved into a popular framework for finetuning open-weight models.
The Weekly Replay · free by email
This week, one year ago — every Sunday.
One email each Sunday: the week's replayed AI news, with the one-year-later annotations included. Written like it's breaking — dated like it isn't.
Free · double opt-in · unsubscribe anytime · privacy