Andrej Karpathy today released llm.c, a new open-source project that implements large language model training in ‘simple, pure C/CUDA’ with no need for 245MB of PyTorch or 107MB of cPython. The repository currently supports reproducing the GPT-2 series, including the 124M parameter model, and says it can reproduce other models from the GPT-2 and GPT-3 series.

The project includes a reference CPU implementation in roughly 1,000 lines of clean C code, as well as a faster CUDA version that the README says is a bit faster than PyTorch Nightly, by about 7%. Karpathy frames the project as both an educational tool and a practical training framework, aiming to keep the mainline code simple while incorporating library calls like cuBLAS and cuDNN for speed.

On Hacker News, where the project drew 1,050 points and 168 comments, the reception was enthusiastic but tempered. Commenters noted that the CUDA portion is still ‘aspirational’ at this point, and some raised concerns about hand-rolled gradients. However, many welcomed the stripped-down approach as a refreshing alternative to Python’s heavy dependency chains.