This live coding course by Sebastian Raschka provides a hands-on, step-by-step journey into building the core components of Large Language Models (LLMs) from scratch using Python. It is designed for learners who want to deeply understand how transformer-based models work at a fundamental level.
The course begins with setting up a Python environment for LLM development, followed by detailed implementation of text tokenization. You will learn how raw text is converted into tokens and then transformed into numerical token IDs that machine learning models can process.
Next, the course introduces special context tokens and explains their role in structuring input for language models. It then covers Byte Pair Encoding (BPE), a key technique used in modern tokenizers to efficiently represent large vocabularies.
You will also learn how to implement data sampling using sliding windows, which is essential for preparing sequential training data. The course continues with creating token embeddings, which convert tokens into dense vector representations, and positional encoding, which helps models understand word order.
Finally, the course introduces self-attention mechanisms and explains how attention weights are computed, forming the foundation of transformer architectures.
By the end of this course, you will have a practical understanding of how LLMs process text step by step, from raw input to attention-based reasoning systems.
