Watch · Statistics and machine learning
Reinforcement Learning with GRPO, Clearly Explained!!!
The Brief
- Explains the GRPO loss function for reinforcement learning of a large language model, one step at a time.
- Covers the main ideas of GRPO and a few of its details.
- Includes a summary and a section on the KL divergence term.
Why watch · It walks through how GRPO, a reinforcement learning method used to train reasoning models, simplifies alignment of large language models.
- GRPO
- Reinforcement learning
- Large language models
- KL divergence
Summarised by Claude from the video’s title and description – no transcript was available, so check the video for the details.
Chapters
Show the full video description ›Hide the video description
In 2024, the GRPO method for training Reasoning Models took the world by storm by simplifying the Reinforcement Learning stage of aligning a Large Language Model. In this StatQuest, we go through the GRPO loss function one step at at time, learning exactly what it does and how it does it. BAM! NOTE: This StatQuest assumes that you are already familiar with... The Main Ideas of Neural Networks: https://youtu.be/CqOfi41LfDw Backpropagation: https://youtu.be/IN2XmBhILt4 Decoder-Only Transformers: https://youtu.be/bQ5BoolX9Ag If you'd like to support StatQuest, please consider... Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join ...buying a book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store... https://statquest.org/statquest-store/ ...or just donating to StatQuest! paypal: https://www.paypal.me/statquest venmo: @JoshStarmer Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer 0:00 Awesome song and introduction 1:00 The main ideas of GRPO 6:18 A few details about GRPO 8:15 GRPO, one step at a time 23:28 Summary 24:20 The KL Divergence term #StatQuest