International
The Morning Science

What science found today

‹
 

Watch · Statistics and machine learning

Reinforcement Learning with GRPO, Clearly Explained!!!

The Brief

  • Explains the GRPO loss function for reinforcement learning of a large language model, one step at a time.
  • Covers the main ideas of GRPO and a few of its details.
  • Includes a summary and a section on the KL divergence term.

Why watch · It walks through how GRPO, a reinforcement learning method used to train reasoning models, simplifies alignment of large language models.

  • GRPO
  • Reinforcement learning
  • Large language models
  • KL divergence

Summarised by Claude from the video’s title and description – no transcript was available, so check the video for the details.

Show the full video description ›

In 2024, the GRPO method for training Reasoning Models took the world by storm by simplifying the Reinforcement Learning stage of aligning a Large Language Model. In this StatQuest, we go through the GRPO loss function one step at at time, learning exactly what it does and how it does it. BAM! NOTE: This StatQuest assumes that you are already familiar with... The Main Ideas of Neural Networks: https://youtu.be/CqOfi41LfDw Backpropagation: https://youtu.be/IN2XmBhILt4 Decoder-Only Transformers: https://youtu.be/bQ5BoolX9Ag If you'd like to support StatQuest, please consider... Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join ...buying a book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store... https://statquest.org/statquest-store/ ...or just donating to StatQuest! paypal: https://www.paypal.me/statquest venmo: @JoshStarmer Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer 0:00 Awesome song and introduction 1:00 The main ideas of GRPO 6:18 A few details about GRPO 8:15 GRPO, one step at a time 23:28 Summary 24:20 The KL Divergence term #StatQuest