What is GRPO, and why does it matter that it eliminates...

GRPO functions by sampling G completions for each prompt and calculating a self-contained advantage estimate based on the group’s statistics. By removing the fourth model copy—the value network—the architecture achieves a significant reduction in peak VRAM usage. Furthermore, it avoids the inherent training instabilities often found when training a separate critic model, making it a robust choice for complex tasks like the reasoning-chain optimization used in DeepSeek-R1.