What is an advantage function and how does it connect t...

The advantage function quantifies how much better or worse an action is compared to the average action in that state. In the context of outcome-reward reinforcement learning for language models, the value function V(s) does not require future-reward bootstrapping; instead, it is simply the mean reward across G rollouts for the same prompt.