You are on a hosted generator with no gradient access....

When working with a hosted generator where you lack gradient access, the model parameters are inaccessible, and the discrete nature of the output (the arg max between generation stages) zeroes out the gradient. To train in this environment, you must bypass direct backpropagation. The solution is to define a substitute signal where you score candidate outputs based on the generator’s assigned likelihood for the gold answer, effectively optimizing the system through this proxy signal rather than direct weight updates.