← All questions
AgentsHard· In-Context Learning· Evaluation

The task changed from four-way classification to free-f...

The change to open-ended generation invalidates label-balance filters and normalization techniques that rely on a finite set of labels. Instead of these methods, you should probe the first generated token or a constrained decision prefix. Freeze an order chosen offline and measure the accuracy spread across 10 random orders. If the spread is small, order is not the primary issue; if it is large, you have a metric to decide how much engineering effort is required.

Code playground (Python)