One colleague wants to fix a positional-bias regression...

The truncation and reranking approach is more efficient and directly targets the cause of the bias. From a compute perspective, a 4x larger context window leads to a 16x increase in the attention bill, which is often unsustainable given a fixed latency budget.

Unless you have evidence that the larger checkpoint has a shallower trough—indicating it was genuinely trained for the longer context rather than just having a larger accepted window—the truncation and reranking path is the correct choice. The larger model often suffers from the same ‘LongChat-style’ failure modes, making it a risky and expensive investment.