You wrap the reranker’s forward pass in torch.compile,...

When input shapes vary, torch.compile cannot reuse the cached graph and must retrace, which is computationally expensive. The solution is to enforce a fixed-size, padded, and masked layout so that there is exactly one shape for the compiler to handle. Reverting to eager mode is a common but suboptimal reaction; fixing the input shape is the correct path to realizing kernel fusion benefits.