Should you fund sparse-autoencoder (SAE) features for interpretability or a cross-encoder reranker for performance?

While SAEs are a principled way to address the oversubscription of the residual stream, they are fitted to a specific model and layer. A reranker is more robust because it improves the retrieval metric directly and survives base model upgrades. You should keep a stream-patching capability for incident analysis, but prioritize the reranker for production impact unless failures are so widespread that they indicate a fundamental weight-level issue.