← All questions
System DesignHard· LLMOps· Evaluation· Maintainability

Three generators sit behind one router. One engineer wa...

Begin by assessing the evaluation size; a four-point gain on a small set may simply be noise. Address the maintainability argument by noting that while KV caches are per-model (making per-model templates technically feasible), multiple templates increase the on-call surface and eval baselines. Establish a policy where a house template is the default, but per-model overrides are permitted if they survive held-out confirmation. Every override must be pinned to a model version to prevent silent inheritance, and if an override cannot be reproduced on a fresh set, the house template wins.

Code playground (Python)