How does your router avoid recomputing the same system...

To avoid redundant computation, use prefix caching. Hash the system prompt tokens and store the associated KV blocks on each replica. The router then directs requests to the replica that already holds the matching cache. Cache eviction should follow an LRU policy, and the KV store must be block-granular (similar to PagedAttention) to prevent the need to pin entire sequences in memory.