179 questions
No questions match those filters.
← All questions
Inference & ServingHard· KV-cache· Attention· Accuracy
After shipping chunk-level KV-cache reuse, accuracy reg...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansWhen chunks are cached in isolation, the model loses the ability to perform cross-document attention, which is essential for comparison tasks. Single-fact lookups remain unaffected because they only require information from a single chunk. A strong answer proposes checking fine-tune coverage on multi-document examples or exempting compare-style queries from the cache path. Blaming position-id reassignment is a common trap, as that would cause ordering confusion rather than specific comparison failures.