179 questions
No questions match those filters.
Finance wants the 70B replaced by a 7B to cut serving c...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe swap fails to reduce compute because the reading burden on the generator is proportional to the number of retrieved chunks. While a 7B model is smaller, the increased FLOPs required to process the retrieved context often negate compute savings. Research’s concern is valid regarding the model’s ability to process deep context, but a reranker can effectively move the reading burden off the generator. If the model still fails when provided with the correct gold chunk at rank 1, then the model’s inherent capability is the bottleneck, not the datastore.