179 questions
No questions match those filters.
← All questions
Inference & ServingHard· System Design· PEFT· Deployment
Should you merge customer-specific adapters into the ba...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansMerging adapters into the base model for 40 customers results in 560 GB of weights and prevents batching, which is inefficient. Keeping adapters unmerged allows for a single 14 GB base model with small, shared adapter overhead. The cost is a minor increase in FLOPs for the low-rank matmul. If one customer accounts for 80% of traffic, merge that specific adapter for performance, but keep the rest unmerged to maintain fleet efficiency.