179 questions
No questions match those filters.
← All questions
System DesignHard· Capacity Planning· Inference· Scaling
How would you size the inference fleet for a new chat p...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansSizing an inference fleet requires a quantitative approach rather than guessing hardware requirements. Start by converting the 10M DAU into peak requests per second based on expected usage patterns.
Next, estimate the average number of tokens per request, including both prompt and completion, to calculate the total tokens-per-second demand. Finally, use these figures to work backward to the specific hardware configuration and the number of nodes required to support the load.