179 questions
No questions match those filters.
← All questions
Inference & ServingVery Hard· Sequence Parallelism· Memory Management
How does sequence parallelism change the parallelism st...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansFor 1M-token context models, activations—which scale with sequence length, batch size, and dmodel—become the primary memory bottleneck. Sequence parallelism (SP) is mandatory here; it shards the sequence dimension across TP ranks, reducing activation memory by the TP degree. This requires replacing TP all-reduces with all-gather operations before QKV projections and reduce-scatter operations after output projections, preventing activation memory from exceeding GPU limits.