179 questions
No questions match those filters.
← All questions
LLMOps & ProductionHard· CUDA· Kernel Launch· Optimization
A profiler shows 60% of GPU time is in CudaLaunchKernel...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansHigh CudaLaunchKernel time occurs when a model contains many small operations, such as an unfused GELU implemented as multiple element-wise operations. Each launch takes 5-20 μs, which becomes a significant percentage of runtime in fast models. To fix this, use torch.compile to automatically fuse element-wise chains, write a custom Triton or CUDA kernel to fuse the compound operation, or increase batch size or hidden dimensions to ensure each kernel performs enough work to amortize the fixed launch cost.