How would you correctly benchmark a custom CUDA kernel’...

Benchmarking a custom CUDA kernel requires careful attention to the asynchronous nature of GPU execution. First, warm-up iterations are mandatory to prime the JIT compilation and caches. Second, you must use torch.cuda.synchronize() before and after the timed region to drain the GPU command queue, ensuring the timer measures actual GPU execution time rather than CPU dispatch time. Finally, results should be averaged over 20 or more trials using time.perf_counter() to ensure statistical significance.