What is memory coalescing and when does it fail?

Memory coalescing is a hardware optimization where the system issues a single memory transaction when 32 threads in a warp read 32 consecutive addresses that align with a 128-byte HBM burst section. This is highly efficient as it retrieves all required data in one go. Coalescing fails when threads access non-contiguous, strided, or random memory addresses. A common failure case is column-major access of a row-major matrix, where threads in a warp read elements with a stride equal to the row width, causing the hardware to touch 32 different cache lines and requiring up to 32 separate transactions.