Isn’t recomputing part of the forward pass during the b...

While recomputing tiles during the backward pass adds compute cycles, it is actually faster than the alternative of reading the precomputed N x N matrix from HBM. The recomputation occurs while the GPU is waiting for HBM loads, effectively hiding the compute cost. In practice, the extra compute costs are negligible (e.g., ~0.07 ms) compared to the time saved by avoiding HBM transfers (~0.32 ms), leading to a net speedup and a massive reduction in memory usage.