What is the difference between shared memory and global...

Shared memory is located on the Streaming Multiprocessor (SM) and provides fast access with approximately 20 cycles of latency. In contrast, global memory (HBM) is off-chip, offering much larger capacity but at the cost of 300-600 cycles of latency. Because of this massive latency gap, any kernel that repeatedly reads the same data from HBM instead of caching it in shared memory will be bottlenecked by memory latency. The optimal design pattern is to load data from HBM into shared memory once, perform all necessary computations on that tile using registers or shared memory, and then write the result back to HBM.