Our corpus is 500M chunks. A refresh is 1.41×1019 FLOPs...

A 32,768 batch size provides sufficient weight to compete with top-ANN negatives, but it suffers from two major drawbacks. First, it consumes significant activation memory. Second, it spreads the signal across many near-identical directions, creating a diffuse gradient that often fails to differentiate between topically adjacent passages. By mining from a 10% corpus sample, you can achieve similar hardness results with significantly reduced compute, allowing for more frequent refreshes without the overhead of a full corpus scan.