Why not train at long context from the beginning?

The primary reason is cost; because attention mechanisms scale at O(L2), training on long sequences is orders of magnitude more expensive per token. Additionally, a model in the early stages of training is not yet capable enough to effectively utilize distant context, meaning the compute cycles spent on long sequences would be largely wasted. Mid-training extension is more efficient because it amortizes the high cost of long-context training over a checkpoint that already possesses strong language priors.