Should we perform continued pre-training on 200,000 pro...

Continued pre-training is the expensive axis of development. With 200,000 documents, most individual facts remain rare within the training data, leading to poor recall. Furthermore, pre-training amplifies co-occurrence bias and cannot provide the freshness required for a product catalog. Retrieval is a more scalable and flexible solution that avoids the compute-heavy overhead of retraining.