Can we retrain a tokenizer and swap it into an existing...

Swapping a tokenizer is not possible without full re-pretraining because the model’s embedding and unembedding matrices are strictly indexed by token ID. A new vocabulary introduces new IDs, meaning the existing trained weights in the embedding rows no longer correspond to the new token representations. Alternatives include continuing pre-training with vocabulary extension (which is expensive and imperfect), serving a separate model for high-fertility languages, or simply accepting the cost differential.