You have two million unlabeled domain sentences and one week. How do you build a domain sentence encoder and validate it?

Start by measuring an off-the-shelf model on your domain, as the evaluation set is the most valuable artifact. If you proceed with training, use a dropout recipe with specific parameters (τ and p). The evaluation must be rigorous: avoid using pairs generated by the same family of edits used in training, as this would merely score the encoder on the distortions you introduced. Instead, use naturally occurring pairs from existing data structures.