How did GPT-3 build its quality classifier, and why did...

The GPT-3 quality classifier was built by treating Reddit karma as a proxy for document quality. By using web pages linked on Reddit with three or more upvotes as positive samples and raw Common Crawl data as negative samples, the researchers created a large-scale training set without manual labeling. A fastText linear classifier was then trained on these labels and applied stochastically to the corpus to retain high-scoring documents, rather than using a hard cutoff.