← All questions
Inference & ServingEasy· Efficiency· Computational Cost· Performance

Why use KenLM instead of a BERT-based classifier for qu...

The primary constraint is the computational budget. Scoring 10^12 tokens with a BERT-based classifier at ~10ms per sentence would cost millions of GPU-hours. KenLM, however, operates on CPUs at a rate of 10^7 tokens per second, allowing the entire web to be processed in hours on a modest CPU cluster.

Code playground (Python)