← All questions
Retrieval & EmbeddingsHard· BM25· Term Frequency· Mathematical Modeling

BM25’s tf term is a rational function. Log damping also...

Log damping is unbounded, meaning an adversary could theoretically outbid an honest document by simply adding more copies of a term. The rational form used in BM25 caps the contribution at k1 + 1, providing a clear limit. Furthermore, k1 acts as a tunable budget, where the first occurrence is worth exactly 1 at average length, allowing developers to control how much weight subsequent occurrences carry.

Code playground (Python)