TAPAS is just BERT. So what does it actually add?

TAPAS modifies the input representation of BERT rather than the attention architecture itself. It uses row-major serialization to map cells to indices, but because standard BERT positional signals are insufficient for tabular structure, TAPAS adds three specific lookups: column, row, and rank embeddings. The rank embedding is particularly critical as it encodes value order, allowing the model to perform comparisons without needing to infer them from surface strings.