← All questions
LLM FoundationsHard· Table Representation· Attention Mechanisms· Serialization

A colleague argues the row and column embeddings are th...

The colleague is correct that absolute row and column IDs impose a rigid, often meaningless, order on relations. However, Markdown is not a solution; it merely re-encodes the same ordering using pipe characters while increasing sequence length by 2.66x and attention costs by 6.8x. A better approach is to use relative structural attention biases to maintain structural awareness without relying on absolute position. Markdown may only be appropriate for small tables where a long-context model is available and the team prefers simple serialization over model maintenance.

Code playground (Python)