Your tables run 40 rows by 12 columns and the encoder h...

With a 512-token limit and 36 tokens required per row (12 columns * 3 tokens/cell), only 12 of the 40 rows fit. To handle this, you should shard the table into row groups while replicating the header in each shard. Since this breaks queries comparing rows across shards, you must implement a router that directs cross-row aggregate questions to a SQL engine. Avoid simple prefix truncation; instead, use retrieval over rows to select the most relevant content.