The paper “Chain-Of-Table” proposes expanding upon the existing Chain-Of-Thought (CoT) reasoning, particularly for complex table-based data with Large Language Models (LLMs).

What’s New

The authors propose adding intermediate tables iteratively into the reasoning process instead of free-form-text methods.

How It Works

Uses a three-phase approach and a greedy search strategy for tabular reasoning instead of relying on self-consistent results or examples:

1. Creates a dynamic plan based on the question and table to select the best table operation, like adding rows or columns
2. Generates arguments for the chosen operation, such as specifying column names
3. Creates a final query from the operation and argument that best answers the question

Performance

The paper reports superior performance compared to existing methods like BINDER and DATER. The results also show less decline in performance as the table size increases or if the number of operations needed to answer the question increases.

Thoughts

For working with structured and semi-structured data in LLMs, program-aided methods like DATER have shown promising results. While Chain-Of-Table performs better on benchmarks, I would be interested in seeing more practical examples.

Read the paper