»Forward
Pass.

Problem

Graph foundation models must choose between GNN-based methods that need lots of graph-specific data and LLM-based methods reliant on text attributes; the paper asks whether tabular foundation models can serve as a text-free GFM backbone instead.

Method

GTAlign pretrains a domain-agnostic graph encoder, then aligns graph structure to the input format of a tabular foundation model (LimiX-16M, also validated with TabPFNv2.5) via community-guided continual pretraining, enabling few-shot in-context node/graph classification.

Results

On five benchmarks (Cora, CiteSeer, PubMed, Photo, Computers), 5-shot node classification beats the strongest baseline BRIDGE by 5-25 points (e.g. Cora 88.65% vs 63.85%); graph-classification gains are smaller and it trails baselines at 1-shot.

Limitations

Graph-level transfer lags node-level transfer due to pooling information loss; evaluated on only two domains (citation networks, e-commerce); generalization to novel graph types beyond the pretraining mixture is untested.

Problem

Causal discovery from tabular observational data is slow and dataset-specific, and struggles when the underlying causal mechanisms vary; the goal is one model that works zero-shot across mechanism types.

Method

DAG-FM trains a transformer 'prior-fitted network' - TabPFN's recipe - on synthetic causal graphs spanning four mechanism families, then infers a DAG in one pass using a tabular row-column interaction block and a mixture-of-leaf-experts router.

Results

Beats prior amortized causal-discovery models (TabCausal, FoundCause) on synthetic data (F1 0.64 vs 0.54/0.57) and on the real Sachs dataset (F1 0.53 vs 0.33); scales to 500 variables or 50k samples on one GPU.

Limitations

Assumes causal sufficiency (no unobserved confounders); trained at a fixed sample size (n=1024) though it generalizes to others at test time; underperforms a baseline on the Causal Chamber dataset.

Problem

Existing causal-discovery algorithms are hand-tailored per data type and don't transfer; CDFM aims for one foundation model that does zero-shot structural inference across unknown causal mechanisms.

Method

Following TabPFN's playbook, CDFM pretrains on a massive synthetic space of structural causal models with a variational framework that treats the unknown causal mechanism as a latent variable, decomposed into tractable learning modules.

Results

On synthetic data (15 mechanism families) it reaches AUROC 0.888 vs 0.786 for TabCausal at 4,000 samples, and holds AUROC above 0.87 at 100 variables where the best competitor drops to ~0.75; also wins on Causal Chamber and Tubingen cause-effect pairs.

Limitations

Likely: only 15% of synthetic training batches include latent confounders, which may limit performance under real-world confounding; no computational-cost or scalability analysis is reported.

Problem

LLMs struggle to reason over large or messy web tables: long tables exceed context limits, and text-to-SQL breaks on tables that lack consistent structure.

Method

ProgramTab uses in-context learning to have the LLM write Python code that cleans and restructures a table, extracts the relevant rows/columns, then generates SQL for the final reasoning step.

Results

With GPT-4o-mini it reaches 89.6% on TabFact and 76.0% on WikiTQ, narrowly beating the strongest baseline H-STAR (+0.2 and +1.1 points); smaller consistent gains also hold with Llama-3.1-70B and GPT-3.5-Turbo.

Limitations

Margins over the best baseline are narrow; the authors note further preprocessing is needed for complex table structures, and code/SQL generation errors are the dominant failure mode, especially on WikiTQ.