»Forward
Pass.

Problem

Discrete choice demand models need individual-level preferences from sparse panel data, traditionally via hierarchical Bayesian (HB) MCMC. Tabular foundation models assume row independence, but choices are set-dependent with persistent per-consumer heterogeneity.

Method

TabPFN applied via in-context learning, with choice sets reformulated as a 'long' table (one row per alternative, rank/set-summary features encoding competitive context) plus consumer ID as a categorical feature so the model pools across consumers while personalizing.

Results

On a 98-consumer yogurt panel, the best TabPFN setup beat the strongest HB mixed-logit baseline by 8% holdout log-likelihood (-0.648 vs -0.725) and 3.6% hit rate (0.772 vs 0.745), 16x faster. Gains concentrated in medium-data consumers (11-40 purchases).

Limitations

HB still wins for very sparse (<10 occasions) and very deep (>40) histories. TabPFN's prior isn't interpretable (no inspectable posterior draws for uncertainty). Tested only on one product category; not a substitute for structural models needed for counterfactuals.

Problem

Power-system dynamic security assessment via ML needs a separate classifier per contingency and huge labeled datasets from expensive time-domain simulations, and existing classifiers fail on new, unseen contingencies.

Method

Apply TabPFN (pretrained on 100M synthetic datasets) via in-context learning, no retraining/tuning, plus a new 'electrical distance coordinate' feature encoding contingency location so one model handles multiple and unseen contingencies.

Results

On IEEE 68-bus (22 contingencies): TFM hits ~90% Macro F1 with just 120 labeled samples/contingency (vs. baselines needing ~100x more). Multi-contingency: 93.86% vs 83.43% balanced accuracy. Unseen contingencies: 10 extra samples matched oracle transfer learning (F1 0.87 vs 0.84).

Limitations

TabPFN is capped at 100,000 samples/2000 features, inadequate for larger grids; predictions aren't guaranteed correct and need operator validation; compute cost grows with feature count; tested only on one 68-bus benchmark.

Problem

Conventional evaluation pools synthetic records into static distributions, missing timestamps that run backwards or repeat and trajectories no real entity followed — a problem for time-sensitive uses like finance and healthcare.

Method

A taxonomy-guided protocol classifies datasets by time representation, sampling regularity, trajectory dependence, and schema structure, then selects applicable metrics (timestamp validity, cross-sectional alignment, within-entity dynamics, relational structure). Applied to 8 generators across 13 datasets, 6 domains.

Results

Static vs. temporal rankings diverge sharply (Spearman rho 0.36-0.53, 27-36% inversions). Autoregressive models top timestamp validity (uniqueness >0.97) but one scores worst on privacy (0.267 vs 0.999 for another); no model wins on all dimensions.

Limitations

Authors stress no model wins on all four evaluation dimensions simultaneously — strong timestamp fidelity can coexist with weak relational structure or privacy leakage — so results must be read per-dimension, not as one leaderboard score.

Problem

Classification heads on frozen multimodal encoders (image/text/audio) are often poorly calibrated, especially in few-shot settings. The authors test whether TabPFN's in-context tabular learner can serve as a gradient-free head that improves calibration and accuracy.

Method

Frozen encoders produce embeddings that are z-normalized, PCA-reduced to low dimensions (8-96), then fed to TabPFN v2.5 as in-context tabular examples; one forward pass outputs calibrated class probabilities with no gradient updates.

Results

Over 22,820 episodes (14 datasets, 11 encoders, 3 modalities): TabPFN cut NLL 48-62% and ECE 2.1-5.3x vs. the average of 8 baselines (kNN, logistic regression, SVM, boosted trees, MLP); accuracy gains concentrate at ≥50 shots and ≤32 PCA dimensions.

Limitations

Fine-tuning comparisons used a fixed schedule on few encoders, no hyperparameter search; PCA becomes a bottleneck at large k/d; findings apply only to classification, not shown to generalize to regression or retrieval tasks.

Problem

Whether Kolmogorov-Arnold Networks (KANs) justify their extra cost over MLPs on tabular classification, tested under fixed, untuned hyperparameters across binary, multiclass, multilabel, and ordinal tasks on 12 UCI/Kaggle/MULAN datasets.

Method

Paired vanilla KAN (spline edge activations, grid 5) vs. MLP (ReLU, same hidden width) trained identically per dataset, compared via accuracy, F1, paired hypothesis tests, and effect sizes.

Results

KAN beat MLP on accuracy in 9/12 and F1 in 10/12 datasets, with a significant but only medium aggregate effect (Cohen's d=-0.46, p=0.001). KAN used ~16x more parameters and was far slower (training >30s vs <5s; inference >100ms vs 3ms).

Limitations

Hyperparameters were fixed/untuned for both models (no search for optimal performance), and multilabel datasets trained for only 20 epochs vs. 100 elsewhere due to compute constraints, likely limiting KAN convergence there; only tabular data tested.

Problem

Clinical prediction usually needs bespoke fusion architectures to combine structured EHR variables (labs, vitals) with free-text notes. The authors ask whether a single LLM, given everything as plain text, can replace these specialized multimodal pipelines.

Method

Serialize structured variables as key-value text pairs, concatenate with clinical notes into one sequence, and fine-tune off-the-shelf LLMs (ModernBERT, Llama 3.1, DeepSeek-R1-Qwen, Qwen3, Gemma) with no architectural changes.

Results

On MIMIC-III mortality, serialized ModernBERT/DeepSeek-R1 hit AUROC 0.93 vs 0.90 for a multimodal fusion baseline. On graft failure, they beat a deployed gradient-boosting system (0.90 vs 0.89 AUROC). On triage, DeepSeek-R1 got macro-F1 0.56 vs fusion's 0.40.

Limitations

Serializing structured data inflates token count, straining context windows for patients with long histories. No cross-institutional external validation was performed; interpretability and calibration are left as open problems.

Problem

Databases can have thousands of tables/columns that don't fit in an LLM's context, so text-to-SQL needs a prior retrieval step to find the right schema elements. This step is rarely benchmarked on its own, so its difficulty is underappreciated.

Method

Convert five text-to-SQL datasets (Spider, BIRD, BEAVER, LiveSQLBench) into retrieval benchmarks, then fine-tune embedders label-free by synthesizing LLM queries from schema documents and mining granularity-aware hard negatives, contrastively.

Results

Arctic-Embed-M (305M) goes from 60.4 to 75.6 recall@10, beating BM25 (47.7) and rivaling Qwen3-Embedding-4B/8B (78.4/77.8) despite being 10-25x smaller; leave-one-corpus-out and leakage audits confirm genuine generalization.

Limitations

English-only schemas/questions; requires the target schema corpus available at adaptation time, so it doesn't handle cold-start databases; gold labels come from SQL parsing and can miss implicit column use; only two backbones tested.

Problem

Early CKD screening is hard in resource-limited settings without lab access to gold-standard markers (creatinine, eGFR, uACR); the paper asks whether LLMs can screen for CKD zero-shot using only cheap, community-accessible features.

Method

Feature selection narrows patient records to 14-24 clinically relevant, non-lab variables, serialized as list-format prompts fed to four LLMs (LLaMA-3 8B, Qwen-3 8B, Mistral 7B, GPT-4o-mini) for zero-shot classification via log-probability scoring, no fine-tuning.

Results

Selected-feature zero-shot beat all-feature prompts (p<0.001) across 3 country datasets; best: GPT-4o-mini 88.0% balanced accuracy/0.943 AUROC on India (n=400) — approaching but not beating prior supervised ML's 90.4% CV accuracy.

Limitations

Datasets are small-to-moderate (56-400 patients) and limited to 3 countries; Dataset-3 (UAE) has no CKD-negative cases, precluding full binary evaluation; no comparison against TabPFN/XGBoost or other tabular baselines is reported.