Guide
Why tree-based models still beat deep learning on tabular data
What the Grinsztajn et al. benchmark found, why gradient-boosted trees keep winning on medium-sized tables, and where the finding stops applying.
The short answer
On typical tabular data, meaning mixed-type columns like business, health or finance tables, tree-based models such as XGBoost and random forests remain the strongest default. A 45-dataset benchmark found they stay state of the art at around 10,000 samples. Start with gradient-boosted trees and make a neural network earn its place.
Natural-language data tools in our data and analytics category answer questions about tables. Under them, the modeling question for a table is older and more settled than the hype around deep learning suggests.
Do tree-based models still beat deep learning on tabular data?
On typical tabular data, meaning rows of mixed-type columns like those in business, health or finance datasets, tree-based models such as XGBoost and random forests remain the strongest default. Deep learning has transformed text and images, but its superiority on tables is not established 1. If you are starting a tabular project, begin with gradient-boosted trees and make a neural network earn its place.
What did the Grinsztajn et al. benchmark actually do?
Grinsztajn et al. benchmarked standard and novel deep learning methods against tree-based models across many datasets and many hyperparameter combinations. They defined a standard set of 45 datasets from varied domains with clear tabular characteristics, and a methodology that accounts for both fitting models and searching for good hyperparameters 1.
The headline: tree-based models remain state of the art on medium-sized data (around 10,000 samples), even without crediting them for their much faster training. The authors also released a standard benchmark and the raw results of a 20,000 compute-hour hyperparameter search for each learner 1. Note the scope: "medium-sized" and "typical" are part of the claim. The paper does not say trees win on every table.
Why do trees win on tables?
The authors investigate the gap empirically by finding data transformations that narrow or widen it, which points to the differing inductive biases of trees and neural networks. That leads to three challenges for anyone designing tabular neural networks 1:
- Be robust to uninformative features. Real tables carry many columns that do not help prediction. Trees are comparatively unbothered by them.
- Preserve the orientation of the data. The columns of a table have meaning individually. Representations that mix features in rotation-invariant ways throw that structure away.
- Learn irregular functions easily. Target functions over tabular features tend to be jagged rather than smooth, which suits axis-aligned splits better than smooth networks.
Which model should you start with?
| Situation | Reasonable default |
|---|---|
| Typical table, up to roughly tens of thousands of rows | Gradient-boosted trees or random forest |
| Many noisy or irrelevant columns | Trees, or do feature selection before any neural model |
| You need end-to-end differentiability (joint training with other deep components) | A neural network, and budget for tuning |
| Much larger data, or tables mixed with text or images | Re-test: the benchmark's conclusion is anchored on medium-sized data |
Whatever you pick, evaluate with the same hyperparameter-search budget for each model family. The paper's methodology builds that fairness in, and unequal tuning is a common reason a neural network looks better than it is.
Where does the evidence stop?
- The result is empirical, on a curated set of 45 datasets. Your data may differ 1.
- Newer tabular-specific architectures and pretrained tabular models have appeared since 2022. For a current claim about the best tabular model, check recent benchmarks instead of relying on this paper alone.
How does this apply to AI analytics tools?
A natural-language analytics tool and a predictive model answer different questions, but both rest on correct handling of tables. When you test an analytics tool, follow the data and analytics category: use a dataset where you already know the answers and check every number. Fair comparison also needs clean train and test splits. See how to read an evaluation and the glossary for related terms.
Evidence in this article Third-party research Work published by researchers or institutions other than us or the vendor. We link it; we have not reproduced it.
Frequently asked questions
Do tree-based models beat deep learning on all tabular data?
No. The benchmark's claim is scoped to typical, medium-sized tables of roughly 10,000 samples. It does not say trees win on every table, and larger or mixed text-and-image data should be re-tested.
Why do trees do better on tables?
The authors point to differing inductive biases. Trees are robust to uninformative features and handle irregular target functions, while many neural networks mix features in ways that discard the meaning of individual columns.
What is the right default for a new tabular project?
Start with gradient-boosted trees or a random forest, then test whether a neural network adds anything, giving every model family the same hyperparameter-search budget.
Is this result still current?
It dates from 2022. Newer tabular-specific architectures and pretrained tabular models have appeared since, so check recent benchmarks before making a claim about the best model today.
Sources
- Why do tree-based models still outperform deep learning on typical tabular data? Grinsztajn, Oyallon, Varoquaux. NeurIPS 2022 (Datasets and Benchmarks). Third-party research paper, as supplied. The figures quoted here are from its abstract and were not re-checked by the site build.