Tabular Deep Learning

In the machine learning literature, the evaluation of models trained on tabular data is often performed on academic datasets that may not accurately reflect real-world use cases. This paper aims to address this issue by critically assessing the quality of popular benchmark datasets and identifying their limitations. We find that many of these datasets are synthetic, represent non-tabular modalities such as images, or consist of snapshots of real-world phenomena or socio-demographic data that are more suited for visualization and analysis rather than prediction. Moreover, these datasets often lack canonical and representative test sets, and some even have data leakage issues. To make the evaluation of tabular models more reliable, this paper constructs a new benchmark consisting of seven industrial-grade datasets that represent real-world scenarios more adequately. In particular, all datasets in our benchmark possess a temporal shift between the train/test subsets, which reflects the fact that in practice models trained on “older” datapoints are to be applied to “newer” ones. Furthermore, we assess the performance of state-of-the-art tabular DL models on these datasets and discover that some recent findings are not consistently beneficial and simpler methods are often preferred.