Do You Even Need More Data? Rethinking LLM Deployment for Label-Scarce Tabular Classification with TabLens

Every tabular classification task requires sufficient labeled data. When labels are scarce, practitioners can collect more examples, encode domain expertise into features, or accept lower performance. Large language models offer an alternative by leveraging knowledge acquired during pretraining. This raises three questions: (i) can zero-shot inference reduce the need for labeled data; (ii) can a richer prompt reduce the need for more examples; and (iii) can encoded domain knowledge compensate for missing data? To answer each under explicit, testable conditions, we introduce TabLens, a benchmark of 91 real-world and synthetic datasets across eight domains, plus eight industrial datasets with free-text columns. First, below a complexity threshold estimable from a dataset's intrinsic dimensionality before running an LLM, zero-shot inference matches the accuracy of up to about 16 labeled examples. This holds whether the model generates an answer or, more cheaply, scores candidate labels by log-probabilities, which loses almost no accuracy except for the smallest models. Above the threshold, the advantage disappears. On datasets with free-text columns, however, zero-shot LLMs outperform 16-shot TabPFN-3 Plus without manual feature engineering. Second, a richer prompt reduces the number of required demonstrations only when it includes both real column names and a plain-language task description; providing only one can be worse than neither, particularly at higher shot counts. Third, on LLM-synthetic datasets, explicit if--then expert rules help only when the prompt provides enough task context to interpret them, and even then, they substitute for at most about 16 real examples. Together, these results establish testable conditions under which LLMs can reduce data-collection and feature-engineering requirements for tabular classification.