Foundation Models and Tabular Data: Can GPT Generate Your Synthetic Dataset?
Large language models can write code, summarize documents, and carry on conversations. Can they also generate synthetic tabular data? The answer, as of 2026, is "sort of, but probably not the way you want."
The Appeal
LLMs like GPT-4 and Claude have been trained on massive text corpora that include CSV files, SQL dumps, API documentation, and countless descriptions of data distributions. They've absorbed a remarkable amount of implicit statistical knowledge. Ask GPT-4 to generate 100 rows of realistic patient demographic data and you'll get something plausible: reasonable age distributions, correlated conditions, realistic lab value ranges.
The appeal is obvious: no training required. No model selection. No hyperparameter tuning. Just prompt and go.
Where It Breaks Down
Distributional Infidelity
LLMs generate data that looks right to a human reader. It doesn't generate data that is statistically right. The marginal distributions are plausible (ages between 18-95, heights between 150-200 cm) but the joint distribution is made up. The correlations between age, BMI, blood pressure, and cholesterol in LLM-generated data reflect the model's pretraining, not your actual dataset.
This matters because the whole point of synthetic data is to preserve the statistical properties of a specific original dataset. An LLM that's never seen your data can't preserve its properties — it generates data from its general knowledge, not from your distributions.
Inconsistency at Scale
Ask an LLM to generate 100 rows and the result is coherent. Ask for 100,000 rows and the statistical properties drift. The 500th row's distributions won't match the 50,000th row's. There's no global consistency guarantee — each generation call is independent, and the implicit distributions shift with prompt phrasing, temperature settings, and even token sampling randomness.
Purpose-built generators (CTGAN, TVAE, GaussianCopula) learn a fixed distribution from the training data and sample from it consistently, no matter how many rows you generate.
No Privacy Guarantees
An LLM generating "synthetic" data from a prompt has no mechanism for differential privacy. It can't bound information leakage about individuals in a training set — because it doesn't operate on a training set in the traditional sense. If you fine-tune an LLM on sensitive data and then prompt it to generate similar records, the model may memorize and reproduce real records. LLMs are particularly prone to memorizing outliers and rare combinations — exactly the records that are most re-identifiable.
Cost
Generating a million synthetic rows via API calls to GPT-4 costs orders of magnitude more than running a local CTGAN or GaussianCopula. At $15-30 per million input tokens (2025 pricing), generating a substantial synthetic dataset via LLM API calls is economically absurd compared to training a purpose-built generator locally.
Where LLMs Actually Help
Schema Design and Augmentation
LLMs are good at generating realistic schemas for synthetic data: column names, data types, plausible value ranges, constraint descriptions. If you're building a synthetic data pipeline and need metadata for a domain you're unfamiliar with, an LLM can jumpstart the schema definition.
Synthetic Text Fields
Tabular generators handle numeric and categorical columns well but struggle with free-text fields (product descriptions, clinical notes, customer feedback). LLMs excel here. A hybrid approach: generate the structured columns with a tabular generator (preserving statistical fidelity) and fill in text fields with an LLM conditioned on the structured values.
Test Data for Development
For quick-and-dirty test fixtures where statistical fidelity to a specific dataset doesn't matter — you just need realistic-looking data to populate a UI or test an API — LLMs work well. The data won't match any real distribution, but that's fine when the purpose is UI testing rather than statistical analysis.
The Emerging Middle Ground: TabuLa and GReaT
Research is exploring purpose-built models that combine LLM-style architectures with tabular data awareness:
GReaT (Generation of Realistic Tabular data; Borisov et al., 2023) serializes tabular rows as text strings and fine-tunes a GPT-2 model to generate new rows. The fine-tuning on your specific dataset means the generator learns your distributions, not generic ones. Early benchmarks show competitive quality with CTGAN on some datasets.
TabuLa uses a similar approach with T5-family models, adding column-name conditioning to improve consistency across generated rows.
These approaches are promising but early-stage. They're more expensive to train than CTGAN (fine-tuning a language model vs. training a lightweight GAN), slower to generate (autoregressive token generation vs. single forward pass), and have limited DP integration.
The Practical Recommendation
For production synthetic data generation from a specific dataset with quality requirements:
Use purpose-built generators (CTGAN, TVAE, TabDDPM, GaussianCopula, PrivBayes). They learn your data's distribution, generate consistently at any scale, integrate with DP mechanisms, and produce evaluable quality scores.
Use LLMs for schema design, text field generation, quick test fixtures, and domain knowledge augmentation.
Watch the research on GReaT, TabuLa, and similar LLM-tabular hybrids. Within a few years, they may offer the best of both worlds: LLM-level flexibility with purpose-built-generator-level statistical fidelity.
References: Borisov et al. (2023), "Language Models are Realistic Tabular Data Generators" (GReaT); research on LLM memorization and privacy risks; CTGAN and TVAE benchmark comparisons; OpenAI and Anthropic API pricing documentation.