Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models. Javanmard, A., Mirzasoleiman, B., & Mirrokni, V. In The 43rd International Conference on Machine Learning (ICML 2026), Seoul, South Korea, July, 2026. arXiv:2603.01293 [cs]
Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models [link]Paper  doi  abstract   bibtex   
Large Language Models (LLMs) are pretrained on massive datasets and later instruction-tuned via supervised fine-tuning (SFT) or reinforcement learning (RL). Best practices emphasize large, diverse pretraining data, whereas post-training operates differently: SFT relies on smaller, high-quality datasets, while RL benefits more from scale, with larger amounts of feedback often outweighing label quality. Yet it remains unclear why pretraining and RL require large datasets, why SFT excels on smaller ones, and what defines high-quality SFT data. In this work, we theoretically analyze transformers trained on an in-context weight prediction task for linear regression. Our analysis reveals several key findings: (i) balanced pretraining data can induce latent capabilities later activated during post-training, and (ii) SFT learns best from a small set of examples challenging for the pretrained model, while excessively large SFT datasets may dilute informative pretraining signals. In contrast, RL is most effective on large-scale data that is not overly difficult for the pretrained model. We validate these theoretical insights with experiments on large nonlinear transformer architectures.
@inproceedings{javanmard_theoretical_2026,
	address = {Seoul, South Korea},
	title = {Theoretical {Perspectives} on {Data} {Quality} and {Synergistic} {Effects} in {Pre}- and {Post}-{Training} {Reasoning} {Models}},
	url = {http://arxiv.org/abs/2603.01293},
	doi = {10.48550/arXiv.2603.01293},
	abstract = {Large Language Models (LLMs) are pretrained on massive datasets and later instruction-tuned via supervised fine-tuning (SFT) or reinforcement learning (RL). Best practices emphasize large, diverse pretraining data, whereas post-training operates differently: SFT relies on smaller, high-quality datasets, while RL benefits more from scale, with larger amounts of feedback often outweighing label quality. Yet it remains unclear why pretraining and RL require large datasets, why SFT excels on smaller ones, and what defines high-quality SFT data. In this work, we theoretically analyze transformers trained on an in-context weight prediction task for linear regression. Our analysis reveals several key findings: (i) balanced pretraining data can induce latent capabilities later activated during post-training, and (ii) SFT learns best from a small set of examples challenging for the pretrained model, while excessively large SFT datasets may dilute informative pretraining signals. In contrast, RL is most effective on large-scale data that is not overly difficult for the pretrained model. We validate these theoretical insights with experiments on large nonlinear transformer architectures.},
	language = {en},
	urldate = {2026-05-09},
	booktitle = {The 43rd {International} {Conference} on {Machine} {Learning} ({ICML} 2026)},
	author = {Javanmard, Adel and Mirzasoleiman, Baharan and Mirrokni, Vahab},
	month = jul,
	year = {2026},
	note = {arXiv:2603.01293 [cs]},
	keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Foundational, SYS: CosmicAI Contact Author, Statistics - Machine Learning, WG: Explorable},
}
Downloads: 0
About
Documentation
Keywords
Search
Users
Terms and Conditions of Use
Privacy Policy
Sitemap
© BibBase.org 2021