Sources
Loading...
Additional media
Loading...

Recent research from Google DeepMind suggests that training language models on synthetic data generated by smaller, weaker models can outperform those trained on data from larger, stronger models. This finding challenges the current practices in the field of language model training, which often rely on high-quality synthetic data from strong models. The study highlights a potentially compute-optimal approach for enhancing the reasoning capabilities of language models, which could lead to more efficient use of computational resources.

