Hugging Face has released the SmolLM2 paper, detailing the development of a state-of-the-art small language model (LM) that boasts 1.7 billion parameters and was trained on 11 trillion tokens. The new model demonstrates superior performance compared to other recent small LMs, including Qwen2.5 (1.5 billion parameters) and Llama3.2 (1 billion parameters). The paper emphasizes the importance of data in training small models, highlighting how the right data mixtures and training strategies can enhance model capabilities. Key insights include the creation of custom datasets and adaptive training methods that contribute to SmolLM2's effectiveness.