The SmolTalk dataset, a 1 million sample synthetic dataset, has been released under Apache 2.0. This dataset was used to train SmolLM v2 and is noted for its ability to enhance instruction following, reasoning, rewriting, summarization, and function calling. The release has been praised for its well-curated nature and its significant contribution to the performance of SmolLM2 Instruct models, showing better results compared to training on the Orca AgentInstruct 1M dataset. Additionally, the RedPajama dataset, containing over 100 trillion tokens, has been released by @togethercompute to support open-source large language models (LLMs) like OLMo from @allen_ai. The new datasets include Smol-Magpie-Ultra.