Apr 9, 06:29 PM

Kimi Moonshot Launches Kimi-VL Model with 3 Billion Parameters and 371 Billion Token MegaMath Corpus, Outperforming GPT-4o

Kimi Moonshot has announced the release of Kimi-VL, a lightweight yet powerful Vision-Language Model (VLM) that features strong multimodal reasoning capabilities. The model, which includes both a mixture of experts (MoE) VLM and an MoE Reasoning VLM, operates with approximately 3 billion activated parameters. It reportedly outperforms larger models, including GPT-4o, on various vision and math benchmarks. Additionally, the company introduced MegaMath, an open-source math reasoning pre-training corpus comprising 371 billion tokens of high-quality web, code, and synthetic data. This new corpus surpasses the previous DeepSeek-Math in scale and enhances math reasoning performance by up to 20%. Both Kimi-VL and MegaMath are available under an MIT license.

#Kimi Moonshot #MoE Reasoning #GPT #MegaMath #MIT

Written with ChatGPT (GPT-4o mini).

Sources

Additional media

Image #1 for story kimi-moonshot-launches-kimi-vl-model-3-billion-parameters-371-billion-token-gpt-61dd3323

Image #2 for story kimi-moonshot-launches-kimi-vl-model-3-billion-parameters-371-billion-token-gpt-61dd3323

Image #3 for story kimi-moonshot-launches-kimi-vl-model-3-billion-parameters-371-billion-token-gpt-61dd3323

Image #4 for story kimi-moonshot-launches-kimi-vl-model-3-billion-parameters-371-billion-token-gpt-61dd3323

Image #5 for story kimi-moonshot-launches-kimi-vl-model-3-billion-parameters-371-billion-token-gpt-61dd3323