Microsoft has unveiled its new AI model, GRIN-MoE (GRadient-INformed Mixture-of-Experts), which has demonstrated superior performance in coding and math tasks. The model, which was pre-trained on 4 trillion tokens, has set key benchmarks, outperforming competitors in scalability and accuracy. GRIN-MoE allows smaller models to achieve high accuracy by leveraging a novel approach to training large Mixture-of-Experts language models.