Meta's Llama 3.1 Achieves SOTA in MMLU-Pro, Surpasses Claude 3.5 Sonnet and Gemini 1.5 Pro

Recent developments in large language models (LLMs) have seen significant advancements from major players such as Meta, Google, Anthropic, and OpenAI. Meta's Llama 3.1 has shown notable improvements, with its 405B model achieving state-of-the-art (SOTA) status in MMLU-Pro, surpassing Anthropic's Claude 3.5 Sonnet. Additionally, Llama 3.1's performance in zero-shot MATH evaluations positions it between Claude 3.5 Sonnet and OpenAI's GPT-4o. Furthermore, Google's Gemini 1.5 Pro has outperformed its predecessor, 3.0 Ultra. These advancements suggest that architectural and data tweaks have led to substantial performance gains across the board.