Mixture of Experts News

released in the month of August 2025

At least 20B Parameters

At least 70B parameters

At least 30B parameters

Uses Mixture of Experts

Is a reasoning model

MIT or Apache license

An instruct model is made accessible

Uses Beyer Teacher/Student distillation ala Gemma 3

better than o3-mini on FrontierMath (with tools)

At least 120B Parameters

The model is accessible to all users on the main chatgpt site

Is at least a Vision-Language Model (ala PaliGemma)

Gets 1400+ Elo on lmarena.ai

At least 300B parameters

my friend Andromeda (who knows ab ai) thinks it unveils an impressive new architecture

A base model is made accessible

Is at least an Image-Language Model (ala JanusPro or 4o)

released in September, October, November, or December 2025

released in the month of June 2025

Very large context window, or another way to give the model long-term memory

Audio Modality

Operates autonomously with instructions from you

capable of online learning

Personalized to you, the user

Online learning of specialized agents(finetuning, PEFTs, etc.)

Video Modality

Exact-query of training data or knowledge databases(similar to Google RETRO)

(Near) real-time update of training data (data delay is within 6 months and is kept updated)

New Neural Network Architecture

Faster speed(smaller model, architecture improvements, mixture of experts keeping most parameters unused during inference, etc.)

Seed option to create the question

Acts autonomously

Designed to inference cheaper than GPT-4(price must be lower)

YES

NO

YES

NO