Developers monitoring open-source platforms say two unreleased OpenAI language-model checkpoints—one sized at roughly 20 billion parameters and a larger 120 billion-parameter version—were uploaded to the code-sharing site Hugging Face in the early hours of 1 August before being removed within minutes. Metadata and configuration files examined by multiple researchers indicate the systems use a Mixture-of-Experts design with 128 specialists, activate four experts per token, and support data types including BF16, FP4 and UE8. The leaked configuration cites advanced features such as Grouped-Query Attention, Sliding-Window Attention for long contexts, SwiGLU activations and NTK RoPE positional encoding—placing the models in direct competition with Meta’s Llama family and Mistral’s Mixtral series. If confirmed, the release would mark the first time OpenAI has offered fully open weights since its early research days and could broaden adoption by allowing outside developers to fine-tune and self-host the system. The appearance of the files has intensified speculation that OpenAI is preparing a wider update to its commercial lineup, including the long-rumoured GPT-5. Some users report limited API access to a model labelled “gpt-5-bench-chatcompletions,” though the company has not acknowledged its existence. In a podcast recorded 30 July, Chief Executive Officer Sam Altman said testing GPT-5 felt akin to the "Manhattan Project," while a separate report from The Information cautioned that the upgrade may deliver more incremental gains than the jump from GPT-3 to GPT-4. OpenAI has not responded to requests for comment on the brief posting or on the timeline for any formal launches. Several independent analysts now expect the company to publicly unveil its open-weight models—and possibly GPT-5—within the coming weeks.
GPT-4 to GPT-5 > GPT-3 to GPT-4
GPT-5 is coming and it'll be good. But, OpenAI and others have been facing slowing gains, which they've been able to find workarounds to with reinforcement learning, "universal verifiers" and other techniques. More here w/ @erinkwoo @amir: https://t.co/KGDdsH9ufu
Interesting. @OpenAI may be releasing their open source models as early as next week. https://t.co/AtcOSoWGSK