OpenAI has launched its Realtime API, which is now in public preview on Azure OpenAI Service. This new API enables real-time, human-like conversations and multilingual mastery for applications. The Realtime API was announced during OpenAI's Dev Day and is expected to be a game-changer for developers looking to build lifelike voice-to-voice experiences. The API supports function calling and will soon be multimodal, handling both text and audio inputs and outputs. Microsoft has also announced new products and features for the Azure OpenAI Service, including the GPT-4o-Realtime-Preview with advanced audio and speech capabilities. The GPT-4o Audio Realtime API promises a 300ms response time.
For the past few months the @livekit team has been working with OpenAI to give developers access to the same technology that powers Advanced Voice in ChatGPT. With the new Realtime API, you can build voice AI that understands the nuances of human speech and responds in 300ms…
OpenAI’s Realtime API is here! We created 4 new resources to help you start building with the same stack OpenAI uses for Advanced Voice in ChatGPT. Bookmark this thread for when you get Realtime API access: https://t.co/0Ax85Q2Gpj
Update: OpenAI Realtime API is not ready yet https://t.co/fWN5EeROAp https://t.co/DZ4gziZW6i