OpenAI has introduced three new artificial intelligence models designed to enhance speech-to-text and text-to-speech capabilities.
These models—GPT-4o-transcribe, GPT-4o-mini-transcribe, and GPT-4o-mini-tts—are integrated into OpenAI’s API and are aimed at improving accuracy and reliability for real-world applications.
While they are expected to outperform OpenAI’s previous Whisper models, they are not open-source.
Advancing AI-Powered Voice Interaction
The San Francisco-based AI company states that these models align with its broader goal of enabling more intuitive, multimodal AI interactions.
OpenAI has previously developed AI agents such as Operator, Deep Research, and the Responses API, which focus on automating tasks and streamlining workflows. The latest update expands on this approach, allowing AI to process and generate speech with greater precision.
One key improvement in the new speech-to-text models is their enhanced performance on the Few-shot Learning Evaluation of Universal Representations of Speech (FLEURS) benchmark, which assesses AI models across 100 languages.
OpenAI reports that its latest models demonstrate lower word error rates (WER), particularly in challenging conditions such as background noise, diverse accents, and varying speech speeds.
Performance Comparison Dashboard
These advancements are attributed to refined training techniques, including reinforcement learning and expanded datasets.
Natural-Sounding Text-to-Speech Capabilities
The GPT-4o-mini-tts model introduces enhancements in text-to-speech synthesis, offering more natural-sounding voices.
According to OpenAI, the model supports customizable inflections, intonations, and emotional expressiveness, making it suitable for applications like virtual assistants, customer service solutions, and interactive storytelling.
The model currently offers only preset synthetic voices, without the ability for users to generate custom voices.
Pricing Structure for Developers
OpenAI has established different pricing tiers for its new models:
- GPT-4o-based audio models:
- $40 per million input tokens
- $80 per million output tokens
- GPT-4o-mini models:
- $10 per million input tokens
- $20 per million output tokens
This structured pricing model provides developers with scalable options for integrating AI-driven speech capabilities into their applications.
Technical Innovations Enhancing Performance
The new GPT-4o and GPT-4o-mini audio models incorporate several advancements:
- Pretraining with High-Quality Audio Datasets – Extensive training on diverse speech datasets has improved the models’ ability to handle various audio-related tasks with higher accuracy.
- Advanced Distillation Techniques – Knowledge transfer from larger models to smaller, more efficient ones enhances performance while preserving conversational realism.
- Reinforcement Learning Integration – The use of reinforcement learning further refines transcription accuracy, making these models competitive in complex speech recognition scenarios.
Availability
The new models are now accessible via OpenAI’s API, with additional support available through the Agents SDK, which helps developers build voice-based AI applications.
OpenAI plans to continue refining its speech-processing technology, exploring ways to enable more personalized experiences through custom voice capabilities while maintaining ethical AI practices.
OpenAI is collaborating with policymakers, researchers, developers, and content creators to address the challenges associated with synthetic voice technologies, reinforcing its commitment to responsible AI development.
