On September 23, Alibaba Qwen completely revamped its own audio stack. The Qwen-Audio-3.1 series released five models at once, covering "understanding—generation—interaction—creation" in audio capabilities, from speech recognition and synthesis to real-time interaction and audio creation and understanding. It also significantly lowered the prices across the board: TTS dropped by about 70%, Realtime by around 85%, and ASR by as much as 95%. This set of moves clearly indicates that voice capabilities are no longer hidden away but are now accessible at a very low cost.
For speech recognition, Qwen-Audio-3.1-ASR is a new generation of large model for speech recognition. Its most notable feature is its ability to "understand context." The newly added native transcription polishing can automatically remove filler words, delete repetitive expressions, and restructure semantics, making the output text smoother and more logical rather than just raw transcriptions. Role-based transcription fully restores who said what and when, providing structured and traceable records for meetings, interviews, and dialog analysis. This end-to-end solution jointly outputs speaker labels, timestamps, and text, handling turn-taking, short interruptions, and overlapping speech, clearly separating each person's speech.
The coverage of this audio system is impressive: it supports 30 languages and 16 Chinese dialects, recognizing industry terms, professional entities, and graded hot words, while referencing long audio historical context to maintain consistent names and terminology. The first-word response delay is as low as about 160 milliseconds, allowing seamless real-time transcription. Performance is equally strong: on the public dialect dataset KeSpeech and WSYue's 11 subsets, the average word error rate is 4.55%, and the average word error rate for 16 Chinese dialects in internal testing is 10.38%. The performance for difficult dialects like Wenzhou dialect has improved significantly, with an average semantic sentence accuracy rate of 82.10% for converting dialects to Mandarin.
A more advanced step comes from Qwen-Audio-3.1-ASR-Next, based on the next-generation architecture. It expands the recognition scope from "text in speech" to "complete audio information"—it can understand emotions, environmental sounds, and mechanical sounds, perform sound descriptions, event localization, and audio question answering. Given an audio clip mixed with dialogue, background music, and ambient sounds, it doesn't just output subtitles, but also tells you what sounds are present, when related events occur, and answers questions about the entire content. In the role-based ASR test, ASR-Flash-Next and ASR-Flash achieved five and three top scores respectively out of eight metrics, thoroughly surpassing the previous Fun-ASR level system.
For speech synthesis, Qwen-Audio-3.1-TTS emphasizes realistic expression rather than just accurate pronunciation. It supports multilingual and dialectal synthesis, enabling natural voice migration across languages, allowing users to speak multiple languages instantly. It also allows control over emotion, speech rate, and expression through instructions, making the voice fit specific content and scenarios.
The real surprise is Qwen-Audio-3.1-TTS-Next, based on the next-generation architecture. It elevates audio creation from "single human voice synthesis" to a "general audio generation system." Previously, creating a complete audio required different professionals such as hosts, sound engineers, mixers, and editors to work separately and then integrate them later. Now, one model can generate human voices, sound effects, and ambient sounds based on text, timestamps, and reference audio. It supports multiple voice replication and multi-turn dialogues, keeping each character's voice consistent in continuous content and performing pauses, follow-ups, agreement, and emotional shifts. It can merge various types of sounds, presenting material, distance, and spatial layers—allowing a podcast to handle narration, two-character dialogues, and add city hums, evening winds, and can-crushing sounds, making the dialogue feel like it's happening in a real story scene. More professionally, it accepts natural language control, allowing specification of tone, speed, volume, music style, and instrumentation, supporting fine-grained timestamps, sound replication, and 48kHz output. It can precisely arrange the start and end of dialogues and the rhythm of sound events, satisfying the needs of podcasts, audiobooks, films, games, and advertisements.
For real-time interaction, Qwen-Audio-3.1-Realtime no longer simply connects recognition, models, and synthesis in a chain. It operates in full-duplex mode, allowing simultaneous speaking and listening, enabling users to interrupt or interject at any time, resulting in an experience close to real human conversation. The model no longer understands only text but also the emotions and intentions behind the sounds—when it senses your depressed tone, it won't mechanically reply with "I understand," but instead slow down its speech rate and adjust its wording. It can switch languages without losing context, tone, or rhythm. It avoids fabricating uncertain information and respects boundaries for high-risk content. It also uses a multi-teacher distillation architecture to improve reasoning and safety while maintaining low latency, and can directly call Agent tools during real-time conversations, accessing APIs, knowledge bases, and business systems to retrieve and execute results, naturally integrating them back into the conversation.
