ElevenLabs, a startup known for its AI-driven voice generation technology, has stepped out of its beta phase, unveiling a groundbreaking multilingual deep-learning model called Eleven Multilingual v2. This advanced model supports 30 languages, offering a powerful text-to-speech and voice-cloning solution that maintains the speaker’s unique voice attributes across all languages.

The startup’s platform, previously used for narrating audiobooks and enhancing content accessibility, has faced challenges related to misuse, as it has been exploited to create fabricated audio of public figures. The expansion aims to harness the platform’s capabilities for positive applications while simultaneously curbing misuse.

Eleven Multilingual v2 leverages deep learning to automatically detect the written language when users input text. This allows the platform to generate speech that boasts an “unprecedented level of authenticity,” according to the company’s announcement. Notably, the platform preserves the distinct voice characteristics of the speaker, whether synthetic or cloned, across all 30 supported languages. This ensures a consistent style of speech, including accent, regardless of the language spoken.

Mati Staniszewski, CEO and co-founder of ElevenLabs, expressed the startup’s vision: “ElevenLabs was started with the dream of making all content universally accessible in any language and in any voice.” He highlighted the significance of the Eleven Multilingual v2 release in bringing this dream closer to reality by offering human-quality AI voices in a multitude of dialects.

The potential applications for this technology are diverse. Indie authors can now easily create audiobooks, video game experiences can be translated for international audiences, and audio content can be adapted for individuals with visual impairments and specialized learning requirements.

The range of supported languages includes Korean, Dutch, Turkish, Swedish, Indonesian, Vietnamese, Filipino, Ukrainian, Greek, Czech, Finnish, Romanian, Danish, Bulgarian, Malay, Hungarian, Norwegian, Slovak, Croatian, Classic Arabic, Tamil, English, Polish, German, Spanish, French, Italian, Hindi, and Portuguese. This extensive linguistic coverage opens doors to a wide array of creative and practical possibilities for users worldwide.

By Impact Lab