The Pulse
ElevenLabs previews enterprise agents and model fine-tuning
ElevenLabs previewed tools for enterprise agents to remember customers, automate work and select models during conversations. It also plans to help businesses fine-tune voice and open-source language models using their own data.
ElevenLabs is preparing tools that let enterprise voice agents remember returning customers and automate work around conversations, alongside a system that selects language models during an exchange. The company also plans to help businesses fine-tune voice and open-source language models using their own data.
At its New York summit on November 11, ElevenLabs outlined the updates to its ElevenAgents platform and previewed a separate workplace app. CEO and co-founder Mati Staniszewski predicted that conversational agents would become essential infrastructure for enterprises within a few years.
Models tailored to businesses
The company said its planned fine-tuning service will allow businesses to bring data from live deployments and work with ElevenLabs researchers to refine models for their own use. Businesses will be able to choose measures such as customer satisfaction or reliability to guide that work.
Fine-tuning for certain open-source language models and ElevenLabs voice models is planned for the coming months. The company also has a model optimized for financial services in development. ElevenLabs previously built Scribe v3 Medical, a transcription model for healthcare.
A separate system called Interaction Engine was previewed at the summit. It is designed to manage turn-taking and interruptions, and can also select a language model for each part of a conversation. A cheaper model could handle initial triage, while a more capable model takes on complex tasks, according to the company.
Agents that remember and follow up
The planned Memory feature will let an ElevenAgents agent recognize an authenticated returning customer and recall details from earlier conversations. Those details can include preferences, constraints and upcoming occasions, while the customer relationship management system remains the primary customer record.
After each interaction, the agent will turn the conversation into a structured record and update it when the customer returns. For example, the company said an agent could remember that a phone subscriber mentioned an upcoming move and later ask about transferring service.
Another feature, Routines, will let users run multistep work before, during and after conversations, connecting to existing tools and systems. The company gave examples including pre-screening a mortgage applicant, calling them back, emailing a report and writing information to a customer relationship management system.
A workplace app and partner network
The keynote included a preview of ElevenAI, an app for enterprise teams. The app is designed to listen to spoken conversations, including in-person meetings, and turn them into notes and next steps. Its dictation and meeting notes use Scribe v3, and it can provide context during a call.
ElevenLabs said notice and consent mechanisms are built into the app, and that meeting notes are taken with everyone’s consent. Free early access to ElevenAI opened on the day of the summit.
ElevenLabs launched a partner program for enterprises deploying ElevenAgents with consultancies, systems integrators, software vendors and agencies. The company said its partner network has grown to more than 550 partners in 44 countries. As part of the program, it announced an alliance with McKinsey and QuantumBlack, AI by McKinsey.
The company also introduced ElevenLabs Marketplace, where enterprises can browse industry-specific applications built by technology partners and contact those partners directly. Beginning next year, businesses will be able to buy agents through the marketplace using their existing ElevenLabs commitment.
ElevenLabs and NVIDIA also expanded their partnership. Nemotron 3.5 Super VL is available in ElevenAgents, and ElevenLabs is adding 15 MW of new Vera Rubin compute capacity. The companies plan to work on an on-premises voice agent for Poland’s national e-Health platform; the agent is intended to schedule appointments for patients who only have landlines.