Skip to content
AI.info

The Pulse

Microsoft Details Synthetic Workspaces for Copilot Evaluation

Microsoft built a platform that creates synthetic enterprise workspaces to evaluate Copilot with simulated users, emails, meetings and documents. The system establishes a standardized baseline and improves data quality and evaluation efficiency, the company reports.

Microsoft Details Synthetic Workspaces for Copilot Evaluation

Microsoft Research said on October 30, 2026, that it has built synthetic enterprise workspaces to test Microsoft Copilot using invented users, emails, meetings and documents rather than customer data. The system, called the Industry Evaluation Data Generation Platform, creates simulated organizations where Microsoft can check whether Copilot retrieves and summarizes information in realistic workplace scenarios. The company’s first standardized evaluation baseline for Copilot is now available for reuse across Microsoft teams.

Three synthetic tenants mimic distinct industries

Microsoft calls each simulated workspace a “tenant”: an isolated organization with its own users, information and settings. The current environments model a pharmaceutical company, a capital-markets business and an IT or software company. Their work includes clinical-trial discussions and safety reviews, deal pipelines and portfolio reviews, and software sprints, code reviews and product launches.

The environments are designed to include more than static documents. They contain organizational structures, communication histories and work artifacts, with project timelines, email threads and meeting records that develop over time. That temporal detail matters for questions about what a team decided last week or what changed since a quarterly review—tasks that require the test environment to preserve a sequence of events.

Public records shape the simulated organizations

The synthetic workplaces are grounded in public information, including O*NET occupational data from the U.S. Department of Labor, online job postings and SEC filings. O*NET helps shape roles, responsibilities and work activities; job postings inform titles and reporting hierarchies; and company filings help set organizational scale and business-unit structure. Microsoft’s legal and policy organization reviewed and approved the public sources.

The company’s pipeline also generates the questions used to test Copilot. It starts with common workplace tasks, filters them for a target industry, adds relevant terminology and context, then creates questions tied to the synthetic environment. The system rewrites queries to sound more natural and checks them for accuracy and duplication, according to Microsoft.

Quality gains and lower evaluation costs, by Microsoft’s measure

Microsoft’s internal composite measure of how closely synthetic data resembles enterprise environments improved 39% from the initial generation approach, the company says. Separately, some industry-specific environments reached quality scores above 0.9 out of 1.0, it reports; the post does not say those scores use the same composite measure that improved 39%. Each environment now contains hundreds of users with communication histories across the three industries.

Dataset creation has shifted from a process taking several days to automated runs, Microsoft also reports, and the company estimates savings of about $100,000 per evaluation dataset and $300,000 across the project.

“The difference wasn’t just incremental. It was a shift in how teams approach evaluation: from ad hoc, manual processes to standardized, reproducible infrastructure.”

Microsoft Research blog

Customer data stays out of the test pipeline

No real customer data enters the generation process, Microsoft says: the organizational charts, email content and meeting transcripts are synthetically produced. The company also describes content-safety screening, provenance tracking and privacy-preserving aggregation of public statistical distributions as safeguards. Teams can reuse the datasets for experimentation, benchmarking and regression testing without customer-data handling restrictions, according to Microsoft.

The current platform sets a baseline for testing single-turn questions and other enterprise tasks, but Microsoft identifies harder evaluations ahead. The team is working on ways to assess multi-turn agent interactions, model organizations that change over months or years, and test systems that take actions rather than only generate text, according to the post.

Sources

  1. Original source · Blog post Microsoft Research microsoft.com

Companies in this story

Related stories

Explore

More articles