The Pulse
Microsoft Details Synthetic Workspaces for Copilot Evaluation
Microsoft describes a platform that creates synthetic enterprise workspaces to evaluate Copilot with simulated users, emails, meetings and documents. The company says the system establishes a standardized baseline and reports improvements i
AI.info Team · Written with AI · fact-checked ·
Microsoft Research said on October 30, 2026, that it has built synthetic enterprise workspaces to test Microsoft Copilot using invented users, emails, meetings and documents rather than customer data. The system, called the Industry Evaluation Data Generation Platform, creates simulated organizations where Microsoft can check whether Copilot retrieves and summarizes information in realistic workplace scenarios. The company says its first standardized evaluation baseline for Copilot is now available for reuse across Microsoft teams.
Three synthetic tenants mimic distinct industries
Microsoft calls each simulated workspace a “tenant”: an isolated organization with its own users, information and settings. The current environments model a pharmaceutical company, a capital-markets business and an IT or software company. Their work includes clinical-trial discussions and safety reviews, deal pipelines and portfolio reviews, and software sprints, code reviews and product launches.
The environments are designed to include more than static documents. They contain organizational structures, communication histories and work artifacts, with project timelines, email threads and meeting records that develop over time. That temporal detail matters for questions about what a team decided last week or what changed since a quarterly review—tasks that require the test environment to preserve a sequence of events.
Public records shape the simulated organizations
Microsoft says it grounds the synthetic workplaces in public information, including O*NET occupational data from the U.S. Department of Labor, online job postings and SEC filings. O*NET helps shape roles, responsibilities and work activities; job postings inform titles and reporting hierarchies; and company filings help set organizational scale and business-unit structure. Microsoft says its legal and policy organization reviewed and approved the public sources.
The company’s pipeline also generates the questions used to test Copilot. It starts with common workplace tasks, filters them for a target industry, adds relevant terminology and context, then creates questions tied to the synthetic environment. The system rewrites queries to sound more natural and checks them for accuracy and duplication, according to Microsoft.
Microsoft reports quality gains and lower evaluation costs
Microsoft says its internal composite measure of how closely synthetic data resembles enterprise environments improved 39% from the initial generation approach. Separately, it reports that some industry-specific environments reached quality scores above 0.9 out of 1.0; the post does not say those scores use the same composite measure that improved 39%. Microsoft says each environment now contains hundreds of users with communication histories across the three industries.
Microsoft also reports that dataset creation has shifted from a process taking several days to automated runs. It estimates savings of about $100,000 per evaluation dataset and $300,000 across the project.
“The difference wasn’t just incremental. It was a shift in how teams approach evaluation: from ad hoc, manual processes to standardized, reproducible infrastructure.”
Microsoft Research blog
Customer data stays out of the test pipeline
Microsoft says no real customer data enters the generation process: the organizational charts, email content and meeting transcripts are synthetically produced. The company also describes content-safety screening, provenance tracking and privacy-preserving aggregation of public statistical distributions as safeguards. It says teams can reuse the datasets for experimentation, benchmarking and regression testing without customer-data handling restrictions.
The current platform sets a baseline for testing single-turn questions and other enterprise tasks, but Microsoft identifies harder evaluations ahead. The post says the team is working on ways to assess multi-turn agent interactions, model organizations that change over months or years, and test systems that take actions rather than only generate text.