From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence

| Source: Microsoft Research Blog

Tags: Microsoft Research, Microsoft Copilot, synthetic data, LLM evaluation, O*NET

Microsoft Research built a platform that generates full synthetic enterprise environments, grounded in public data like O*NET, job postings and SEC filings, to test Copilot without touching real customer data.

Details

Microsoft cannot use real enterprise data to evaluate Copilot, so it built the Industry Evaluation Data Generation Platform. The platform creates synthetic organizations and business workflows grounded in public sources such as O*NET occupational data, online job postings and SEC filings, with industry-specific datasets. Automated content-safety checks keep generated data free of harmful content and personal information. Microsoft says the environments surface issues that generic benchmarks miss, such as whether a meeting summary is accurate and grounded in the transcript.