Human Review of Microsoft Copilot Image Uploads Exposes Users to Unchecked Abuse
What Happened – Contractors hired by Microsoft‑linked third‑party firms are manually reviewing uploaded photos and edit requests from Copilot users. Reviewers see the original faces, evaluate two AI‑generated edits, and rate technical quality — but they are not instructed to flag illegal or non‑consensual content. The process includes sexualized requests involving minors and other disturbing material. Microsoft’s public FAQ does not disclose an opt‑out for this human‑review pipeline.
Why It Matters for Trust & Control Assurance
- Highlights a gap in third‑party data‑processing oversight: contractors can view personally identifiable images without documented consent or monitoring.
- Undermines privacy‑governance controls that require documented user opt‑out mechanisms and audit‑ready evidence of lawful processing.
- Demonstrates the need for continuous control‑assurance evidence (e.g., logs of data flow to reviewers, policy enforcement) to satisfy regulators and auditors.
Who Is Affected – SaaS AI platform providers, cloud‑based productivity tools, and their enterprise customers (technology / software‑as‑a‑service sector).
Recommended Actions
- Map the data‑handling workflow to the privacy control objective of “third‑party processing oversight” and capture evidence of consent, opt‑out, and reviewer access logs.
- Conduct a privacy impact assessment (PIA) for any human‑in‑the‑loop review, and update policies to require contractors to flag illegal content.
Technical Notes – The exposure stems from a third‑party dependency (human reviewers) rather than a software vulnerability. No CVE is involved. Microsoft states that uploaded files may be used for AI training unless the user opts out; however, an opt‑out for human review is not offered. Source: Malwarebytes Labs