Losing Your Voice to AI

Losing your voice to AI: Privacy risks of health-related machine listening

Funded by the Office of the Privacy Commissioner of Canada as part of their 2025 funding call for research projects studying personal information collected by smart devices. 

Project Description:

Smart speakers are the most popular smart home device in Canada. Millions of Canadians use the voice assistant Alexa or Siri on a daily basis. In addition to recording conversations, these machine listening platforms can automatically produce biometric voice profiles of adult end-users, children, and others nearby.

There is good reason to qualify voice data as sensitive health information as Big Tech corporations are planning to use voice data for health-related applications. Adding to this, start-ups have launched health-related machine listening applications and platforms that monitor depression, anxiety, PTSD, and respiratory conditions such as COVID-19, sleep apnea, and childhood asthma.

As voice data becomes increasingly processed by generative-AI systems, there is real risk of significant harm that could lead to unfair, unethical, or discriminatory treatment that contravenes human rights law – representing what the Office of the Privacy Commissioner of Canada labels as “no-go zones”. For example, health-related voice data could be processed by insurance companies to adjust premiums, lower reimbursements, or even refuse coverage. Additionally, this data could be used by employers to deny work opportunities or evaluate workplace disability claims.

The overarching objective of this project is to map the array of applications for health-related machine listening so as to anticipate privacy risks and protect the privacy rights of Canadians. Drawing from empirical findings, the project will lead a public education and awareness campaign that targets four segments of the Canadian public. First, a short documentary film will be used to help educate Canadian consumers about the privacy risks of health-related voice data. Second, a digital literacy primer will be developed to deliver to community stakeholders in support of ongoing advocacy efforts. Third, a research report will be developed to help support policy and decision makers in their work of protecting Canadian privacy rights, focusing on the regulatory scope of PIPEDA which federally governs the collection and use of health-related voice data in non-medical settings. And finally, a short primer will be developed for organizations developing or using health-related machine listening applications impacting Canadians as a resource to ensure PIPEDA compliance.

Research Team

Stephen Neville, Alex Borkowski & Greg Elmer

Collaborators include Dr. Natalie Coulter, Institute for Digital Literacies (York University), Media Literacy Week, the Centre for Free Expression (TMU), and the Bell Media Research Chair, TMU.

Events: 

Media Literacy Week online workshop: Privacy & Machine listening apps.

Ever wonder if Alexa or Siri hear more than they should? Join us for an online interactive workshop where we will talk about educating Canadians, our workplace, colleagues and families about how apps and devices turn voices into data and what that means for our privacy.

Co-hosted by The Institute for Research on Digital Literacies, York University and the Infoscape Research Lab, Toronto Metropolitan University as part of the Media Literacy Week programme from MediaSmarts. Event program link: 

Date: October 29, 2025, 12pm EST

This is a public event designed for students, educators, researchers and policy advocates concerned with the datafication of sound and voices. 

Resources:

• Children and voice data privacy

• Voice profiling

• Voice authentication and bias

• Security tips for voice assistants

Machine listening Primer - TBA

________________________________________________________