Voice AI is moving into direct patient contact faster than most organizations have governance for. Virtual agents are triaging symptoms, managing post-discharge follow-up, and coordinating care across specialties, often without a clinician reviewing the interaction before it happens. That’s a different risk profile than a flawed AI suggestion sitting in an EHR waiting for a human to catch it. An error in an autonomous voice interaction can reach a patient before anyone has the chance to review it.
That’s the gap the Healthcare Risk Management for Voice AI Taskforce was convened to discuss.
Who’s in the room
The taskforce is chaired by Anil Saldanha (Rush University System for Health), Chethan Sarabu, MD (Cornell Tech Health Tech Hub), and Melanie Fernando, MBA (Aetna Better Health of Illinois). Members span health systems, academic medicine, health plans, industry, and patient advocacy, including Andrea Downing of The Light Collective, Sudha Jayaraman, MD (Suki AI), and clinicians from Stanford, Rush, and UT Medical Branch, among others. I represent Infinitus on the taskforce.
Patient advocacy sits at the table from the start here, not as a stakeholder consulted after the framework is built. A clinical safety framework for patient-facing AI is only as good as the perspective of the patients it’s meant to protect.
The clinical safety case
Security and data privacy are non-negotiable for any healthcare AI system, but they’re not unique to voice AI. HIPAA compliance, encryption, and access controls apply the same way they would to any system touching PHI. What’s new, and what most existing governance frameworks don’t yet address, is the clinical risk that shows up specifically when an AI agent talks to a patient directly and acts on what it hears.
What the taskforce focuses on
The taskforce doesn’t build or enforce oversight mechanisms itself. Its role is to convene clinical, technical, and patient perspectives, confer on where the real risks are, and publish insights that organizations already building and deploying voice AI can use. Think of it as a shared forum for hard-won lessons, not a standards body.
Within that, the group’s discussions center on a small set of clinical risk domains: clinical safety itself, model bias, and automation bias, the tendency for a clinician to over-trust an AI output simply because it’s there. Cybersecurity and data privacy come up too, as baseline requirements for any system that touches PHI, but they’re not the differentiating risk the taskforce spends most of its time on.
Those conversations follow a voice agent across its full lifecycle: what validation should look like before deployment, what ongoing monitoring catches that a one-time review would miss, how incidents get reported, and what continuous auditing could look like in practice. Equity is a recurring thread throughout: voice AI shouldn’t exacerbate disparities in care delivery or outcomes, and the taskforce treats that as worth examining explicitly rather than assuming it away.
The scope of that conversation covers any voice AI tool that interacts with members, patients, clinicians, or staff across the payer-provider continuum: virtual agents and chatbots, call center and triage systems, voice-enabled documentation tools, and generative AI systems handling PHI. You can read our first paper here.
Why Infinitus is at the table
Infinitus builds voice AI agents that operate directly in healthcare workflows, so the risks this taskforce discusses aren’t abstract to us. We see where transcription errors, missed escalations, or gaps in security best practices can enter a real deployment. That gives us something concrete to bring to the conversation, and a direct stake in getting the answers right.
Participating in a multidisciplinary group alongside patient advocates, academic clinicians, and payors keeps us honest about tradeoffs. We’d rather help build the shared understanding of what safe looks like than wait for someone else to define it for us.
How the work is organized
Four subgroups organize that conversation: Technology focuses on the baseline security and privacy layer, PHI encryption, secure deployment, identity management, audit logging, and threat detection, aligned with HIPAA, NIST, and HHS guidance. Operations looks at rollout, training, performance monitoring, and incident response as these systems integrate into real payer and provider workflows. Research supports the taskforce’s broader publication and dissemination efforts. The group focused on clinical safety specifically is Clinical Use Cases.
Clinical Use Cases explores what’s acceptable to automate, discusses escalation protocols, and emphasizes the importance of human oversight across triage, care coordination, documentation support, and administrative automation. It’s the subgroup closest to the actual patient interaction, and the one asking the hardest questions about what a clinically safe voice AI deployment should look like.
Voice AI is already in production, touching patients who are sick, scared, and not always able to self-advocate. Organizations that treat clinical safety as a primary design constraint, rather than an afterthought bolted onto a secure system, will be better positioned to scale this technology with confidence, and that’s the thinking this taskforce exists to share.


