Clinicians are pasting patient summaries into ChatGPT to draft discharge instructions. Billing staff are uploading claim data to AI writing tools to speed up appeals letters. Nurses are using consumer AI assistants to look up drug interactions between patient visits. None of this was approved by the security team, and most of it would surprise the compliance officer.
Protected health information (PHI) is moving into AI systems that healthcare organizations do not control, at a pace that outstrips most security programs. The legal and reputational exposure is real. So is the pressure not to block tools that genuinely help clinical staff do their jobs. The question is not whether to allow GenAI in healthcare settings. It is how to allow it without losing control of patient data.
What Is PHI Exposure Risk in GenAI Tools?
PHI exposure risk in GenAI tools refers to the likelihood that protected health information will be transmitted to, stored by, or used to train an external AI system without the data owner's authorization or a valid business associate agreement (BAA) in place.
When a healthcare employee pastes patient data into a consumer AI application, that data typically leaves the organization's control. Depending on the tool's terms of service, the data may be retained, used for model training, or accessible to vendor staff. Most consumer AI tools are not HIPAA-compliant and are not covered by a BAA. That means every interaction involving PHI is a potential HIPAA breach, regardless of whether the employee intended to cause harm.
The risk is compounded by the nature of AI prompts. Unlike a file upload, which a data loss prevention (DLP) tool can easily detect, PHI entered in natural language is harder to classify and intercept. A prompt that reads "My 67-year-old patient with Stage 2 kidney disease is on metformin..." contains PHI in a form that most legacy DLP tools were not built to recognize.
How to Protect PHI When Employees Use GenAI Tools
Protecting PHI in a GenAI environment requires a layered approach, including:
- Data-level visibility to surface where PHI is going
- Access and policy controls to govern which tools can receive it
- An employee strategy that offers approved alternatives to the tools staff are already using.
Each layer addresses a different failure point.
Establish data-level visibility before enforcing policy
Security teams cannot enforce policies on data movement they cannot see. Network monitoring and endpoint-only controls can identify traffic to known AI domains, but they miss AI features embedded in tools employees already use: AI writing assistants in Microsoft 365, AI summarization in EHR platforms, and AI copilots integrated into clinical communication tools. These are approved tools with AI capabilities that were not in scope when the tools were originally evaluated. This is compounded by the rise of endpoint AI application adoption, which grew 509% in a single year.
Effective PHI protection starts with tracking where data goes after it leaves its source system, not just which applications employees access. Data Lineage maps the movement of sensitive data from the moment it is created or accessed, surfacing PHI movement to external AI services regardless of which application initiated the transfer. That visibility is the prerequisite for every enforcement decision that follows.
Enforce access controls at the data level, not just the tool level
Most healthcare organizations approach GenAI governance by building an approved tool list and blocking everything else. This is necessary but not sufficient. Approved tools can still receive PHI in ways that violate HIPAA's minimum necessary standard, and employees who need a capability not on the approved list will find a workaround.
Data-level policy enforcement allows security teams to apply fine-grained controls that go beyond allow/block.
For example: permitting Microsoft Copilot for clinical documentation while blocking bulk patient record uploads to the same tool. Or flagging prompts that contain structured PHI patterns while allowing general clinical queries.
These distinctions require the security control to understand data context, including where the data originated and what it has been combined with, not just what the data looks like at the point of egress.
Require BAAs before any AI tool handles PHI
Any vendor that handles PHI on behalf of a covered entity must sign a BAA under HIPAA. This requirement does not change because the vendor is an AI platform. Consumer AI tools and many enterprise-tier AI products do not offer HIPAA-compliant BAAs, which means every interaction involving PHI is a potential reportable breach.
Healthcare security and legal teams need a formal AI tool evaluation process that gates BAA availability before approval. Tools without a BAA should be blocked from receiving PHI at the data layer, not just removed from the approved tool list. The gap between "removed from the list" and "technically prevented from receiving data" is where most unauthorized PHI disclosures occur.
Reduce demand for unauthorized tools by providing approved alternatives
A security strategy built entirely around blocking will generate shadow AI usage rather than eliminate it. Employees turn to consumer AI tools because those tools solve real problems faster than approved workflows. According to Cyberhaven Labs, one-third of employees access AI tools via personal accounts, including 58% of Claude users and 60% of Perplexity users.
The friction in healthcare documentation is significant, and AI tools offer a visible productivity gain. When clinical staff weigh policy compliance against time savings, the outcome is often predictable.
Durable PHI protection requires pairing restrictions with approved alternatives. When a HIPAA-compliant AI documentation tool is available, the case for using an unapproved one weakens. When the acceptable use policy explains what is permitted alongside what is not, adoption tends to be higher. Security programs that treat GenAI as a productivity challenge, not just a compliance problem, are better positioned to close the gap between what employees use and what security teams can see.
HIPAA's Requirements for AI Tool Governance
The Health Insurance Portability and Accountability Act (HIPAA) does not have AI-specific provisions, but its existing rules apply directly to GenAI tool use. Healthcare security teams need to address four requirements in particular.
- Business associate agreements: Any vendor handling PHI on behalf of a covered entity must sign a BAA. Security and legal teams need a process to evaluate AI tools against BAA availability before approving them for clinical or administrative use.
- Minimum necessary standard: HIPAA requires that staff only use and disclose the minimum PHI necessary to accomplish a task. Pasting an entire patient record into an AI tool to answer one question likely violates this standard, even if the tool is approved.
- Access controls and audit trails: HIPAA's Security Rule requires covered entities to implement technical controls that limit PHI access and generate audit logs. PHI movement to external AI tools must be visible and logged to satisfy this requirement.
- Breach notification obligations: If a GenAI tool receives PHI without a valid BAA and the disclosure was not authorized, the organization may have a reportable breach. The breach notification rule does not require proof of harm, only proof that PHI was disclosed in violation of the Privacy Rule.
How Cyberhaven Protects PHI in Healthcare AI Environments
Cyberhaven's AI Security capability addresses each layer of the PHI protection problem: visibility into where data moves, policy enforcement at the data level, and audit documentation for HIPAA compliance.
Cyberhaven tracks PHI movement from the moment data is accessed inside the environment. When that data moves toward an external AI application or agent, Cyberhaven identifies the movement in real time, classifies the data type, and applies a policy response based on the destination, the user's role, and the data's sensitivity level.
This is different from conventional DLP, which inspects content at a specific egress point and classifies it based on pattern matching. Linea AI understands context: where data came from, what it has been combined with, and whether the destination is an approved tool with a valid BAA. That context is what allows healthcare security teams to enforce meaningful policies rather than keyword blocklists that produce high false positive rates and frustrated clinical staff.
Key capabilities for healthcare environments include:
- PHI detection in AI prompts, including natural language inputs that do not match structured data patterns
- Policy enforcement by AI tool, allowing fine-grained rules such as approving Microsoft Copilot with PHI restrictions while blocking consumer AI tools entirely
- Audit-ready logging that maps data movement to specific users, tools, and timestamps for HIPAA compliance documentation
- Risk-based alerting that distinguishes between a clinician querying an approved tool and a billing employee uploading bulk patient records to an unapproved service
Protecting PHI in an environment where employees increasingly rely on AI tools is a solvable problem, but not with the controls most healthcare organizations had in place two years ago. Data-level visibility, policy enforcement tied to data context, and a governance model that pairs restrictions with approved alternatives are the three components that close the gap. Cyberhaven's AI Security capability gives healthcare security teams the tools to enforce those policies without turning the security program into an obstacle to clinical productivity.
Better understand the security risks of AI, and how to build an AI security and governance program, with "Governing the Autonomous Enterprise: A Security Framework for Agentic AI."
Frequently Asked Questions
Does HIPAA prohibit healthcare employees from using AI tools?
HIPAA does not prohibit AI tool use. It requires that any vendor handling PHI sign a business associate agreement and that PHI disclosures are authorized and limited to the minimum necessary. Healthcare organizations can approve AI tools that meet these requirements. The problem is that many popular consumer AI tools do not, and employees often use them anyway.
What counts as PHI in an AI prompt?
Any information that could identify a patient and relates to their health condition, treatment, or payment for healthcare is PHI under HIPAA. This includes names, dates of birth, diagnoses, treatment details, medication lists, and geographic data smaller than a state. PHI in an AI prompt does not need to be in a structured format to be covered by HIPAA.
How can a healthcare security team detect when employees send PHI to AI tools?
Effective detection requires visibility at the data level rather than just the network level. Tools that track where data moves after it leaves its source system, such as Cyberhaven's Data Lineage capability, can surface PHI movement to AI tools regardless of the application or channel used. Network monitoring alone misses AI features embedded in approved tools.
What should a healthcare organization do if PHI is sent to a non-HIPAA-compliant AI tool?
The organization should assess whether the disclosure qualifies as a reportable breach under HIPAA's breach notification rule. Factors include whether the AI tool has a BAA in place, whether the disclosure was authorized, and whether there is low probability that PHI was compromised. Consulting legal counsel before determining breach status is advisable. Security teams should also document the incident and use it to update AI tool policies and training.
Is a business associate agreement enough to make an AI tool HIPAA-compliant?
A BAA is necessary but not sufficient. The AI tool must also have appropriate technical safeguards, access controls, and audit logging in place. Security teams should evaluate AI vendors against HIPAA's Security Rule requirements, not just BAA availability. Some AI vendors offer HIPAA-eligible versions of their tools with additional configuration required.
How should healthcare organizations build an AI acceptable use policy that clinical staff will actually follow?
Policies that only restrict tend to drive shadow AI usage rather than reducing it. Effective AI acceptable use policies in healthcare pair restrictions with approved alternatives. When staff cannot use consumer AI tools for documentation, providing a HIPAA-compliant alternative removes the gap. Policies developed with clinical input tend to achieve higher adoption than those written without it.
.avif)
.avif)
