HIPAA applies to AI healthcare applications whenever they create, receive, store or transmit protected health information (PHI). That means an AI app must meet the same HIPAA Security Rule safeguards as any other system that touches PHI: encryption, access control, audit logging and secure hosting, plus a business associate agreement (BAA) with any vendor or AI service that processes PHI on your behalf. AI does not get a HIPAA exemption; if it handles PHI, HIPAA is in scope.
What makes AI different is not the core rules but the new places PHI can leak: training data, prompts sent to third-party models, and outputs that could re-identify a person. Getting HIPAA right for AI means applying the familiar safeguards and closing these new gaps. One honest note up front: no software is "HIPAA compliant" on its own, and this guide is technical, not legal advice. It explains what to build; your compliance and legal experts confirm the rest.
Does HIPAA apply to AI in healthcare?
Yes. HIPAA applies to any AI application that creates, receives, stores or transmits PHI, in the same way it applies to any other software handling that data. The technology being "AI" changes nothing about whether HIPAA is in scope; what matters is whether PHI is involved.
There is one nuance worth understanding: if data is properly de-identified to the HIPAA standard, it is no longer PHI, and HIPAA does not apply to that de-identified data. This is why de-identification is an important tool for AI, for example in training. But the moment identifiable health data flows through your AI app, whether into a model, a prompt or a log, HIPAA obligations attach.
What does HIPAA require for an AI application?
HIPAA requires the same three categories of safeguard for an AI app as for any PHI system: administrative, physical and technical. In engineering terms, that means the controls below, designed in from the start rather than bolted on.
- Encryption of PHI in transit and at rest.
- Access control, including unique user IDs, role-based access and least privilege.
- Audit logging of who accessed PHI and when.
- Authentication and automatic session controls.
- Secure hosting, private or segmented as needed.
- Integrity controls so PHI is not improperly altered or destroyed.
- Business associate agreements with any vendor or AI service handling PHI.
- Documentation of the controls, so they can be evidenced.
These are the engineering half of compliance. The other half, policies, training, risk analysis and contracts, sits with your organisation. Software can be HIPAA-aligned; full compliance depends on both halves.
What new risks does AI add under HIPAA?
AI adds risk in three specific places that traditional apps do not have: training data, model inputs, and outputs. Each is a new way PHI can be exposed if it is not handled deliberately.
- Training data. Using identifiable PHI to train or fine-tune models can expose it; de-identification or proper safeguards and agreements are essential.
- Prompts and inputs. Sending PHI to a third-party AI model (for example a general LLM API) means that vendor now processes PHI, which requires a BAA and appropriate controls, or de-identification first.
- Outputs and re-identification. AI outputs can inadvertently reveal or reconstruct identifiable information, so outputs need review and safeguards.
- Logging. AI systems often log prompts and responses; if those contain PHI, the logs are PHI and must be protected.
- Model and vendor sprawl. Multiple AI services each touching PHI multiply the BAAs and controls you must manage.
The theme: AI creates new PHI pathways. HIPAA-aligned AI means mapping every place PHI could flow, then securing or de-identifying it.
Business associate agreements and AI vendors
If you use a third-party AI service that processes PHI on your behalf, that vendor is a business associate, and you need a business associate agreement (BAA) with them. Without a BAA, sending PHI to that service is a HIPAA problem, however good the technology is.
This is one of the most common AI-and-HIPAA traps. Many general-purpose AI APIs are not covered by a BAA by default, and some are not available under a BAA at all. The safe patterns are to use AI services that will sign a BAA and are configured for healthcare, to de-identify data before it reaches a service that will not, or to run models in an environment you control. Decide this deliberately, because it shapes your whole architecture.
AI does not change HIPAA's rules. It changes where PHI can escape: training data, prompts to third-party models, outputs and logs. HIPAA-aligned AI means finding and closing every one of those gaps.