Bringing AI into a practice raises an obvious question: what happens to patient data? Owners and managers ask it in every first conversation, and they should. Artificial intelligence tools are new, the marketing around them is often vague, and it can be hard to tell whether a product is built for healthcare or simply repackaged for it. The good news is that the core requirements have not changed. HIPAA still applies, and the same questions that govern your scheduling software and EHR apply to AI.
This article explains what HIPAA asks of any tool that touches protected health information, how those requirements map onto AI systems, and what to check before you sign anything. It is a practical overview, not legal advice, and it is worth involving a qualified attorney or compliance professional when you are making decisions with significant risk.
The basics: what counts as protected health information
Protected health information, or PHI, is individually identifiable health information held or transmitted by a covered entity or a business associate. In a practice, that includes obvious items like names, dates of birth, and insurance numbers, along with clinical details, appointment records, billing data, and anything else that links a person to their care. A recording of a patient visit, a transcript of a phone call, or a scanned referral all count as PHI when they contain identifying information.
If a tool receives, stores, transmits, or processes PHI on your behalf, the company providing that tool is usually a business associate. That status carries obligations, and those obligations are what you should be checking.
The core requirements that still apply
HIPAA's Security Rule requires covered entities and business associates to protect electronic PHI through administrative, physical, and technical safeguards. For AI tools, the technical safeguards deserve the closest attention. Data should be encrypted in transit using current standards and encrypted at rest. Access should be controlled so that only authorized users and systems can reach patient information, and each user should have the minimum access needed for their role. Systems should keep audit logs that record who accessed or changed PHI and when.
The Privacy Rule governs how PHI can be used and disclosed. Using a patient's information to provide care or to run your practice is generally permitted, but using it for other purposes, such as training a model that serves other customers, requires careful thought and a clear contractual basis. The Minimum Necessary standard also applies. A tool should receive only the information it needs to do its job, not the entire chart.
The Business Associate Agreement
Before any vendor handles PHI, you need a signed Business Associate Agreement, or BAA. The BAA is a contract that defines how the vendor will protect the information, what it can and cannot do with it, how breaches will be reported, and what happens to the data when the relationship ends. Without a BAA, sharing PHI with a vendor can put your practice out of compliance, even if the vendor is otherwise secure.
Not every AI company offers a BAA, and not every tool that offers one applies it to every feature. Read the agreement carefully. Confirm which products and services are covered, and check whether any subprocessors, such as cloud providers or transcription services, are also bound by appropriate terms. If a vendor says it is "HIPAA compliant" without being willing to sign a BAA, treat that statement with caution.
Questions to ask a vendor before you sign
A focused set of questions will reveal most of what you need to know. Start with the basics. Where is patient data stored, and in which geographic region? Is that data processed in the United States, or does it leave the country? Who has access to it, both at the vendor and among any subprocessors? How is access controlled, and does the vendor support single sign-on and role-based permissions?
Questions to ask a vendor before you sign
- Where is patient data stored, and in which region?
- Is that data used to train or improve models, and if so, can that be turned off for your account?
- How long is data retained, and how is it deleted when you ask?
- Will the vendor sign a Business Associate Agreement that covers this specific product?
- Which subprocessors handle PHI, and do they also sign appropriate agreements?
- How is data encrypted in transit and at rest?
- What audit logs are available to you, and how long are they kept?
- How are security incidents detected, reported, and handled?
Pay particular attention to the question of model training. Some AI services use customer inputs to improve their models by default. For a healthcare practice, that may be unacceptable, and it may conflict with the terms of your own patient agreements. Ask directly whether your data is used for training, and get the answer in writing.
Data retention and deletion
AI tools often keep copies of inputs and outputs for logging, debugging, or quality review. Those copies are still PHI if they contain identifying information. Find out how long the vendor keeps them, where they are stored, and how you can request deletion. Your contract should specify retention periods that match your own policies, and it should describe what happens to data at the end of the relationship, including how it is returned or destroyed.
Retention is also an operational question. If a tool keeps recordings or transcripts indefinitely, you may be creating a larger exposure than you intended. Decide how long your practice needs the output, and configure the tool to match.
Human oversight and clinical accuracy
HIPAA does not regulate the clinical accuracy of AI output directly, but your clinical and professional responsibilities do. If a tool drafts a note, extracts data from a document, or suggests a scheduling action, a qualified person should review that output before it affects care or billing. Build review steps into the workflow, document who approved what, and make it easy for staff to correct errors. A tool that produces wrong information without anyone noticing is a risk to patients and to your practice, regardless of how well it protects data.
Your own responsibilities
Choosing a compliant vendor is necessary, but it is not sufficient. The practice still has obligations. Staff need training on what can and cannot be entered into a tool, how to recognize a potential breach, and whom to contact when something looks wrong. Access should be reviewed regularly, especially when someone changes roles or leaves the practice. Policies should be written down and kept current, including how AI tools are approved, how they are monitored, and how incidents are handled.
A risk assessment is a useful exercise before deployment. Map out where PHI flows through the proposed tool, identify the points where it could be exposed, and decide what safeguards address each risk. Revisit the assessment whenever the tool changes significantly, the vendor changes its terms, or your practice expands to a new location or service line.
Building compliance into the project from day one
The most common mistake is treating compliance as a final checkbox. By the time a system is nearly finished, it is expensive to change its data flows, storage locations, or access model. Compliance should shape the design from the first conversation. Decide early which data the system needs, where it will live, who will access it, and how long it will be kept. Build audit logging and access controls into the foundation rather than adding them later.
This is also where the choice of build partner matters. A developer who understands both the software and the regulatory context can make sensible architectural decisions, such as selecting US-region infrastructure that supports HIPAA, limiting data copies, and designing review steps into the workflow. A developer who treats compliance as an afterthought will create problems that take months to unwind.
A practical checklist
Before you bring any AI tool into your practice, confirm that the vendor will sign a BAA that covers the product you plan to use, that PHI is encrypted in transit and at rest, that access is role-based and logged, that your data is not used for training without your explicit agreement, that retention and deletion terms match your policies, and that there is a named human responsible for reviewing the tool's output. If any of these answers is unclear, pause and get clarity before you proceed.
The takeaway
HIPAA does not ban AI in healthcare. It asks the same questions it has always asked: who can see patient information, how it is protected, how it is used, and who is accountable when something goes wrong. Answer those questions carefully, get the commitments in writing, and keep a human in the loop for anything that affects care. Done that way, AI can be a useful part of your practice without putting your patients or your license at risk.
If you want AI built for healthcare from the foundation up, with the BAAs, encryption, and audit logging handled properly, that is what we do. Get in touch and we will show you how we approach compliance.