How AI Is Making Healthcare IT Smarter and More Fragile at Once

Much like other industries, healthcare is finding new ways to make artificial intelligence useful across its daily operations. However, healthcare organizations don't always have the same freedom to experiment, especially when sensitive information and critical systems are involved. 

In the effort to bring AI into existing workflows, some organizations are moving faster than their internal controls can support. A 2025 HFMA survey of 233 U.S. health systems found that only 18% had mature AI governance and fully developed strategies.

Sometimes, technology itself becomes a problem. Another 2025 survey revealed that 77% of surveyed healthcare IT leaders identified immature AI tools among their two biggest implementation barriers.

Both findings paint a clear contradiction between speed of adoption and strength of support. Going forward, we’ll dig into why this contradiction exists and how it plays out in daily operations. We will also cover how to gain the benefits without inheriting the risks.

Healthcare IT Has a Tireless Teammate in the Form of AI 

AI is giving healthcare IT teams room to do more with the systems and people they already have. It’s particularly useful in areas where healthcare systems handle large volumes of information and repetitive tasks.

From clinical documentation and coding assistance to patient scheduling and operational forecasting, it can help teams work through information faster. IT departments are also using AI to categorize support tickets, retrieve internal records, automate routine workflows and detect suspicious network activity.

A 2025 MGMA survey of 351 medical groups found that 68% had introduced or expanded AI use during the year. Documentation, scheduling, coding and analytics were among the reported applications.

There’s a catch, though. Every new capability can also require additional integrations, system permissions and access to sensitive information. As AI becomes more involved in daily operations, these dependencies introduce risks of their own.

Wider Access Sometimes Equals Wider Exposure

AI tools need access to documents, applications, databases, and internal knowledge sources to be useful. The same access removes information barriers. Fewer barriers can weaken the boundaries around sensitive data, e.g., patient medical records and insurance information.

In a recent interview, Alex Tyrrell, CTO of Wolters Kluwer Health, warned, “We're familiar with the idea of a hallucination and how that can have an impact, but perhaps not with the idea that exposing protected and private data to these models is really an existential risk.”

Access permissions can create problems of their own, especially when AI tools are connected to several internal systems. An AI-powered search tool connected to hospital databases, for instance, could retrieve confidential information if its permissions aren't properly configured.

Similarly, using AI assistants to summarize documents or prepare internal communications can expose patient records, financial details and proprietary information through employee prompts. Mainstream conversational AI assistants retain these conversations or use submitted data for model training, depending on their settings.

A business AI assistant with no-log processing, encryption and contractual privacy protections can reduce such exposure to a great extent. However, securing the tool itself is only part of the responsibility.

Healthcare organizations also need to control which information AI systems can retrieve, how employees interact with them and where processed data ultimately goes. Wider access can improve productivity considerably, provided existing privacy boundaries remain intact.

More Automation Creates More Failure Points Too

Data exposure is only half of the risk picture. The other half shows up in daily operations. Look, the more work AI handles, the more dependent daily operations become on models, APIs, external vendors and connected systems.

Each connection introduces another possibility for something to go wrong. An integration might send incomplete information, or an automated workflow could act on records that haven't been updated.

Even the AI-generated results themselves require attention. A recent study by JAMA Internal Medicine compared AI-generated and physician-written hospital discharge summaries. Researchers found an average of 2.91 errors per AI-generated summary, compared with 1.82 in physician-written summaries.

Similar problems can become harder to manage when automated decisions feed directly into other workflows. For example, an incorrect coding suggestion could influence a billing process, while an unavailable AI service could interrupt tasks dependent on its output.

The real concern, therefore, is how far an individual error can travel before someone catches it. AI can make operations faster and more manageable, provided organizations retain enough oversight to detect failures and intervene when needed.

Governance Has to Catch Up With Capability

Governance tends to get treated as a brake on progress. Done well, it does the opposite by clearing the bottlenecks that slow AI down in the first place. It starts with ownership. Every AI system needs a named owner who understands what it does and answers for how it behaves. 

Clear data-access boundaries come next, so each tool only reaches for what its task requires and nothing more. Higher-risk decisions deserve human review, and monitoring has to continue long after deployment. When an output or integration fails, a documented escalation path gets the right people involved quickly rather than leaving teams to guess who’s responsible.

Approving a tool once isn’t enough, because AI doesn’t stay still. Models get updated, vendors change their products, integrations expand, and employees find new ways to use the tools.

Governance has to follow AI through its entire working life, not just stop at procurement. What we are striving for is a balance with enough structure to keep risk visible and enough freedom to keep teams moving.

FAQs

Can an AI tool be HIPAA compliant?

No AI tool is HIPAA compliant on its own. Compliance depends on how a tool is configured, what data it handles, and whether the vendor signs a business associate agreement. Organizations should confirm these details with their privacy and legal teams first.

What is shadow AI?

Shadow AI is the use of AI tools by employees without approval or oversight from IT. A clinician pasting notes into a public chatbot is one example. It creates blind spots because teams can’t protect or audit data flowing into tools they don’t know exist.

Will AI replace healthcare IT professionals?

It’s unlikely. AI handles repetitive work well, but someone still needs to configure integrations, manage access, review alerts, and respond when systems fail. As adoption grows, the work tends to move toward oversight, vendor management, and security, which keeps skilled IT professionals central.

Key Data Points at a Glance

Controlled Adoption Is the End Goal

Healthcare IT has always involved finding practical ways to improve systems while keeping essential services dependable. AI brings a different set of possibilities, but the underlying responsibility remains familiar. What has changed is how quickly technology can act and how widely its decisions can influence daily operations. 

As organizations gain experience with AI, they'll also develop a clearer sense of where automation works well and where human judgment remains important. Some lessons will come through careful testing, others through everyday use. With the right approach, healthcare IT can continue to become more capable without sacrificing the reliability people depend on.