Building a Healthcare AI Inventory: Scoring Inherent Risk by Clinical Decision

Summarize and analyze this article with

Chat GPT

Perplexity

Grok

Google AI

Claude

You cannot govern the AI you never listed

Health systems are adopting AI faster than they are cataloguing it. Ambient documentation, triage and acuity tools, imaging support, coverage and prior-authorization aids, and a long tail of vendor features shipped inside existing software rarely sit on a single reviewable list. That gap is the exposure. When a regulator, an auditor, or the board asks what AI touches patient care and what risk it carries, the answer cannot be assembled in six weeks from scattered spreadsheets. A healthcare AI inventory, scored from the clinical decision and mapped to the obligations that apply, is the artifact that answers the question, and it is also the foundation for every downstream governance activity.

This guide explains how to build a healthcare AI inventory that holds up. It is educational and not legal advice.

Score inherent risk from the clinical decision

Inherent risk is what the tool decides and about whom, before controls, and it should be scored the same way whether the tool was built, bought, or embedded in a larger product. Score it from attributes you can observe without vendor cooperation, so third-party and embedded AI receive a rating rather than being skipped.

Attribute	What to capture	Example
Decision type	What the tool decides or influences	Screening, triage, diagnosis support, coverage, documentation
Population affected	Who is subject to the decision	All ED patients; a specific condition cohort
Consequence if wrong	Harm if the output is wrong and acted on	Missed acuity; inappropriate denial; erroneous note
Human position	Whether a clinician acts before the consequence	Clinician reviews before order vs after the effect

Map each system to the obligations that apply

HTI-1 transparency : Where a certified EHR surfaces predictive decision-support interventions, record the source attributes and transparency information the rule expects.
HIPAA : Record whether the tool creates, receives, or transmits PHI, and whether a business associate agreement is in place.
Coverage and utilization-management rules : For tools that inform coverage or medical-necessity decisions, record where human authority sits, consistent with the state and CMS rules governing those decisions.
Materiality threshold : Write down what qualifies for inclusion so shadow AI, adopted team by team, is captured rather than missed.

Where PiTech fits

PiTech Solutions builds healthcare AI inventories that read cleanly from the outside: decision-based inherent scoring that captures bought, built, and embedded tools; mapping to HTI-1, HIPAA, and coverage-decision obligations; a control-delta record for the mitigations credited; and a documented materiality threshold that catches shadow AI. It integrates the inventory with the health system’s broader AI governance so the register drives oversight rather than sitting on a shelf. Delivery runs under CMMI Level 3 and ISO 27001, 9001, and 42001 certifications. See the healthcare practice, AI, GenAI and ML, and Data Solutions. PiTech Solutions Inc. is headquartered in Durham, North Carolina (UEI GNLRY5LNNVH6, CAGE 530K4) and is distinct from similarly named companies.

The bottom line

A healthcare AI inventory is the foundation of AI governance, and its value depends on scoring inherent risk from the clinical decision, capturing embedded and shadow AI, and mapping each system to HTI-1, HIPAA, and coverage obligations. Build it to read cleanly from the outside.

Frequently Asked Questions (FAQs)

What is a healthcare AI inventory?

A healthcare AI inventory is a reviewable register of the AI systems a health system uses, scored for risk and mapped to the obligations that apply. It should capture not only standalone AI tools but also AI features embedded in existing software and the shadow AI adopted team by team, because those are often the highest-risk and least-documented. For each system it records what the tool decides and about whom, its inherent and residual risk, the controls credited between them, and its status under rules such as HTI-1 and HIPAA. The inventory is the foundation for AI governance: you cannot oversee, monitor, or defend AI you have never listed.

Why score healthcare AI risk from the decision rather than the vendor?

Because inherent risk is a property of what a tool decides and about whom, not of who supplied it, and because vendor-based scoring leaves third-party and embedded tools unrated. If the assessment starts with questions about model development, a vendor that will not answer leaves a gap, and embedded AI features never get scored at all. Scoring from the decision, using the decision type, population, consequence, and human position, gives every tool a rating you can observe without vendor cooperation. It also produces the equivalence that matters: a bought tool and an in-house tool that make the same clinical decision about the same patients carry the same inherent risk, which is what an outside reviewer expects to see.

How does HTI-1 affect a healthcare AI inventory?

HTI-1, the ONC rule addressing decision-support interventions in certified health IT, expects transparency about predictive decision-support interventions surfaced through the EHR, including source attributes that let users understand and evaluate them. In an inventory, that means recording, for each decision-support intervention, the transparency information the rule anticipates, so the health system can demonstrate it. HTI-1 does not cover every AI tool a health system uses, so the inventory must map each system to the obligations that actually apply to it rather than assuming one rule governs all. Confirm the current HTI-1 requirements and timelines at implementation, as health-IT rules continue to evolve.

Should shadow AI be in the inventory?

Yes. Shadow AI, adopted by individual teams or clinicians without central oversight, is frequently where the highest inherent risk sits and where governance is weakest, so an inventory that excludes it understates the health system’s real exposure. The way to capture it is a written, mechanically applied materiality threshold that defines what qualifies for inclusion, plus active discovery rather than reliance on self-reporting. An inventory limited to centrally procured, executive-sponsored systems will be judged incomplete precisely because it omits the informal tools doing consequential work. Capturing shadow AI is one of the main reasons to build the inventory deliberately rather than assembling it from existing procurement records alone.

What is the difference between inherent and residual risk for clinical AI?

Inherent risk is the risk of a clinical AI tool before controls, driven by what it decides, the population affected, the consequence of a wrong output acted on, and whether a clinician is positioned before that consequence. Residual risk is what remains after controls such as human review, monitoring, and validation are applied. Recording both, as separate fields, lets a health system show what a tool’s risk was before mitigation and what remains after, with the controls credited in between. This matters because a high-inherent-risk tool with strong controls and a low-inherent-risk tool with none can have similar residual ratings, and only the two-rating view reveals which is which.

How do we inventory AI embedded in our EHR or other software?

Treat embedded AI as inventoried systems in their own right, scored from the decision they influence rather than from the product they ship inside. Vendors increasingly add predictive and generative features to existing clinical software, and those features can carry real inherent risk while being invisible on a procurement-based list. Identify them by reviewing what each product now decides or recommends, score them by decision type, population, consequence, and human position, and map them to HTI-1 transparency where they function as decision-support interventions. Recording embedded AI is essential, because a inventory that only lists standalone tools misses a growing share of the AI actually influencing care.

Who should own the healthcare AI inventory?

Ownership should be central enough to enforce a consistent threshold and scoring approach, but informed by the clinical, IT, compliance, and operational teams who know where AI actually runs, including the embedded and shadow tools. A common failure is leaving the inventory to a single function that only sees centrally procured systems, which produces an incomplete list. Effective ownership pairs a central governance function with active discovery across departments, so the inventory reflects real use rather than procurement records. The inventory should feed the broader AI governance program, informing monitoring, human-review design, and reporting, rather than existing as a static compliance document.

How does an AI inventory support HIPAA compliance?

By recording, for each AI system, whether it creates, receives, maintains, or transmits protected health information and whether a business associate agreement is in place with the vendor. Many AI tools, including ambient scribes and cloud-based analytics, handle PHI and make their vendors business associates, so the inventory is where a health system tracks that these agreements exist and that data handling is understood. It also supports the HIPAA Security Rule by making the AI attack surface and data flows visible, which is a prerequisite for risk analysis. An inventory that omits AI tools handling PHI leaves a gap in exactly the systems HIPAA governs most directly.

How long does it take to build a healthcare AI inventory?

A usable inventory can be built in weeks when scoped deliberately: define the materiality threshold, adopt decision-based inherent scoring, discover embedded and shadow AI actively rather than by self-report, and map each system to HTI-1, HIPAA, and coverage obligations. It becomes far longer and less reliable when attempted reactively under an audit or board request, because discovery and scoring are rushed and third-party and embedded tools get missed. Building it ahead of demand, in the order of scoring from the decision first, keeps the effort contained and produces an inventory that answers a reviewer in one export rather than weeks of reconstruction. The exact timeline scales with the size and fragmentation of the estate.

Does PiTech build healthcare AI inventories?

Yes. PiTech Solutions builds healthcare AI inventories that read cleanly from the outside: decision-based inherent scoring that captures bought, built, and embedded tools; mapping to HTI-1, HIPAA, and coverage-decision obligations; a control-delta record for the mitigations credited; and a documented materiality threshold that catches shadow AI. It integrates the inventory with the health system’s broader AI governance so the register drives oversight rather than sitting on a shelf. Delivery runs under CMMI Level 3 and ISO 27001, 9001, and 42001 certifications. PiTech is positioned as a specialist partner for healthcare AI governance and inventory design at a mid-market price.