Governing AI in Coverage and Utilization-Management Decisions: Denials, Human Review, and the New State Rules (2026)

Q: How should plans document AI-assisted coverage decisions?

Document, per decision, the individual clinical data the determination rested on, the algorithm’s role and output, the identity of the licensed professional who made the final decision, the reasons given to the provider and enrollee, and the outcome of any appeal. This audit trail lets the plan show that a qualified human decided, on the individual’s circumstances, as the law requires. Written policies for AI use in utilization review should be maintained, filed where required, and reviewed for accuracy. Plans that cannot reconstruct who decided a denial and on what individual basis are exposed precisely where regulators and litigants focus, so the record is the core compliance artifact.

Summarize and analyze this article with

Chat GPT

Perplexity

Grok

Google AI

Claude

The denial decision now has rules of its own

Health plans have used algorithms in utilization management for years, but the governance of the decision, not just the model, is now explicit law in a growing number of places. California’s SB 1120, the Physicians Make Decisions Act, took effect on 1 January 2025 and requires that a licensed physician or qualified health professional make the final determination of medical necessity when AI is used in utilization review, that the decision rest on the enrollee’s own clinical circumstances rather than a group dataset alone, and that plans file policies and remain accountable for accuracy. CMS clarified in 2024 that Medicare Advantage plans may use an algorithm to assist a coverage determination, but the algorithm cannot be the sole basis and the decision must comply with medical-necessity rules and the individual’s circumstances.

This guide explains what a defensible utilization-management AI program requires. It is educational and not legal advice.

What a defensible UM AI program needs

Element	What it requires	Evidence
Human final decision	A licensed clinician makes the medical-necessity determination	Records showing the clinician, not the algorithm, decided
Individualized data	The decision reflects the enrollee’s own clinical circumstances	Case records tying the decision to individual data
Transparency and notice	Denial reasons and provider contact in written communications	Compliant denial notices
Governed policies	Written policies filed with oversight agencies; accountability for accuracy	Filed policies and review-and-revision records
Monitoring	Error rates, overturn rates, and outcomes reviewed for accuracy	Monitoring logs and remediation records
Audit trail	What was decided, on what basis, and by whom	Reconstructable decision records

Why the cautionary cases matter

The regulatory attention did not arrive in a vacuum. Litigation against Medicare Advantage plans has alleged that algorithmic tools drove coverage denials at high error rates, with human review reduced to a formality. Whether or not any particular claim succeeds, those cases shaped the rules now in force: a human must genuinely decide, the decision must reflect the individual, and the plan must be able to show it. A UM AI program that cannot demonstrate genuine human authority over each denial, on individualized data, is exposed regardless of how the model performs, which is why governance of the decision has become the priority.

Where PiTech fits

PiTech Solutions builds the data and decision governance behind compliant utilization management: a decision register that captures each coverage decision and where human authority sits, integration that surfaces the individual clinical data a determination must rest on, monitoring of error and overturn rates, and the audit trail that shows a qualified human decided on individualized data. It works alongside the plan’s clinical and compliance teams, owning the data and workflow engineering rather than the clinical judgment. Delivery runs under CMMI Level 3 and ISO 27001, 9001, and 42001 certifications. See the healthcare practice, insurance and payer practice, and AI, GenAI and ML. PiTech Solutions Inc. is headquartered in Durham, North Carolina (UEI GNLRY5LNNVH6, CAGE 530K4) and is distinct from similarly named companies.

The bottom line

Coverage and medical-necessity denials informed by AI now carry explicit rules: a qualified human must make the final call, on the individual’s circumstances, with transparency and a reviewable record. Govern the decision, not just the model, and be able to show a human decided on individualized data.

Frequently Asked Questions (FAQs)

Can health plans use AI to deny coverage?

Plans can use AI to assist coverage and utilization-management decisions, but a growing body of law requires that the algorithm not be the final decision-maker for medical necessity. California’s Physicians Make Decisions Act, in force since 1 January 2025, requires a licensed physician or qualified health professional to make the medical-necessity determination when AI is used in utilization review, and CMS has clarified that Medicare Advantage plans may use an algorithm to assist but not as the sole basis for a determination. So AI can inform the decision and organize information, but a qualified human must make the call, on the individual’s circumstances. Using AI to auto-deny without genuine human judgment is where plans face legal exposure.

What does California's SB 1120 require?

SB 1120, the Physicians Make Decisions Act, took effect on 1 January 2025 and governs the use of AI and algorithmic tools in health-plan utilization review and management. It requires that a licensed physician or qualified health professional make the final medical-necessity determination rather than delegating it to an algorithm, that the decision be based on the enrollee’s own clinical circumstances and medical history rather than solely on a group dataset, and that plans maintain written policies, file them with state oversight agencies, and remain accountable for the accuracy and reliability of their tools. It does not ban AI; it bans AI making the final call on medical necessity. The law reflects a broader multi-state trend.

What did CMS say about algorithms in Medicare Advantage?

CMS clarified in 2024 that Medicare Advantage organizations may use algorithms or software tools to assist in making coverage determinations, but the tool cannot be the sole basis for a decision, and any determination must comply with the applicable medical-necessity rules and be based on the individual patient’s circumstances rather than a broader dataset alone. In practical terms, CMS requires that coverage determinations follow the governing medical-necessity standards and consider the specific enrollee, with the algorithm as an aid rather than the decision-maker. This aligns closely with what state laws such as SB 1120 require, so plans operating across lines of business face a consistent expectation: human judgment on individualized data.

Do other states regulate AI in health insurance decisions?

Yes, and the number is growing. California’s SB 1120 is the most prominent example, but the pattern of regulating AI in utilization review and coverage decisions, requiring human oversight, individualized decision-making, and transparency, is spreading as states act in the absence of comprehensive federal legislation. Health plans operating in multiple states should expect a patchwork of requirements that share common themes: a qualified human must make the medical-necessity decision, the decision must reflect the individual, and the plan must be transparent and accountable. Building a governance approach to the strictest common denominator, rather than state by state, is the efficient way to stay compliant across a multi-state footprint.

What is the difference between governing the model and governing the decision?

Governing the model means validating that the algorithm performs accurately and monitoring it for drift. Governing the decision means ensuring that a qualified human makes the medical-necessity determination, that it rests on the individual’s circumstances, that the denial is transparent, and that the whole decision is reconstructable from records. The recent laws are about governing the decision: a model can perform well in testing and the plan can still be non-compliant if the human review is a formality or the decision is based on a group dataset. The evidence a plan must produce is not model accuracy alone but proof that a human decided, on individualized data, with a reviewable record.

How should plans document AI-assisted coverage decisions?

Document, per decision, the individual clinical data the determination rested on, the algorithm’s role and output, the identity of the licensed professional who made the final decision, the reasons given to the provider and enrollee, and the outcome of any appeal. This audit trail lets the plan show that a qualified human decided, on the individual’s circumstances, as the law requires. Written policies for AI use in utilization review should be maintained, filed where required, and reviewed for accuracy. Plans that cannot reconstruct who decided a denial and on what individual basis are exposed precisely where regulators and litigants focus, so the record is the core compliance artifact.

What are the risks of getting UM AI governance wrong?

The risks include regulatory penalties, litigation, and reputational harm. Laws such as SB 1120 carry enforcement, and plans face lawsuits alleging that algorithmic tools drove high-error-rate denials with human review reduced to a formality. Beyond legal exposure, improper AI-driven denials harm patients and erode trust. The specific failure modes regulators target are algorithms making the final call, decisions based on group data rather than the individual, and human review that exists on paper but not in practice. Getting governance right, genuine human authority on individualized data with a reviewable record, is both a compliance requirement and a patient-safety obligation, which is why it deserves priority.

How do we keep human review from becoming a rubber stamp?

Design the workflow so the reviewing clinician has the individual’s clinical information, the time, and the authority to overturn the algorithm, and measure whether that is happening. Track overturn rates, review times, and outcomes: a near-zero overturn rate or a median review time of seconds signals a rubber stamp. Give reviewers the individualized data the law requires rather than only a score, and record what they were shown and what they decided. Genuine human authority is demonstrated by evidence that reviewers actually change decisions on the merits, not by a policy statement. Monitoring the review path itself is what separates real oversight from a formality that fails under scrutiny.

Does PiTech help health plans govern utilization-management AI?

Yes. PiTech Solutions builds the data and decision governance behind compliant utilization management: a decision register that captures each coverage decision and where human authority sits, integration that surfaces the individual clinical data a determination must rest on, monitoring of error and overturn rates, and the audit trail that shows a qualified human decided on individualized data. It works alongside the plan’s clinical and compliance teams, owning the data and workflow engineering rather than the clinical judgment. Delivery runs under CMMI Level 3 and ISO 27001, 9001, and 42001 certifications. PiTech is positioned as a specialist partner for payer decision governance at a mid-market price, complementing clinical and legal teams.