AI is already underwriting policies, evaluating risks to decide on coverage, flagging fraud, routing claims, and answering customer questions. What’s changing now isn’t whether insurers can use AI, but how closely regulators are watching its use.
If you’re a carrier experimenting with automation, machine learning, or generative AI, the regulatory conversation is no longer theoretical. It’s here, and it’s evolving fast.
Regulators aren’t trying to shut down innovation, but “We didn’t mean for the model to do that” isn’t an acceptable answer anymore. Let’s explore the key ways oversight is changing.
Regulators Are Shifting From “What Is AI?” to “Show Us How It Works”
A few years ago, most regulators were still trying to understand basic AI concepts. The focus has now shifted.
Instead of asking “Are you using AI?”, regulators are asking:
- How does this model make decisions?
- What data went into training it?
- How do you test for bias?
- Who is accountable when it gets something wrong?
This shift matters because regulators view AI systems as any other core insurance function, subject to governance, documentation, and oversight.
If your AI vendor says their model is a “black box”, that’s no longer a comfortable place to be.
Bias and Fairness Are the Center of the Conversation
Underwriting and pricing have always been sensitive areas. AI adds gasoline to that fire.
Regulators focus on:
- Proxy discrimination using variables that indirectly reflect protected classes, such as race or gender
- Historical bias baked into training data
- Models that drift over time without detection.
Many states are aligning with frameworks like the NAIC’s AI Model Bulletin, which emphasizes fairness, accountability, and transparency. The translation for carriers is that even if your intent is clean, you’re still responsible for the outcomes.
“Vendor Did It” Is Not a Defense
One of the biggest misconceptions carriers still hold is that responsibility ends with the vendor, when it doesn’t. Regulators are making it clear:
- If you deploy the tool, you own the risk.
- If you use a third-party model, you must understand and defend it.
- If harm occurs, a software contract offers no defense.
This situation is why vendor due diligence now includes:
- Model explainability
- Data governance practices
- Audit rights
- Ongoing performance monitoring.
AI procurement is becoming a compliance decision, not just a technology one.
Human Oversight Is No Longer Optional
One of the strongest regulatory themes emerging is the requirement for meaningful human oversight. That doesn’t mean humans must redo every decision, but it means a focus on:
- Clear escalation paths
- The ability to override automated outcomes
- Documented review processes for edge cases.
Especially in claims and underwriting, regulators want to see that humans remain accountable, even as machines do the heavy lifting. Oversight-free automation is now a regulatory red flag.
Documentation Is the New Differentiator
The carriers moving the fastest with AI are the ones that document best. Regulators don’t expect perfection; they expect:
- Clear model documentation
- Defined risk management frameworks
- Ongoing testing and reporting
- Evidence that leadership is involved.
If you can explain why a system exists, how you monitor it, and what happens when it fails, you’re already ahead of most of the market.
What Smart Carriers Are Doing Right Now
The most forward-thinking insurers aren’t waiting for enforcement actions. They’re:
- Creating internal AI governance committees
- Involving compliance teams early in AI pilots
- Updating model risk management frameworks to include AI
- Training non-technical leaders to ask better questions.
They’re treating AI like underwriting rules or claims guidelines, just another system that needs guardrails. Regulators don’t oppose AI; they oppose surprises.
Treat AI as a strategic tool, and regulators will collaborate rather than resist. The actual risk isn’t AI use itself.
The risk of using AI you can’t explain is that explanation is now everything. Welcome to the future of insurance that runs at the speed of now.
Agility Holdings Group (AHG) invests in innovative InsurTech, HealthTech, and related companies that aim to revolutionize access to insurance products, establish patient care, and improve health outcomes. Please visit our LinkedIn page for more information about AHG.