Four signals arrived in healthcare AI
this week.
Most leaders read them separately.
A law. A coalition paper.
A benchmark. A product announcement.
Each one stands on its own.
Together they describe the same room.
The room where your health system
has to stand behind what its AI did.
The room where the accountability
infrastructure either exists
or it does not.
This issue connects all four.
SECTION 1 — The Direction California Named
California passed a law last week
most health system leaders read
as someone else's problem.
It belongs to all of them.
AB 853 requires AI systems to document
where their outputs came from. Large
platforms must provide tools to detect
AI-altered content. By 2027 they must
label AI-generated content with
provenance data. By 2028 capture devices
must embed disclosure metadata at the
point of creation.
Provenance. Traceability. Accountability
for every AI output.
And it carries an exemption most legal
teams highlighted and then filed away.
The exemption is for hospitals.
That exemption will not last.
California is not asking whether your AI
works. California is asking whether your
institution can show its work.
Those are different questions.
The first is answered by your vendor.
The second is answered by your
accountability infrastructure,
or it is not answered at all.
The law does not yet require health
systems to document AI outputs. But the
direction is unmistakable. Regulators are
moving toward a world where every AI
action in a clinical environment must be
traceable, documented, and defensible at
the institutional level. Not at the vendor
level. At yours.
Most health systems read AB 853 and
concluded they were not yet affected.
They are right about the timeline.
They are wrong about the implication.
AB 853 does not read as a compliance
deadline. It reads as a design
specification.
The question is whether you build that
capability before or after your first
accountability crisis.
SECTION 2 — The Gap 35 Institutions Named
In October 2025, thirty-five of the most
credentialed institutions in healthcare AI
signed their names to a problem statement.
Not a solution. A problem statement.
The Medlog coalition, researchers from
Harvard, Stanford, Mayo Clinic, and
thirty-two other institutions, published
a framework for logging clinical AI
decisions. The paper did not announce
a breakthrough. It announced a gap.
Clinical AI systems are making decisions
inside health systems right now. Those
decisions are not being logged in any
standardized, retrievable, institutionally
defensible way.
The vendor has a record.
The EHR has a record.
The institution has nothing that connects
them to a coherent accountability trail.
Medlog exists because thirty-five
institutions looked at their own
deployments and could not answer a
simple question.
What did the AI do, and why.
Not in theory. Not in the demo.
In the actual clinical environment
where patients were present.
The logging gap is not a technical
problem. It is an architectural decision
no one made.
When a health system deploys an AI agent
it inherits three things from the vendor.
A performance benchmark, telling you what
the AI scored in controlled conditions.
A workflow audit trail, telling you what
the system recorded. A contract, telling
you what the vendor will not be
responsible for.
None of the three tells you what your
institution is prepared to say when a
patient, a regulator, or a lawyer asks
what the AI did in their specific case,
and why.
Thirty-five institutions named that gap
in October. Most boards still do not
know it exists.
SECTION 3 — The Benchmark That Made
The Logging Gap Urgent
Thirty-five institutions named the
logging gap. Most leaders filed it under
future infrastructure problems.
Then Stanford showed why it is a present
one.
Stanford built MedAgentBench. A simulated
hospital. 100 realistic patient profiles.
Over 800,000 data points. 300 tasks
designed by physicians. Ordering tests.
Prescribing drugs. Reconciling conflicting
records. The exact conditions a clinical
AI agent meets in a real health system.
The best-performing agent scored
approximately 70% accuracy.
Wrong on roughly 1 in 3 physician-designed
tasks, in controlled conditions.
Most other agents scored in the 60s.
Now hold that next to Section 2.
Your health system is deploying agents
that are wrong often enough to matter.
And you have no infrastructure to tell you
which actions were wrong, which patients
were affected, or who is responsible for
the gap between what the agent did and
what should have happened.
The benchmark did not test edge cases.
The hardest failures concentrated in
multi-step workflows, ambiguous
information, and incomplete or messy data.
The exact conditions every real hospital
runs on every single day.
And there is a distinction in the data
most leaders missed. The benchmark
separated retrieval from action.
Retrieval means look up information.
Action means write to the record, send
the referral, execute the order.
Models performed noticeably better at
retrieval than at action.
The failure is not random. It concentrates
exactly where the stakes are highest.
Your agent that retrieves a lab result is
probably fine. Your agent that sends the
referral, updates the care plan, or
modifies the prior authorization is
operating in the exact category where the
benchmark showed the highest failure
rates.
And if you cannot log what those action
tasks produced at the institutional level,
you cannot catch the errors that matter
most before they reach a patient.
Section 2 named the logging gap.
Section 3 is why it is not a future
infrastructure investment.
It is an active patient safety obligation.
SECTION 4 — The Factory That Made
The Gap Larger
The week ended with an announcement that
made all three signals more urgent.
Not a new AI tool. A factory for
building them.
Until this week Epic shipped three fixed
AI agents, each in a defined lane. Agent
Factory flips that. It gives every health
system a drag-and-drop builder for custom
agents that reason through multi-step
workflows, make decisions, and act across
silos. Documentation. Billing. Referrals.
Without manual intervention.
A referral agent checks whether the
referral was sent, confirms the
appointment, identifies what fell through,
and notifies the care team. End to end.
No human in the loop.
This is not AI assisting a clinician.
This is AI acting inside the workflow.
Epic built something important into Agent
Factory. A full audit trail of every
action every agent takes. They know the
accountability problem is real. They built
for it at the workflow level.
But a workflow audit trail is not an
institutional accountability infrastructure.
When your custom agent checks a referral,
confirms an appointment, identifies a gap,
and notifies a care team without a
clinician in the loop, Epic's audit trail
records what the agent did.
It does not record who in your institution
is accountable for it.
It does not record whether the action was
within the governance parameters you
approved.
It does not record your escalation protocol
when the agent is wrong.
It does not produce the institutional
accountability document a regulator, a
lawyer, or a patient's family would need
to see.
Epic built the factory.
Nobody built the room.
CLOSING
California named the direction. The most
credentialed coalition in healthcare AI
named the logging gap. Stanford proved the
agents are wrong often enough that the gap
is a patient safety obligation. Epic gave
every health system a factory to deploy
those agents at scale.
Four signals. One week. One room.
Your board meeting is in 90 days. By then
every one of these will be older, and the
room will be no more built than it is
today unless someone starts building it.
You already know whether you could show
that board what your AI deployments
actually did, and who was accountable
when they did it.
That answer is the room. Build it before
the quarter does it for you.
MedicoVigilance™ is published every two weeks by Mo Johnson, MD MBA, founder of GPe Research. Each issue teaches one piece of the clinical AI accountability discipline your institution needs before the next deployment decision.
Forward this to a colleague whose institution is running on output no one has been named to own.