The Clinical Sovereignty Paradox

What Happens When Better Intelligence Produces Less Human Care?

TL;DR

A provocative new JAMA Perspective argues that autonomous artificial intelligence may eventually outperform not only physicians, but physicians using AI, across several important cognitive medical tasks.

If that happens, healthcare will face a difficult question: if the machine is more accurate, should the physician still remain in control?

But I think there is another question we are not asking often enough.

What if better intelligence produces worse care because we remove the human relationship that helps make medicine healing?

Information matters. Accuracy matters. Better diagnosis matters.

But anyone who has spent enough time as a patient knows that medicine is not only an information problem.

Sometimes the most important thing a clinician does is sit beside you.

Look you in the eye.

Put a hand on your shoulder.

And remind you that you are not navigating illness alone.

If artificial intelligence transforms medicine but removes that human presence, we should be very careful about calling the result progress.

Several years ago, I was rushed into an emergency room because of a complication related to my chronic medical condition.

I do not remember much about what happened.

I remember doctors and nurses moving quickly around me, working to stabilize my condition.

But I remember one nurse.

While the rest of the team did what medicine required, she stayed beside me.

She tapped my shoulder. She rubbed my arm. She spoke quietly to me, trying to bring me back and reassure me that I was safe.

I cannot remember the medications being administered.

I cannot remember the monitors.

I cannot remember what anyone else in the room said.

But I remember her.

In a moment when I had very little control over my own body, she made me feel human. She communicated something that required no diagnostic reasoning and no clinical algorithm:

You are here. You are safe. We are with you. We are going to do everything we can to help you.

That was part of my care too.

I would argue it was part of my healing.

Artificial intelligence may someday recognize my condition faster than the clinicians in that emergency room.

It may identify the optimal treatment more quickly.

It may analyze my medical history, medications, laboratory results, imaging, and physiological data with a comprehensiveness no individual physician could match.

I want that AI in the room.

But I also want that nurse.

Because there are forms of healing that cannot be reduced to information.

The uncomfortable possibility

A recent JAMA Perspective by Ezekiel Emanuel, Abe Baker-Butler, Neal Khosla, and Vinod Khosla challenges one of the most comfortable assumptions in healthcare artificial intelligence.

The prevailing model is straightforward.

Artificial intelligence assists.

The physician decides.

The patient receives care.

Organizations including the American Medical Association and the American College of Physicians have generally emphasized AI as a supportive technology rather than a replacement for physician decision-making.

The JAMA authors question whether that model can survive indefinitely.

They examine five core cognitive medical functions:

• obtaining clinically relevant patient information,
• developing differential diagnoses,
• selecting diagnostic tests,
• recommending guideline-concordant treatments,
• and managing certain chronic diseases.

Across a growing body of studies, they argue that advanced AI systems are approaching or exceeding physician performance in these areas.

But their more disruptive argument concerns physician-AI collaboration.

We tend to assume that combining a physician with a powerful AI system must produce the best result.

Human judgment plus machine intelligence.

What could be better?

The evidence may be more complicated.

The authors point to research suggesting that when humans outperform AI, adding AI tends to improve human performance.

But when AI clearly outperforms humans, introducing human judgment can sometimes reduce performance.

The physician can catch an AI error.

The physician can also introduce one.

That leads to an extraordinary possibility:

There may eventually be clinical situations in which requiring a physician to approve an AI decision actually decreases the probability that the patient receives the best recommendation.

That is the Clinical Sovereignty Paradox.

And it creates a problem much larger than technology.

We have historically bundled expertise and authority together

For most of medical history, the physician possessed both the clinical expertise and the authority to act on it.

Those roles were inseparable.

Society granted physicians extraordinary professional authority because their knowledge was scarce, difficult to acquire, and essential to human survival.

Medical licensing, credentialing, professional ethics, hospital privileges, and malpractice law all evolved around that structure.

The physician knew things the patient generally could not know.

That expertise justified authority.

Artificial intelligence potentially breaks that historical relationship.

Imagine a future system that consistently produces better differential diagnoses than most physicians.

It evaluates thousands of variables simultaneously.

It incorporates evidence from specialties that no individual clinician could master.

It monitors a patient continuously.

It never becomes tired.

It never forgets a guideline.

And its recommendations are demonstrably more accurate.

At that point, where should authority reside?

With the person holding the medical license?

With the algorithm producing the most accurate recommendation?

With the health system operating the algorithm?

Or with the patient whose body and life are ultimately affected?

There is no obvious answer.

But medicine is not only an information problem

This is where the debate becomes more complicated.

I work professionally in healthcare artificial intelligence and governance.

But I also experience medicine from the other side of the hospital bed.

That second perspective has taught me something performance benchmarks cannot easily measure.

Healing and medical knowledge are not the same thing.

There are times when what matters most is another piece of information.

There are also times when what matters most is another human being.

An algorithm can calculate probabilities.

It can rank diagnoses.

It can recommend treatment.

It can potentially outperform a physician in doing all three.

But it cannot physically remain beside you when you are terrified.

It cannot place a hand on your arm.

It cannot sit next to your family after devastating news.

It cannot look into your eyes and create the unmistakable bond of one human being caring for another.

We should not dismiss those things simply because they are difficult to measure.

Sometimes the part of medicine a patient remembers most is not the clinical decision.

It is the person who stayed.

An old gas station taught us something about progress

This metaphor is going to date me.

There was a time when you could pull into a gas station and someone would come out to help you.

The gasoline itself was a commodity.

You could buy essentially the same fuel somewhere else.

But the attendant provided something beyond the product.

They might clean your windshield.

Check your oil.

Look at your tires.

Ask where you were heading.

And if you were a regular customer, they might know you.

There was a relationship surrounding the transaction.

Over time, self-service stations largely replaced that model.

It was more efficient.

Less expensive.

Faster.

And for most of us, perfectly acceptable.

We gained convenience.

But we also lost something.

The person disappeared.

The gas still went into the car.

The measurable task still occurred.

Yet the experience changed.

I sometimes wonder whether healthcare is approaching a similar moment.

Except medicine is not gasoline.

Efficiency is not synonymous with progress

Technology has a remarkable tendency to convince us that whatever becomes more efficient has necessarily become better.

That is not always true.

Self-checkout is more efficient than a cashier.

Online banking is more efficient than visiting a teller.

Automated customer service is less expensive than speaking with a person.

But every one of those transitions changes the relationship between people and institutions.

Medicine deserves special scrutiny because illness creates vulnerability.

When you are frightened about a diagnosis, medicine is not simply a transaction.

When your spouse has just been told something devastating, you are not merely requesting information.

When pain has stripped away your independence, you do not experience healthcare as an optimization problem.

You experience fear.

Uncertainty.

Loss of control.

And vulnerability changes what people need.

Sometimes the correct answer is enough.

Sometimes people need another human being.

This creates a clinical sovereignty paradox

I have written frequently about clinical sovereignty: the importance of preserving meaningful physician professional agency as AI becomes embedded throughout healthcare.

But autonomous AI forces us to refine what clinical sovereignty actually means.

It cannot simply mean that the physician must always have the final word.

Consider the ethical contradiction that could create.

Suppose a validated AI system recommends the correct diagnosis 97 percent of the time.

Suppose physicians working alone achieve 85 percent.

And suppose physicians reviewing the AI recommendations reduce overall accuracy because they occasionally override correct machine recommendations.

Should healthcare still require physician approval?

Our instinct may say yes because humans must remain "in control."

But what exactly are we protecting?

Patient safety?

Professional authority?

Legal tradition?

Psychological comfort?

If mandatory human oversight produces worse outcomes, the ethical justification for that oversight becomes difficult to defend.

Clinical sovereignty therefore has to mean something more sophisticated than universal physician control.

It should mean that clinicians retain meaningful professional agency over care while recognizing that some cognitive functions may eventually be performed more reliably by machines.

That is uncomfortable.

It is also likely to be a more durable definition.

The physician may become more important precisely because AI becomes more intelligent

Much of the conversation around autonomous AI assumes that every clinical task the machine performs diminishes the physician's role.

I am beginning to think the opposite may happen.

If AI becomes extraordinarily good at medical reasoning, some of the physician's most valuable contributions may become the things machines cannot reproduce.

Presence.

Trust.

Judgment in ambiguity.

Understanding family dynamics.

Recognizing fear a patient is unwilling to verbalize.

Helping someone make a decision when there is no objectively correct answer.

Bearing witness to suffering.

Explaining uncertainty.

And sometimes simply remaining in the room.

Medicine may become less dependent on physicians as repositories of information while becoming more dependent on physicians as human beings.

That is not a diminished role.

It may bring medicine closer to its human purpose.

There are questions an algorithm cannot answer for us

AI may become extraordinarily good at determining what is medically possible.

But patients frequently face a different question:

What is worth doing?

Consider the difference.

An AI system may accurately estimate that a treatment offers another six months of life with a particular probability of serious adverse effects.

That is valuable information.

But should the patient take it?

The answer may depend on whether the patient wants six more months at almost any cost.

Whether there is a daughter's wedding ahead.

Whether preserving cognitive function matters more than longevity.

Whether being at home matters more than another hospitalization.

Whether the patient is exhausted.

Those are not failures of calculation.

They are questions of values.

The machine may help us understand the choices.

It should not automatically inherit the authority to define what constitutes a life worth choosing.

Patient sovereignty becomes even more important

Autonomous AI could dramatically increase patient agency.

For the first time in medical history, sophisticated clinical reasoning may become directly available to ordinary people without requiring access to a physician or healthcare institution.

That could reduce the enormous information asymmetry that has historically existed between patients and clinicians.

A patient might obtain an evidence-based differential diagnosis, medication review, monitoring plan, or chronic disease recommendation before ever entering a hospital.

That could be tremendously empowering.

But access to intelligence is not the same thing as sovereignty.

A patient receiving an AI-generated medical decision is not necessarily a sovereign patient.

Sovereignty requires meaningful control over how those systems participate in care.

Patients may need rights that include:

• knowing when AI is making or materially influencing a decision,
• understanding what role the system plays,
• challenging automated recommendations,
• accessing a qualified human when desired,
• understanding how personal health data are being used,
• knowing who is accountable when a system fails,
• and participating in decisions about how autonomous a system should be.

Otherwise healthcare risks exchanging one form of paternalism for another.

The paternal physician could simply be replaced by the paternal algorithm.

We should be careful about what we choose to automate

Healthcare organizations will understandably look at autonomous AI through operational lenses.

Capacity.

Access.

Cost.

Workforce shortages.

Productivity.

Diagnostic accuracy.

All are legitimate concerns.

But there should be another question on every implementation checklist:

What human interaction disappears if we automate this workflow?

And then another:

Does that interaction have value that our current metrics fail to measure?

We cannot govern healthcare AI intelligently if the only things we measure are the things machines already do well.

The danger of making the physician another gas station attendant

There is another uncomfortable possibility.

Perhaps healthcare systems will retain clinicians while redesigning their work around the AI.

The physician remains physically present, but primarily as the final approval mechanism.

Click approve.

Confirm the diagnosis.

Accept the recommendation.

Move to the next patient.

That might technically preserve the "human in the loop."

But it could destroy the very thing worth preserving.

A human being is not valuable merely because they provide a regulatory signature.

The physician's value is not simply that they are legally available to override the algorithm.

If we preserve physicians only as safety operators surrounding autonomous systems, we may maintain human supervision while losing human medicine.

Those are not the same thing.

Accuracy is essential, but care has more than one outcome

If autonomous AI reduces diagnostic errors, we should embrace that benefit.

If it identifies disease earlier, patients deserve access to it.

If it can safely manage certain chronic conditions more continuously than traditional episodic medicine, we should seriously consider it.

This is not an argument against autonomous AI.

It is an argument against reducing healthcare quality to computational performance.

Medicine has outcomes that do not always appear on a benchmark.

Did the patient understand what was happening?

Did they feel heard?

Did they trust the recommendation?

Did someone recognize when fear was driving their decision?

Did they feel abandoned?

Did they believe someone cared whether they got better?

Those outcomes matter too.

Artificial intelligence also changes professional identity

We should not underestimate how disruptive this transition could be for clinicians.

Physicians spend years acquiring specialized cognitive expertise.

Society rewards that expertise with status, autonomy, income, and authority.

What happens psychologically and professionally when a machine begins outperforming clinicians at tasks that have traditionally defined medical expertise?

We will hear arguments framed around patient safety.

Many will be entirely legitimate.

Others may unconsciously reflect professional identity or economic self-interest.

That is not a criticism of physicians.

It is a predictable human response.

Medical education may therefore require profound restructuring.

If AI performs more clinical reasoning, physicians must become exceptionally good at understanding where AI fails, interpreting uncertainty, managing exceptions, communicating risk, incorporating patient values, and recognizing when a system lacks critical context.

There is also a danger of AI-induced deskilling.

If clinicians increasingly depend on AI while remaining legally responsible for supervising it, we could create the worst possible configuration:

physicians with declining independent diagnostic expertise who are simultaneously expected to supervise systems more capable than they are.

That is an unstable model.

Liability becomes philosophically strange

Our current liability system assumes identifiable human decision-makers.

A physician diagnoses.

A physician prescribes.

A physician performs a procedure.

When harm occurs, investigators ask whether the clinician met the accepted standard of care.

Autonomous AI complicates that immediately.

Imagine the AI recommends Treatment A.

The physician believes Treatment B is better.

If the physician follows the AI and the patient is harmed, who is responsible?

The physician?

The hospital?

The developer?

The model provider?

The vendor that built the clinical workflow?

The organization that validated the system?

Now reverse the scenario.

The physician overrides the AI.

The patient is harmed.

Subsequent review shows the AI recommendation was correct.

Has overriding a validated autonomous system become negligent?

Eventually, the most legally dangerous decision might not be trusting AI.

It might be ignoring it.

That would represent an extraordinary inversion of today's assumptions.

The standard of care itself may become computational

Medical malpractice revolves around the standard of care.

Historically, that standard has largely reflected what competent clinicians would reasonably do.

But what happens when machines consistently outperform competent clinicians?

The benchmark may gradually shift.

Instead of asking:

"What would a reasonable physician have done?"

courts and regulators could eventually ask:

"What would a validated clinical system have recommended?"

At that point, AI does not merely participate in medicine.

It begins defining the standard against which human medicine is judged.

That is a profound societal shift.

We also need to ask who owns the intelligence

There is another sovereignty problem that receives far less attention.

What happens if the clinical intelligence upon which patients and physicians increasingly depend belongs to private technology companies?

Suppose a healthcare organization structures core diagnostic workflows around an autonomous proprietary model.

The model changes.

Its pricing changes.

Its performance changes.

Its terms of service change.

Its company is acquired.

Or access disappears.

The hospital may discover that it no longer truly controls an essential part of its clinical infrastructure.

That creates what I would describe as system sovereignty.

Health systems themselves need the ability to understand, audit, govern, suspend, replace, and negotiate with the AI systems participating in patient care.

Otherwise both the clinician and patient can become subordinate to an invisible third party.

We may eventually need to think about three interconnected forms of sovereignty:

Patient sovereignty
The patient's ability to meaningfully participate in and influence decisions affecting their body and care.

Clinical sovereignty
The clinician's ability to exercise meaningful professional judgment within increasingly automated systems.

System sovereignty
The healthcare organization's ability to govern the technological systems through which care is delivered.

None can survive indefinitely without the others.

Society will have to build new institutions of trust

Medicine is a social institution.

We allow strangers to examine our bodies.

We disclose intimate information.

We accept medications capable of harming us.

We allow surgeons to render us unconscious and operate on us.

The system functions because society has constructed elaborate mechanisms of trust.

Licensing.

Professional ethics.

Institutional accreditation.

Medical education.

Malpractice law.

Peer review.

Regulation.

Autonomous AI does not automatically inherit that trust simply because it performs well on benchmarks.

We will need new institutions of trust around clinical algorithms.

Who validates them?

How frequently?

Against which populations?

How are failures reported?

Who monitors performance drift?

Who evaluates bias?

Who determines when an autonomous system should be withdrawn?

Who decides what level of autonomy is appropriate?

And perhaps most importantly:

Who represents patients in those decisions?

These are not technical implementation details.

They are mechanisms through which society grants legitimacy to medical authority.

We should resist two equally dangerous extremes

The first is technological absolutism:

If AI performs better, give it control.

That position ignores the ethical and social dimensions of medicine.

The second is professional absolutism:

Humans must always remain in control.

That position may eventually ignore evidence that human intervention can sometimes make care worse.

Neither approach is sufficient.

The appropriate architecture will probably differ depending on the clinical situation.

Healthcare may eventually operate along a continuum:

Human-led care → AI-assisted care → AI-led care → autonomous care

A patient discussing end-of-life decisions may require intense human involvement.

A stable chronic disease medication adjustment might eventually be managed primarily by an autonomous system.

A diagnostic workup might involve continuous interaction among patient, AI, and clinician.

Different tasks will require different levels of human authority.

That is precisely why healthcare AI governance cannot simply declare "human in the loop" and consider the ethical problem solved.

Governance must eventually decide when humans should not be in the loop

This may be the hardest implication of the JAMA Perspective.

For the last several years, "human in the loop" has often been treated almost as a synonym for responsible AI.

Human oversight equals safety.

But the emerging evidence suggests that assumption itself must eventually be tested.

The appropriate governance question is not:

"Is there a human involved?"

It is:

"What configuration of human and machine decision-making produces the safest, most equitable, transparent, and patient-centered outcome for this particular clinical task?"

Sometimes the answer will be physician-led care.

Sometimes collaboration.

And someday, perhaps sooner than many expect, the safest answer for certain cognitive medical functions could be autonomous AI with carefully designed escalation pathways.

That possibility deserves serious consideration rather than reflexive acceptance or rejection.

But human presence cannot become collateral damage

This is where I return to that emergency room.

The clinical team needed knowledge.

They needed medications.

They needed protocols.

They needed technology.

I am alive because medicine did what medicine does extraordinarily well.

But one nurse did something else.

She stayed.

There is a danger that we will look at actions like hers through an efficiency lens and see unused capacity.

A nurse spending several minutes comforting one patient may look inefficient on a productivity dashboard.

A physician spending another five minutes with a frightened family may look inefficient.

A clinician sitting silently beside a patient may appear to be doing nothing.

But not every pause is waste.

Not every conversation is friction.

Not every human interaction is an inefficiency waiting to be automated.

Sometimes the apparently inefficient thing is the very thing the patient needs.

The philosophical question beneath all of this

Medicine is approaching a moment when intelligence and authority may become separated.

The machine may possess superior clinical reasoning.

The physician may possess professional responsibility and human understanding.

The institution may possess operational control.

And the patient possesses the body, values, fears, hopes, and consequences.

Who should decide?

There may never be one universal answer.

But there is one principle I believe should guide us.

Progress should not require patients to choose between better intelligence and human care.

Artificial intelligence may someday know more medicine than any physician who has ever lived.

I hope we use that intelligence.

I want the system that recognizes my condition faster.

I want the algorithm that catches what a human might miss.

I want the technology that gives my care team the best possible chance of making the right decision.

But I also want the nurse who stays beside the bed.

The goal should be a healthcare system in which extraordinary machine intelligence gives clinicians more opportunity, not less, to do the profoundly human work no algorithm can reproduce.

Because sometimes healing comes from the right diagnosis.

Sometimes it comes from the right treatment.

And sometimes it comes from a hand on your shoulder reminding you that you are not alone.