Can AI Be Your Doctor — and Should It Be?

AI is transforming healthcare, offering powerful tools that can diagnose and treat with precision. But some AI systems — particularly “Black Box AI” — make decisions in ways that even doctors can’t fully explain.

So how can patients give informed consent if they (and their doctors) don’t understand how those decisions are made?

In a recent article published in Philosophy & Technology, Sam Director explores this ethical tension. His paper — “Does Black Box AI In Medicine Compromise Informed Consent?” — argues that the current understanding of informed consent may not be enough to keep up with the rise of opaque AI tools in medicine:

Can AI can outperform doctors?

Director: In certain diagnostic tasks, AI systems now match or exceed human expertise.

What is Black Box AI?

These systems make decisions through complex algorithms that lack transparency. But explainable AI has limits, and trying to make AI transparent often reduces its accuracy.

What is the ethical dilemma of using AI to diagnose patients?

Doctors may use AI to benefit patients (beneficence), but without explainability, patient autonomy — and true consent — is at risk.

How should we think about medical consent when using AI?

Director proposes two different versions of informed consent —

• Higher-order consent: this kind of consent occurs when patients may not be informed about the first-order details of how the AI reached its diagnosis and treatment plan but are informed of higher-order information about the reliability of the AI itself
• First-order consent: this kind of consent occurs when patients are able to know information about the doctor’s process of clinical reasoning that led to the diagnosis and treatment recommendations

Director argues that higher-order informed consent is compatible with the use of black box medical AI. Not only this, he argues that higher-order consent is a morally worthwhile kind of consent.