Doctors, bots and the buck

Medical Protection (a not-for-profit protection organisation for doctors, dentists, and healthcare professionals) has released a paper calling for law reform to protect doctors, and other healthcare professionals, when their decisions are influenced by AI recommender systems. They also worry about how doctors are going to be on the hook for the outcomes of “AI scribes, automated documentation assistants, triage algorithms, and other forms of clinical decision support”.

Medical Protection says “Clinicians could be left in a bind. If they reject an AI output and things go wrong, there is a real and significant risk that they could face allegations of negligence if the AI output was deemed to be appropriate. But if they follow an inappropriate recommendation and the outcome is still poor, they risk becoming the ‘mark’ or the ‘liability sink’ - the obvious target for a damages claim. All of this despite having little control over, and perhaps only a limited grasp of, how the AI system reaches its conclusions.”

There are two key areas of law which are most relevant to this discussion:

• negligence and, in particular, professional negligence; and
• product liability law, namely the Consumer Protection Act 1987.

Medical Protection calls out a"gap" in product liability legislation in England and Wales since it has been held that software is not a “product” for the purposes of the Consumer Protection Act, so that this legislation would not cover liability for harm caused by AI. They highlight how, in contrast, European legislators are amending product liability law to encompass AI and deal with issues like proving causation.

The position isn't entirely settled in England and Wales, with the Law Commission saying it plans to review product liability law in the future. 

The UK Jurisdiction Taskforce (UKJT) gave us some helpful commentary in their consultative statement on liability for AI harms earlier this year (final statement being released in July):

Public consultation – Liability for AI Harms under the private law of England and Wales - LawtechUK : LawtechUK

In their paper, the UKJT said that if AI were incorporated into a product, such as a smart fridge that the product liability regime would apply, but essentially, as Medical Protection says, standalone AI is not otherwise caught.

Of course, the attraction of strict liability under product liability regimes is appealing, particularly as many would argue it is impossible to establish issues like causation in a complicated AI supply chain, with AI tools that are hard to explain. 

However, before we reach for a strict liability regime and apportion all the blame to the manufacturer of the AI product, the UKJT highlight that the beauty of negligence is its flexibility to apply to the facts of each case. In negligence, we can ask ourselves things like who should owe a duty of care to the person harmed. What standard should they be held to, did their breach of that standard cause the harm and could this harm be too remote? We get to examine the deployment of the AI tool, all the parties involved in the supply chain and the impact of their behavior on the harm caused. 

Policy-wise, we have to protect doctors who are being put in impossible situations — faced with AI tools they don't have time or knowledge to assess, or they simply will not use them. At the same time, we also want to protect patients, who have zero choice over the use of the AI tool and who might find that the AI tool provider is hard to pursue when things go wrong, e.g. in another country or insolvent. Doctors could say they should not be expected to review the effectiveness and safety of AI being deployed, but surely someone in the procurement team in the hospital or trust does owe some obligation to the patient and the doctor to assess the tool, to understand it and to put in place measures to continually monitor it? Negligence, applied to a particular case, could attribute blame both to a party who failed to deploy AI responsibly and a manufacturer who made claims about its effectiveness, which turned out not to be accurate. 

In all of this debate, what we lack is predictability. Whilst negligence gives us flexibility, the absence of a body of case law applying negligence law to medical AI use cases destroys any certainty. Doctors, NHS Trusts, medical AI manufacturers do not yet know what standard of care they will be expected to comply with. What we do know is that courts take into account, when assessing the standard of care, the guidance and rules of professional bodies, so these bodies could have some influence on determining what a “reasonable doctor” should do before accepting the decision of AI, what a “reasonable NHS Trust” might do to govern AI. 

Whilst we wait for the debate about liability for AI harms to take its course, the best course of action is deep scrutiny of the effectiveness of AI tools at the point of purchase and deployment, strong assurance and testing, policies about user interaction (developed with healthcare professionals and patients), ethical guardrails and engagement with the manufacturer to understand how the tool was designed and trained.

Subscribe to receive more articles like this here.