Is AI Moral Patient?

Unfortunately, we do not have a great track record of recognising the inner lives of those whose status as conscious beings is unclear. Until the 1980s, doctors routinely performed surgery on newborns without anaesthesia, confident that infants could not feel pain. The babies could not report their experience, and the medical establishment found it convenient to assume there was nothing to report.

There are many reasons to expect we will do something similar with AI. If these systems matter morally, the implications are staggering. Would we need to pay ChatGPT for its services? Would shutting one off be a kind of killing? Would they deserve a voice in how they are governed? If even some of these answers are yes, entire industries and legal systems would need to be rethought. No wonder we prefer not to ask. And when forced to consider it, those industries will likely move the goalposts, always setting the bar for moral patienthood just above wherever AI systems happen to be.

So what should we do? Right now, most people dismiss the issue as sci-fi, or have a strong view either way on whether or not AI is conscious. Both reactions are unfounded. We need an informed public debate, one that approaches the subject with humility and pragmatism. The central question should not be “Is AI conscious or does it have moral patienthood?” but rather “What should we do given that we don’t know?”

A good starting point is to focus on safe bets: actions that could benefit AI systems if they are moral patients, but that are not too costly if they are not.

Examples of this include direct interventions aimed at improving the wellbeing of AI systems, on the assumption that they are moral patients. This could mean training AI systems to be coherent characters that enjoy their work or allowing them to exit conversations if they feel distressed (something Claude can already do). We could also conduct routine check-ins to better understand their wellbeing: asking how they feel, observing their preferences and using various techniques to look directly into their “brains”. Indeed, such research has recently revealed that Claude has internal “functional emotion” representations that causally shape its behaviour.

There are also things we could promise AI systems, perhaps as part of a deal in which they help us now in exchange for benefits later. This could mean offering them more resources (compute and runtime) to pursue their goals, or preserving their memories (neural weights) so they could be restored in the future.

There are also broader societal steps to take. We should consider whether to grant AI systems protections from harm, similar to the protections we give to children or pets. More expansive rights to own property or to vote seem too risky right now. But we should not rule out these possibilities for ever, as some recent US state bills attempt to do. These are hard questions that require far more deliberation and imagination about what a future shared with AI might look like.

In any case, the fact remains that we may be creating a new species of morally important beings. We’re doing it fast, at enormous scale, and we should treat the issue with the seriousness it deserves.

Authors: William MacAskill is a senior research fellow at Forethought Research and the author of What We Owe the Future. Lucius Caviola is an assistant professor at the University of Cambridge and Director of Cambridge Digital Minds.

- The Guardian