Artificial intelligence is already in our hospitals. 5 questions people want answered

Artificial intelligence (AI) is already being used in health care. AI can look for patterns in medical images^[1] to help diagnose disease. It can help predict who in a hospital ward might deteriorate^[2]. It can rapidly summarise^[3] medical research papers to help doctors stay up-to-date with the latest evidence.

These are examples of AI making or shaping^[4] decisions health professionals previously made. More applications are being developed.

But what do consumers think of using AI in health care? And how should their answers shape how it’s used in the future?

Read more: AI is already being used in healthcare. But not all of it is 'medical grade'^[5]

What do consumers think?

AI systems are trained to look for patterns in large amounts of data. Based on these patterns, AI systems can make recommendations, suggest diagnoses, or initiate actions. They can potentially continually learn, becoming better at tasks over time.

If we draw together international^[6] evidence, including our own^[7] and that^[8] of others^[9], it seems most consumers accept the potential value of AI in health care.

This value could include, for example, increasing the accuracy of diagnoses^[10] or improving access to care^[11]. At present, these are largely potential, rather than proven, benefits.

But consumers say their acceptance is conditional. They still have serious concerns.

1. Does the AI work?

A baseline expectation is AI tools should work well. Often, consumers say AI should be at least as good as a human doctor^[12] at the tasks it performs. They say we should not use AI if it will lead to more incorrect diagnoses or medical errors.

Read more: AI chatbots are still far from replacing human therapists^[13]

2. Who’s responsible if AI gets it wrong?

Consumers also worry that if AI systems generate decisions – such as diagnoses or treatment plans – without human input, it may be unclear who is responsible for errors. So people often want clinicians to remain responsible for the final decisions, and for protecting patients^[14] from harms.

Read more: Who will write the rules for AI? How nations are racing to regulate artificial intelligence^[15]

3. Will AI make health care less fair?

If health services are already discriminatory^[16], AI systems can learn these patterns from data and repeat or worsen^[17] the discrimination. So AI used in health care can make health inequities worse. In our studies consumers said this is not OK^[18].

4. Will AI dehumanise health care?

Consumers are concerned AI will take the “human” elements out of health care, consistently saying AI tools should support rather than replace^[19] doctors. Often, this is because AI is perceived to lack important human traits, such as empathy^[20]. Consumers say the communication skills, care and touch of a health professional are especially important when feeling vulnerable.

Read more: Chatbots for medical advice: three ways to avoid misleading information^[21]

5. Will AI de-skill our health workers?

Consumers value human clinicians and their expertise. In our research with women^[22] about AI in breast screening, women were concerned about the potential effect on radiologists’ skills and expertise. Women saw this expertise as a precious shared resource: too much dependence on AI tools, and this resource might be lost.

Consumers and communities need a say

The Australian health-care system cannot focus only on the technical elements of AI tools. Social and ethical considerations, including high-quality engagement with consumers and communities, are essential to shape AI use in health care.

Communities need opportunities to develop digital health literacy^[23]: digital skills^[24] to access reliable, trustworthy health information, services and resources.

Respectful engagement with Aboriginal and Torres Strait Islander communities must be central. This includes upholding Indigenous data sovereignty, which the Australian Institute of Aboriginal and Torres Strait Islander Studies describes as^[25]:

the right of Indigenous peoples to govern the collection, ownership and application of data about Indigenous communities, peoples, lands, and resources.

This includes any use of data to create AI.

References

1. ^{^} medical images (journal.achsm.org.au)
2. ^{^} deteriorate (www.jmir.org)
3. ^{^} rapidly summarise (elicit.com)
4. ^{^} or shaping (theconversation.com)
5. ^{^} AI is already being used in healthcare. But not all of it is 'medical grade' (theconversation.com)
6. ^{^} international (www.sciencedirect.com)
7. ^{^} our own (www.uow.edu.au)
8. ^{^} and that (journal.achsm.org.au)
9. ^{^} of others (humanfactors.jmir.org)
10. ^{^} accuracy of diagnoses (www.jmir.org)
11. ^{^} access to care (mental.jmir.org)
12. ^{^} human doctor (journal.achsm.org.au)
13. ^{^} AI chatbots are still far from replacing human therapists (theconversation.com)
14. ^{^} protecting patients (www.nature.com)
15. ^{^} Who will write the rules for AI? How nations are racing to regulate artificial intelligence (theconversation.com)
16. ^{^} already discriminatory (theconversation.com)
17. ^{^} repeat or worsen (www.science.org)
18. ^{^} is not OK (journals.sagepub.com)
19. ^{^} support rather than replace (journals.sagepub.com)
20. ^{^} such as empathy (journals.sagepub.com)
21. ^{^} Chatbots for medical advice: three ways to avoid misleading information (theconversation.com)
22. ^{^} research with women (journals.sagepub.com)
23. ^{^} digital health literacy (theconversation.com)
24. ^{^} digital skills (www.goodthingsfoundation.org.au)
25. ^{^} describes as (aiatsis.gov.au)
26. ^{^} regulators (theconversation.com)
27. ^{^} citizens’ jury on AI in health care (www.uow.edu.au)
28. ^{^} national roadmap (aihealthalliance.org)
29. ^{^} Worried about AI? You might have AI-nxiety – here's how to cope (theconversation.com)
30. ^{^} analysis (pubmed.ncbi.nlm.nih.gov)