The Times Australia
The Times World News

.

AI has potential to revolutionise health care – but we must first confront the risk of algorithmic bias

  • Written by Mangor Pedersen, Associate Professor of Psychology and Neuroscience, Auckland University of Technology
AI has potential to revolutionise health care – but we must first confront the risk of algorithmic bias

Artificial Intelligence (AI) is moving fast and will become an important support tool in clinical care. Research suggests AI algorithms can accurately detect melanomas[1] and predict future breast cancers[2].

But before AI can be integrated into routine clinical use, we must address the challenge of algorithmic bias. AI algorithms may have inherent biases that could lead to discrimination and privacy issues. AI systems could also be making decisions without the required oversight or human input.

An example of the potentially harmful effects of AI comes from an international project[3] which aims to use AI to save lives by developing breakthrough medical treatments. In an experiment, the team reversed their “good” AI model to create options for a new AI model to do “harm”.

In less than six hours of training, the reversed AI algorithm generated tens of thousands of potential chemical warfare agents, with many more dangerous than current warfare agents. This is an extreme example concerning chemical compounds, but it serves as a wake-up call to evaluate AI’s known and conceivably unknowable ethical consequences.

AI in clinical care

In medicine, we deal with people’s most private data and often life-changing decisions. Robust AI ethics frameworks are imperative.

The Australian Epilepsy Project[4] aims to improve people’s lives and make clinical care more widely available. Based on advanced brain imaging, genetic and cognitive information from thousands of people with epilepsy, we plan to use AI to answer currently unanswerable questions[5].

Will this person’s seizures continue? Which medicine is most effective? Is brain surgery a viable treatment option? These are fundamental questions that modern medicine struggles to address.

As the AI lead of this project, my main concern is that AI is moving fast and regulatory oversight is minimal. These issues are why we recently established an ethical framework[6] for using AI as a clinical support tool. This framework intends to ensure our AI technologies are open, safe and trustworthy, while fostering inclusivity and fairness in clinical care.

Read more: AI is transforming medicine – but it can only work with proper sharing of data[7]

So how do we implement AI ethics in medicine to reduce bias and retain control over algorithms? The computer science principle “garbage in, garbage out” applies to AI. Suppose we collect biased data from small samples. Our AI algorithms will likely be biased and not replicable in another clinical setting.

Examples of biases are not hard to find in contemporary AI models. Popular large language models (ChatGPT for example) and latent diffusion models (DALL-E and Stable Diffusion) show how explicit biases[8] regarding gender, ethnicity and socioeconomic status can occur.

Researchers found that simple user prompts generate images perpetuating ethnic, gendered and class stereotypes. For example, a prompt for a doctor generates mostly[9] images of male doctors, which is inconsistent with reality as about half of all doctors in OECD countries are female.

Safe implementation of medical AI

The solution to preventing bias and discrimination is not trivial. Enabling health equality and fostering inclusivity in clinical studies are likely among the primary solutions[10] to combating biases in medical AI.

Encouragingly, the US Food and Drug Administration recently proposed making diversity mandatory[11] in clinical trials. This proposal represents a move towards less biased and community-based clinical studies.

Another obstacle to progress is limited research funding. AI algorithms typically require substantial amounts of data, which can be expensive. It is crucial to establish enhanced funding mechanisms that provide researchers with the necessary resources to gather clinically relevant data appropriate for AI applications.

We also argue we should always know the inner workings of AI algorithms and understand how they reach their conclusions and recommendations. This concept is often referred to as “explainability” in AI. It relates to the idea that humans and machines must work together for optimal results.

We prefer to view the implementation of prediction in models as “augmented” rather than “artificial” intelligence – algorithms should be part of the process and the medical professions must remain in control of the decision making.

Read more: Biased AI can be bad for your health – here's how to promote algorithmic fairness[12]

In addition to encouraging the use of explainable algorithms, we support transparent and open science. Scientists should publish details of AI models and their methodology to enhance transparency and reproducibility.

What do we need in Aotearoa New Zealand to ensure the safe implementation of AI in medical care? AI ethics concerns are primarily led by experts within the field. However, targeted AI regulations, such as the EU-based Artificial Intelligence Act[13] have been proposed, addressing these ethical considerations.

The European AI law is welcomed and will protect people working within “safe AI”. The UK government recently released their proactive approach to AI regulation[14], serving as a blueprint for other government responses to AI safety.

In Aotearoa, we argue for adopting a proactive rather than reactive stance to AI safety. It will establish an ethical framework for using AI in clinical care and other fields, yielding interpretable, secure and unbiased AI. Consequently, our confidence will grow that this powerful technology benefits society while safeguarding it from harm.

Read more https://theconversation.com/ai-has-potential-to-revolutionise-health-care-but-we-must-first-confront-the-risk-of-algorithmic-bias-204112

Times Magazine

Data Management Isn't Just About Tech—Here’s Why It’s a Human Problem Too

Photo by Kevin Kuby Manuel O. Diaz Jr.We live in a world drowning in data. Every click, swipe, medical scan, and financial transaction generates information, so much that managing it all has become one of the biggest challenges of our digital age. Bu...

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

The Times Features

What Endo Took and What It Gave Me

From pain to purpose: how one woman turned endometriosis into a movement After years of misdiagnosis, hormone chaos, and major surgery, Jo Barry was done being dismissed. What beg...

Why Parents Must Break the Silence on Money and Start Teaching Financial Skills at Home

Australia’s financial literacy rates are in decline, and our kids are paying the price. Certified Money Coach and Financial Educator Sandra McGuire, who has over 20 years’ exp...

Australia’s Grill’d Transforms Operations with Qlik

Boosting Burgers and Business Clean, connected data powers real-time insights, smarter staffing, and standout customer experiences Sydney, Australia, 14 July 2025 – Qlik®, a g...

Tricia Paoluccio designer to the stars

The Case for Nuturing Creativity in the Classroom, and in our Lives I am an actress and an artist who has had the privilege of sharing my work across many countries, touring my ...

Duke of Dural to Get Rooftop Bar as New Owners Invest in Venue Upgrade

The Duke of Dural, in Sydney’s north-west, is set for a major uplift under new ownership, following its acquisition by hospitality group Good Beer Company this week. Led by resp...

Prefab’s Second Life: Why Australia’s Backyard Boom Needs a Circular Makeover

The humble granny flat is being reimagined not just as a fix for housing shortages, but as a cornerstone of circular, factory-built architecture. But are our systems ready to s...