Thu Sep 9

why home quarantine apps are a bad idea

Written by: Toby Walsh, Professor of AI at UNSW, Research Group Leader, UNSW

South Australia has begun a trial ^[1] of a new COVID app to monitor arrivals into the state. SA Premier Steven Marshall claimed “every South Australian should feel pretty proud that we are the national pilot for the home-based quarantine app”.

He then doubled down with the boast that he was “pretty sure the technology that we have developed within the South Australia government will become the national standard and will be rolled out across the country.”

Victoria too has announced impending “technologically supported ^[2]” home quarantine, though details remain unclear. Home quarantine will also eventually be available for international arrivals ^[3], according to Prime Minister Scott Morrison.

The South Australian app has received little attention in Australia, but in the US the left-leaning Atlantic magazine called it “as Orwellian as any in the free world ^[4]”. Right-wing outlets such as Fox News and Breitbart ^[5] also joined the attack, and for once I find myself in agreement with them.

Location tracking and facial recognition

The South Australian home quarantine app uses facial recognition software to identify users. Government of South Australia ^[6]

Despite the SA Premier’s claims, this isn’t the first such app to be used in Australia. A similar home-quarantine app ^[7] is already in use for arrivals into WA, and in some cases the Northern Territory ^[8].

Both apps uses geolocation and facial recognition software to track and identify those in quarantine. Users are required to prove they are at home when randomly prompted by the application.

In SA, you have 15 minutes to get the face recognition software to verify you’re still at home. In WA, it is more of a race. You have just 5 minutes before you risk a knock on the door from the police.

Another difference is that the SA app is opt-in. Currently. The WA app is already mandatory for arrivals from high risk areas like Victoria. For extreme risk areas like NSW, it’s straight into a quarantine hotel.

Reasons for concern

But why are we developing such home-quarantine apps in the first place, when we already have a cheap technology to do this? If we want to monitor that people are at home (and that’s a big if), wouldn’t one of the ankle tags already used by our corrective services for home detention be much simpler, safer and more robust?

There are many reasons to be concerned about home-quarantine apps.

First, they’ll likely be much easier to hack than ankle tags. How many of us have hacked geo-blocks to access Netflix in the US, or to watch other digital content from another country? Faking GPS location on a smartphone ^[9] is not much more difficult.

Second, facial recognition software is often flawed, and is frequently biased against people of colour and against women. The documentary Coded Bias ^[10] does a great job unpicking these biases.

The documentary Coded Bias explains the common inbuilt flaws of facial recognition software.

Despite years of effort, even the big tech giants like Google and Amazon have been unable to eliminate these biases from their software ^[11]. I have little hope the SA government or the WA company GenVis, the developers of the two Australian home-quarantine apps, will have done better.

Indeed, the Australian Human Rights Commission has called for a moratorium on the use of facial recognition software ^[12] in high-risk settings such as policing until better regulation is in place to protect human rights and privacy.

Third, there needs to be a much more detailed and public debate around issues like privacy, and safeguards put in place based on this discussion, in advance of the technology being used.

With COVID check-in apps, we were promised the data would only be used for public health purposes. But police forces around Australia have accessed this information for other ends on at least six occasions ^[13]. This severely undermines the public’s confidence and use of such apps.

Before it was launched, the Commonwealth’s COVIDSafe app had legislative prohibitions put in place on the use of the data collected for anything but contact tracing. This perhaps gave us a false sense of security as the state-produced COVID check-in apps did not have any such legal safeguards. Only some states have retrospectively introduced legislation to provide such protections.

Fourth, we have to worry about how software like this legitimises technologies like facial recognition that ultimately erode fundamental rights such as the right to privacy.

If home-quarantine apps work successfully, will they open the door to facial recognition being used in other settings? To identify shop lifters? To provide access to welfare? Or to healthcare? What Orwellian world will this take us to?

The perils of facial recognition

In China, we have already seen facial recognition software used to monitor and persecute the Uighur minority ^[15]. In the US, at least three Black people have already wrongly ended up in jail due to facial recognition errors ^[16].

Facial recognition is a technology that is dangerous if it doesn’t work (as it often the case). And dangerous if it does. It changes the speed, scale and cost of surveillance.

With facial recognition software behind the CCTV cameras found on many street corners, you can be tracked 24/7. You are no longer anonymous when you go out to the shops. Or when you protest about Black lives mattering or the climate emergency.

High technology is not the solution

High tech software like facial recognition isn’t a fix for the problems that have plagued Australia’s response to the pandemic. It can’t remedy the failure to buy enough vaccines, the failure to build dedicated quarantine facilities, or the in-fighting and point-scoring between states and with the Commonwealth.

I never thought I’d say this but, all in all, I think I’d prefer an ankle tag. And if the image of the ankle tag seems too unsettling for you, we could do what Hong Kong has done and make it a wristband ^[17].