How much is NZ Covid Tracer really being used?
Much has been made in the media about the usage of New Zealand’s bespoke QR code based contact tracing aid New Zealand Covid Tracer. In an upcoming research article, Dr Bronwyn Howell and Professor Petrus Potgieter explore it's strengths and weakenesses.
Much has been made in the media over recent days about the apparent increase in usage of New Zealand’s bespoke QR code based contact tracing aid New Zealand Covid Tracer. The app was released in May 2020 to facilitate app users to keep a diary of locations visited, to aid contact tracers in the event that the individual was subsequently diagnosed with Covid-19. Our analysis of the app, compared to Australia’s, a version of which is soon to be published in the journal Digital Policy Regulation and Governance, found strengths and weaknesses in its usefulness in supporting contact tracing activities. A significant weakness was the high compliance burden it placed on users.
In January, the Minister for Covid-19 declared that use of the app had fallen to woefully low levels over the summer holiday period, and intimated that increased use was essential if New Zealand was to be successful in its virus elimination strategy. His pleas were supported by academics working with the government. A substantial advertising campaign has promoted greater usage.
Following the recent episodes of local breakouts after quarantine breakdowns at the Pullman hotel and community-based transmission in South Auckland associated with Papatoetoe High School, it appears that usage has increased substantially. Ministry of Health data show the number of scans registered daily increased by more than 540,000 in the 24 hours after the Papatoetoe cases were revealed.
On face value, surely this increase shows that the campaign for increased use has been successful?
We have been analysing usage of the app since its May 2020 release. In our view, the recent increase in usage is real. However, when considered across the life of the app, there is less reason for rejoicing.
To be sure, there was a surge in scanning activity following the recent outbreak, but it pales in comparison to the surge observed following the community outbreak in Auckland in August. The graph above shows the number of scans per day per downloaded app. In August, scanning activity peaked at 1.2 scans per downloaded app per day. The recent peak hit just over 0.7 scans per day. Yes, a lot of scans have been recorded (over 1.85 million on February 18-19). But an awful lot of apps have been downloaded and users registered (nearly 2.7 million, more than 50% of the “team of 5 million”, by February 23). The average over the period from May 2020 to February 2021 is 0.32 scans per registered app per day. The average since the August usage uptick is just 0.44 scans per day amongst the 2.7 million app users. And as for the advertising campaign driving use – it appears that it is reported outbreaks – such as the January 24 reporting of the Pullman quarantine breach – and not the advertising campaign beginning on December 13 that correlates best with increased usage.
What the above data suggest is that there is a very large number of individuals who have downloaded the app, but simply aren’t using it. While there are upticks when community outbreaks occur, the extent appears contingent upon the number of cases in each outbreak, and thus the perceptions of increased risk of transmission. The February outbreak has (so far) been much smaller and less widespread than the August one, so fewer individuals comparatively appear to be responding with increased app usage. And notably, usage appears to be falling off as the outbreak appears to be coming under control. Despite revitalised urging in the media to increase usage.
The low user numbers are further confirmed by the next graph, showing the number of active app users in a given day, as a proportion of total registered. On a given day, at best, during the August outbreak, fewer than 50 percent (0.5) of those with the app downloaded and registered scanned at least one QR code. In the recent outbreak, this has reached just 35% (0.35). Things are not much better for users engaging the Bluetooth feature, made available in December. Fewer than 50 percent of registered users have activated this feature. There could be some degree of substitution occurring, with users activating Bluetooth not scanning so frequently, which might explain the lower usage in the more recent outbreak. However, as we note in our analysis, Bluetooth registers contact with individuals whereas scanning traces locations. They serve different purposes in the contact tracing process, and are complements not substitutes. While media advertising stresses the need to do both, the message does not seem to be getting through to registered app users.
Furthermore, when we analyse usage per active user, we find even less reason to celebrate the recent uptick in the absolute number of scans in a day. The final graph shows the number of scans per registered app (as in graph 1) compared to the number of scans per active app per day and the number of scans per registered QR code. What this reveals is that of those users who do bother to scan, on average each only scans two codes in a day. This number has been unchanged since August. While the number of active users a day has fluctuated, the number of scans they take on average has not altered. That is, even those who do scan are not doing it much. While some people are recording lots of codes (such as the Pullman quarantine case in January), others do so very sporadically indeed, likely scanning one code on one day, and then not scanning at all for many days.
Finally, the above graph reveals that the vast majority of QR codes issued and displayed are generating very little scanning activity currently (3.44 per day per registered code at peak on February 20), compared to the August outbreak (7 per code per day on September 5, the day that scanning on public transport was signalled as mandatory).
In sum, therefore, there is little comfort to be had from the latest usage data. The pattern confirms our original assessment that the burden of QR code scanning was high and compliance would be difficult to establish and maintain. Bluetooth usage is less onerous, so is more likely to be adopted. But even here, usage by 50 percent of registered users is not overwhelming (27% of the “team of five million”). After one year of the epidemic, a serious cost-benefit assessment might be due. On the one hand, the cost of development of the app and its marketing lies in the past and the operational costs are probably low. The real cost, as we point out in our paper, is that use of the app will generate false putative contacts that will be costly to follow up and could lead to the distraction of resources from the testing and following up of contacts defined in another way. That is, the proper performance measure does not lie in simple usage statistics but in how the app impacts on the overall contact tracing process.
Dr Bronwyn Howell is a Senior Lecturer in Management at the Wellington School of Business and Government. Professor Petrus Potgieter is a Professor in the Department of Decision Sciences at the University of South Africa.