Preserving the privacy of large data sets: Lessons learned from the Australian census

Preserving the privacy of large data sets is hard, as the Australian Bureau of Statistics found out. These are the big takeaways for the upcoming U.S. census and others dealing with large amounts of personal data.

Who needs hackers when the government puts sensitive information about every person in the country online and invites the internet to look at it? That's what happened last year in Australia, and it sends a warning message of what not to do during the upcoming U.S. Census 2020.

The Australian Bureau of Statistics published data from its last census online, but anonymized the data so poorly that it is vulnerable to a database reconstruction attack, researchers at Macquarie University in Sydney tell CSO. The database contains highly sensitive information about Australian residents, including address, age, ethnicity, salary, marital status, religious affiliation, number of children and so forth.

The Australian census data is not published in bulk as one database, but allows researchers, journalists and businesses to query the database. A hostile nation-state adversary would have the time, patience and resources to query the entire database over the course of several months and reconstruct it, rendering null the ABS's anonymization algorithm, which adds noise to the database in an attempt protect citizens from identification, the researchers warn.

Nor is simply unplugging the database from the internet a good option. The value of this data to journalists, academic researchers and enterprises in the energy, health and agricultural sectors makes it untenable to flick a switch and stop sharing.

Finding the right balance in the privacy-utility tradeoff is a hard, unsolved problem, Macquarie University professor of computing, and scientific director of the Macquarie Cyber Security Hub, Dali Kaafar tells CSO. "In a perfect world we would offer 100 percent utility and 100 percent privacy, but that's not possible."

To continue reading this article register now

FREE Download: Get the Spring 2019 digital issue of CSO magazine today!