Identification leaks from unstructured data

Andy Green points out that the need to anonymise data goes further than many data storers and controllers may realise

If you know someone's phone, social security or credit card number, you have a direct link to their identity. Hackers can use these identifiers, along with other personal details, to unlock data and steal identities and money. And the lines between the data that can be used to identify an individual and those that cannot are becoming blurred.

And then of course, specific pieces of data, which may appear anonymous, when taken together are just as effective at identifying a person.

An example is the trio of full birth date, postcode and gender. If a company publishes a dataset that had been "de-identified" by removing all the standard personally-identifiable information but retained those three data items, it can be relatively easy to find the name and address of the person behind those three pieces of data.

At a very basic level, the identity thief, like a detective, simply goes through lists such as electoral rolls looking for matches. In the UK this information is available for free. Electoral rolls contain name, address and birth date. Postcodes can be determined from an address, and once gender is added, it can be very easy to isolate one person.

Additional information or clues can be gathered from the web, especially social media websites, making it much easier to filter and narrow down a list to one candidate.

I have read that a researcher back in 2000 was able to identify 87 per cent of the people in the US using just those three pieces of data.

This should be even easier in the UK, as UK postcodes often cover a single street or an even smaller area.

When medical research on US patients is published today, so-called "Safe Harbour" de-identification rules mandate that no geographical unit smaller than a state can be included in the public data. Full dates (such as date of hospital admission or birth) cannot be included – they must have the year removed.

The US Federal Trade Commission, an influential regulatory agency on privacy matters, has issued new best practices on data de-identification, calling for all US companies to achieve a "reasonable level of confidence" that their public data cannot be linked to an individual.

The combination of birth date, postcode and gender would fail that test.

A larger problem is that consumers are sharing all kinds of information about themselves on web sites and social forums. Think of an online retailer collecting preference data about its customers, such as interests and hobbies, along with their location, and perhaps even income information.

If hackers pulled this "anonymous" data from a poorly permissioned file on a server, it could be used to mine various special interest sites, looking for names and geo data that match those interests.

The next step might be a phishing attack, with the hacker pretending to be the retailer.

For resellers that want their customers to stay ahead of stricter de-identification rules that will likely become law in the EU, it would be worth reviewing all non-personally identifiable data, wherever it sits on the filing system.

Andy Green is a technical content specialist at Varonis