Posted by: Clarinette | May 26, 2011

Mass indexing Google profiles

Reading today, thanks to @-pidder_ post on Twitter :

Student (@mrkoot) built a HUGE database of 35.000.000 Google Profiles. It contains 15.000.000 Gmail addresses. #privacy

Matthijs R. Koot explains in his blog how, in February 2011, without even violating Google’s terms of use, he could ‘create a database containing ALL ~35.000.000 Google Profiles without Google throttling, blocking, CAPTCHAing or otherwise make more difficult mass-downloading attempts. It took only 1 month to retrieve the data, convert it to SQL using spidermonkey and some custom Javascript code, and import it into a database’.

How easy it was to create a database containing:
– person names,
– aliases/nicknames,
– multiple past educations (institute, study, start/end date), – multiple past work experiences (employer, function, start/end date),
– links to Picasa photoalbums, …. —
and in ~15.000.000 cases, also the username and therefore address.

Additionally, this data base could use face recognition software and other data aggregators to create a more exhaustive profile.

He assures his activities were performed as part of his research on anonymity/privacy at the University of Amsterdam.

He believes that ‘the misuse value of personal data on social networks ought to be elicited before publishing it under a false perception of informed consent. One possible outcome‘.

One more example of mass aggregation and processing of data available and the risks for users as I was pointing out earlier this week with the case of data collected by TalkTalk customer service.

Data minimisation as well as data security should be the principle rule or we will see an rise of identity theft.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


%d bloggers like this: