I am currently involved in a project to assess and improve the quality of (meta-)data on open data portals. My fine colleagues at Vienna University of Economy showcased their current state of affairs on PortalWatch and presented some yet to disclose statistics on data quality of checked portals. That day I learned that Socrata powered open data portals store all their data in a proprietory data vault as opposed to CKAN, where internal storage is optional. Storing data within one system has it’s advantages like uniform access for analysis. But in the long run it is not such a great idea.
TL;DR: Big Data Analytics while retaining privacy is solved using distributed and encrypted processing on the Enigma network1. The article describes fundamental principles of the involved cryptography, Enigmas architecture and use-cases in public administration.
Authors: Johann Höchtl (@myprivate42) & Bettina Rinnerbauer (@BRinnerbauer) Continue reading “Solving a hard problem – retaining privacy in Big Data Analytics”