Big data is like therapy, the answers are in there you just need help to get them out. You need to like back on the couch and regress. Blame your mother, your home town or the conditions you had to endure. Let it all out.
The main issue with any open data project is not that the data does not exist. We’ve got loads of it. It’s everywhere and every day we produce more and more of the stuff. The problem isn’t either with the tools that can extract the data and make it meaningful. There is a growing array of very effective pieces of software and hardware that will do the job for you – if you have the money.
No the main issue is that people are not comfortable in giving the data away. People see the data that is produced within their service area as their data. They’ve bought it, they’ve paid for it and nobody else is going to get their hands on it. They are the appointed guardians of the data and even if they wanted to let you see it they wouldn’t be allowed for data protection and security reasons. Data is just too risky and needs to be protected at all costs.
Yet life is changing. The amount of data is growing exponentially as is the amount we are giving away. It is becoming increasingly easy to find anything out about anyone. The rise of social media means that more and more of us is out there somewhere. An increasing part of our private lives is lived out in public. Most people have little compunction when it comes to giving knowledge about themselves away. The lure of free email or free access to the internet is a cheap price indeed.
Yet in the public sector all of the data already belongs to the public. Doesn’t it? It is their money that has produced it and it is data about them. So shouldn’t we be thinking of getting as much of it out there as we can. Public data should be in the public arena for the benefit of the public. To do this we need to turn things on their head. Our default position should be that all data is created as if it was (and will be made) public unless there is an absolute reason why this cannot be so and that all data is published in a format which is easily accessible.
Is it up to us as servants of the public to decide whether or not the information contained is of value especially when there are those who can see patterns and forms that we can’t? Taking into account all of the genuine safeguarding issues, data should be free, liberated to release and realise its inherent value. We just need to make people want to let go. It will be therapeutic.
Now just lie back, make yourself comfortable, close your eyes and listen to my voice as I count slowly down from ten.