Following the tracks of Big Data

Is it a coincidence that political drama series House of Cards has become so hugely successful? Netflix doesn’t think so, in fact, they engineered it so that it would be. By analyzing data they concluded that the same people that liked the original 1990 BBC miniseries were also fond of movies staring Kevin Spacey and/or directed by David Fincher. Therefore, a re-make with Spacey and Fincher seemed like a good idea (and, as it turned out, it was).

Netflix knows everything there is to know about their audience’s viewing habits and has used that data to promote relevant content for a long time. This was, however, the first time that a new show was created based on that extensive knowledge, and it might just be the beginning.

So what happens when Big Data is used not only to understand what consumers already like but also what they might like in the future? Is this new way of designing products and services some sort of democratization of the production process, or does it risk turning us in to mindless marionettes? Does artistic creativity become irrelevant when all output becomes tailored to the exact needs of the market?

Obviously, monitoring the audience too closely could result in white-washed and populist trivialities adapted to the passive wants of the great masses, but that’s not really a problem as long as it’s not self-fulfilling. Also, and perhaps paradoxically, Big Data probably requires even more creativity, imagination and brainpower than traditional market analysis. The sheer amount of information alone means endless ways of combining data points to find meaningful connections: Michael Karasick, a V.P. at IBM Research, has said that in two years there will be ten thousand billion gigabytes of information around the world in two years. That was two years ago.

Big Data is also already used to do actual good. The LAPD has created an algorithm that predicts crime, and information from millions of mobile phones is analyzed to prevent further outbreaks of Ebola in West Africa. Reading the future doesn’t just mean producing salable products, it can obviously also hinder epidemics and lower crime rates.

The danger (besides the whole Big Brother aspect) lies in relying too much on the data, not being able to see the forest because of all the trees. However, on a highly competitive market it is those that draw the most accurate conclusions that will emerge victorious, to the great benefit of everyone whether they be consumers or citizens.