The gathered crowd looked on with amazement and let out an audible gasp. The volunteer thought he had stumped the magician with his chosen number, 83. The magician had written 16 numbers in a four by four matrix — and 83 was nowhere in the mix.

SwissCognitiveBut then the magician broke into a wide grin. “Check this out. If you add up all the numbers across each row, they add up to 83. The numbers in each column? That’s right, 83. In fact, every combination adds up to your number. Amazing, right? It’s just like what happens when you get data right — it’s like magic!”

Data can do magic

The magician was pitching for Diwo , a new cognitive decision-making platform, at their booth at the Strata Data Conference held last week in New York. It was a fun way to introduce their new solution — and, perhaps unwittingly, it was also allegorical to the evolution of the big data market. While we have been talking about big data, data science and analytics for quite some time, there is an evolution of the market that was on full display at this year’s event. It could be seen in the several interwoven themes that permeated both the keynote stage and the exhibit floor. The overarching message: It’s time to get big data right.

Make Big Data Big

These themes all touched on the same broad idea that it is time to move beyond the exploration stage and apply big data, data science and analytics in real life and at scale so that the power of data can transform business models and the customer experience — and perhaps make it all feel a bit like magic. For most of its existence, big data has been a technically focused domain. While the business implications have (almost) always been clear, the focus of the market has predominately been on experimentation and figuring out how to solve the big, hairy technical problems incumbent with massive datasets. There were, of course, successful applications of big data that produced significant business results, but the primary driver of the market was technical development — not business application. This year, however, there was a visible change. First, there was significant airtime given to the societal impact of data and the important role that data scientists and practitioners must play as the industry continues to evolve. […]