The surge of interest in “big data” has swung our telescopes towards analytics … but we’ve yet to adjust the focus to see what really matters.

I’m going to try something a little unprecedented in this article. I’d like to persuade you that we should divert our collective enthusiasm for “Big Data”, and focus instead on the enormous value of carefully considered analytics. Big or little, either way is fine.

The important thing is the kinds of executive decision we can support with data, and how reliably we can support them.  Analytics should be about quality, not quantity, so experimentation (rather than specific technologies like Hadoop) should become the most valuable tool in our analytics armoury.  By separating causality from mere correlation, experimentation homes-in on the real reasons for our business problems.

Let’s take an example:

You’re a regional director for a supermarket chain.  You hypothesise that reducing the prices for fresh fruit might improve footfall in your stores, generating significant extra revenue. 

By chance, the CIO has just appointed a “Big Data” supremo.  You meet him, and he suggests combining analysis of CCTV footage from the fruit aisle with inflation data, and mixing-in customer comments from Twitter and Facebook.  It can all be accomplished (he says) using “Hadoop in the cloud”, in conjunction with several external market data feeds.

A week later you attend a university reunion, and meet an old friend who studied statistics.  When the subject of work comes up, you outline your problem, and you’re offered a very different approach …

“If you want to know whether something will work”, she says, “try it out and watch what happens.

“Reduce fresh fruit prices by 5%, 10% or 25% in a small number of stores, and let your customers know you’re doing it.  Compare the uplift in sales that week with the equivalent uplift in the same period last year.”

Amidst the hype about big data technologies, this is refreshing to hear.  It’s more likely to yield reliable results more quickly, and could cost a lot less too.

In fact retailers have always been savvy about pricing, and they’ve been keen fans of experimentation too.  But other sectors may have much more to learn from that statistician. 

Simple experimental approaches to problem-solving are always preferred.  “Big data” including image analysis, audio analysis, social media “scraping”, etc. all have their place.  But businesses should begin with rigorous analyses of existing structured data sources, and with the discipline of experimentation.  More complex, unstructured data sources (“Big Data”) will be most valuable when the suspect variables or the business outcomes can be monitored no other way.

So scientists, speak-up!  Your robust appraisals of business situations are finally coming into vogue.  You’re less likely than ever before to be rebuffed by impatient managers, and should exploit this long-awaited opportunity to the full.  For moral support in your campaign for real analytics, feel free to get in touch.

Add new comment

Comment editor

  • No HTML tags allowed.
  • Lines and paragraphs break automatically.
Blog moderation guidelines and term of use