First published in LinkedIn August 9, 2016

It’s not the quantity of data that’s important – it’s the way you use it that truly matters

– Matthew Sharp, Senior Hacker at LinkedIn

Weigh the pig

We all know (especially if you’ve ever listened to me ranting on about it) that you don’t Grow the Pig by weighing it.  And yet everyone’s talking about Big Data. The truth is you should only measure what you can use.

The first example of Big Data dates as far back as William the Conqueror.  His commission in 1086, twenty years after the invasion and defeat of the English in 1066, heralded the end of the Dark Ages. He commissioned the report to assess the extent of the land and resources owned in England so he could maximise the tax due. He needed the tax revenue to wage more war in northern France.

It took around a year to complete the data collection.  Some 900+ years later, a vault in Kew holds the resulting two volumes.  It really is an astonishing story. Commissioners were dispatched to create a comprehensive survey that would provide irrefutable evidence of every single landowner, villager, smallholder, tenant and slave, together with the extent of land owned, occupied and the livestock grazed upon it, as well as details of all buildings. Across the whole of England! It was said at the time that “there was no single…pig left out” (which seems apt here). Its definition as irrefutable fact led it to being named after the Christian tradition of Judgement Day, or Domesday, where every soul is laid bare for judgement.

Anyway, William the Conqueror knew how he was going to use his Big Data, to tax people. Ironically, he died before Domesday was completed. Do we know what Big Data is being collected for nowadays? Do you know why you would want to collect Big Data on your customers, processes, people, and what you are going to use it for?

Domesday today

At a societal level, here in the present day, governments and large corporations appear to be maintaining Domesday databases through Big Data. In fact, Big Data goes beyond databases.  Those governments and large corporations managed it through interconnected networks using the power of the internet and social media. And now, instead of manually collecting all our data by hand in large ledgers, they do it automatically in these vast technology networks that track and model everything from simulating the outcome of referendums to predicting what you will do tomorrow based on your credit score, geographical location and shoe size. Ok, so I made that last bit up, but you get my drift.

Liz Ryan (CEO of the Human Workplace) observes that “when in doubt, fearful humans put their trust in data! (Bad choice)”. Should we put our trust in Big Data, or is it all a bit too much like Big Brother, all a bit too scary? Surely it is us who provide the data that makes Big Data scary. It’s scary not because the machines we have created are getting smarter, but because we are now processing information on such a scale it is impossible to keep track of it.

Paralysis by analysis

And the truth is that data will only go so far. The measurement of demographics, credit cores, shoe sizes, whatever, is useless. What matters is the interpretation of that data into meaningful decisions.  Keep your data limited to what you can use to make effective decisions, and learn to use it, talk to your customers and staff and discuss it. Progress your collection to more sophisticated analytics when you need to, but avoid the pull to paralysis through information overload. Do not let the tail wag the dog, or the pig, for that matter.

(rant over)

Jeremy J Lewis

@growthepig

#growthepig is my philosophy of organisation design and development