Warning: Creating default object from empty value in /home/johnnz/public_html/wp-content/themes/simplicity/functions/admin-hooks.php on line 160

Waging War on Bad Data? Give Up. You’ll lose!

War on want. War on drugs. War on terror. Have these wars eliminated want, drugs and terror? No. Some would argue, quite the opposite.
There are two main reasons for this, both allied to the human condition. The first is based on a phenomenon first observed by Carl Jung, that made him coin the term, “what you resist persists”. He arrived at this conclusion after years of observing that, when people put a lot of energy into suppressing or resisting a negative behavior or trait that, instead of disappearing, it persists and, in many instances, grows worse.

The second reason is a little darker. Imagine that tomorrow all of the agencies that have been set up around the world to fight drugs and terror could, with the wave of a magic wand, bring an end to all drug wars and acts of terror and, as a result of doing this, they could dismantle themselves and free up all of their personnel to do something else with their lives. Imagine this and then tell me, do you think that they would wave the wand and make it happen?

THIS ARTICLE HAS BEEN UPDATED AND MOVED TO http://jo-international.com/waging-war-on-bad-data-give-up-youll-lose/


Working Together

If you would like to work with me and discover ways in which your enterprise can give up its war on bad data and instead achieve the high quality data that will enable it to thrive at far less stress and cost, then contact me now.

3 Responses to “Waging War on Bad Data? Give Up. You’ll lose!”

  1. Richard Ordowich June 3, 2014 1:55 pm #

    The problem with data, going back to philosophy, is that there is no such thing as good data. All data lies.

    Data is a representation of some reality created by a group of people, in a specific context at a specific point in time. When designed, data is a consensus reality that the people shared at that specific moment in time. Others who were not present have their own realties of the data and those who come later create their own realities. Just ask the question “what is a customer” or “what is a product” to the same people one month apart. The answers will differ.

    As a result data is temporal. It changes with every instance. How do you set quality metrics for something as elusive as this? Imagine products that change their shape, or color or dimensions and you can see the problem.

    The data design, if it works, is “good enough”. This is unfortunate but this is the real world. Data is a representation of something else. It is a human creation and subject to all the human foibles like, bias, experience, skill etc. Jung might have concluded that trying to design data is itself “resisting what will persist”, bad data. It is the nature of data.

    The best that can be accomplished is to realize data lies and is nothing more that an accepted reality. When designing data it is important to focus on the semantics, syntax and context of the data to limit the “realties” that may be interpreted. I call this Data Literacy.

    You can’t design good data. You can only limit how much people misinterpret data.

    • Darrell Comeau June 4, 2014 2:21 am #

      In response to Richard’s comment “..there is no such thing as good data. All data lies.”

      I say that data can be meaningful, with common understanding and maintain its meaning accurately across time.

      How? By accurately and identifying data entities and their relationship to other data entities at the time the data is captured.

      I agree that it is easy to create bad data. It happens when we try to shortcut the process of breaking down our data entities to their basic elements. It happens when we don’t build the proper relationships between those basic elements correctly. It happens when we confuse persons with the Roles they perform.

      As long as we identify elements and relationships correctly at the time of data capture, we will maintain consistency and validity in data.

    • John Owens June 4, 2014 8:05 am #

      Hi Richard

      Thank you for the comments. Your philosophical assertion that all data lies is definitely not correct. Data never lies. Data merely is what it is. People’s interpretations may vary enormously, yet the data remains constant. If data says that the dimensions of a box are 100cm x 200cm x 75cm and people choose to interpret that as something else, that has nothing to do with the data.

      Data quality has nothing to do with philosophy nor is it subjective. It is totally objective and fundamentally very simple. If a person’s name is Mary Smith and the name entered into an information is Mary Smith, then the data is correct. If the name is entered as Mray Simth, then the data is incorrect.

      Neither is data temporal. If the name is entered as Mary Smith and then Mary marries and becomes Mary Brown, the data that was originally entered is still correct as, at the time it was entered, that was Mary’s correct name. Good analysis would tell us that names are time dependant and good design would implement this.

      You very definitely can design good data, this is how data quality is built into every good information system. On the other hand, you cannot control how much people misinterpret it. The capacity of the data quality community to misinterpret even the simplest facets of this subject seem to know no limits.

      Again, thanks for you comments.

      Kind regards

Leave a Reply