Warning: Creating default object from empty value in /home/johnnz/public_html/wp-content/themes/simplicity/functions/admin-hooks.php on line 160

Can Pooping on Your Own Doorstep Improve Data Quality?

Quality Begins

There are three things that have to happen in an enterprise if Quality is to exist there. The first is that the management and staff come to know and appreciate what Quality is, the second is that they are committed to bringing it about in their enterprise, the third, and most important thing, is that they take the necessary actions to bring Quality into being.

Traditionally, what happens after the benefits of Quality have come to be appreciated, is that staff in each  area of the enterprise then set about trying to improve the Quality for all of the tasks and products for which they have responsibility.  The approach is used for both product quality and data quality.

Quality Stops!

The above approach will definitely improve the quality of the Business Functions and Processes over which staff have total control.  However, in areas where they have only partial control, the level of quality will very quickly reach a plateau due to receiving input of  low quality raw materials or data.

In the area of Data Quality the standard approach to this is to use software products to find and repair data errors.  No matter how good these products are, this approach actually creates what is a never ending process of first creating and then finding and fixing data errors.

This ‘Quality Control’ approach to Data Quality has spawned a global ‘dirty data industry’ that actually thrives on the fact that, as data volumes grow,  more and more data errors have to found and corrected.  Its protagonists are proud of the fact that they are so good at finding errors – and do not see the irony of this in the field of Quality.  They say that the concept of ‘zero defects’ might be feasible in all other areas of industry but that it is not achievable with data.

The Use of Lateral Thinking

In the late 1960s Edward De Bono introduced the concept of lateral thinking.  In his book ‘The Use of Lateral Thinking’ he explained how problems that seem intractable when people use straight-line thinking, become relatively easy to solve when they are looked at from a completely different perspective.

At this time, the problem of river pollution was plaguing all industrialised nations.  Local and national governments were trying to fight it, with minimal success. In the UK many large industries polluted rivers at will. The fines that local government had the power to levy against these huge corporations proved to be no deterrent at all.

In one of his articles, Edward De Bono, demonstrated how a simple, lateral shift in thinking could solve this problem at a stroke, without the need for any policing or punitive fines.

Traditional Layout

The traditional layout of manufacturing plants along these rivers was as shown below. The water needed for processing was taken in upstream from the manufacturing plant and, after being used, was discharged downstream from the plant.

Conventional water flow in industry

In most cases this water was completely polluted and unfit for use by people or industry downstream from the plant.

Lateral Thinking Layout

So how can you, without spending millions of dollars to totally rebuild or reposition this plant, get the owners to reduce the pollution in the river so that it is suitable for everyone to use?

Lateral thinking water flow in industry

The answer is amazingly simple. Instead of allowing the plant to take its water from upstream and discharge it downstream, you simply reverse the flow. As the plant now takes its water from downstream of its own discharge, it is forced to make every effort to ensure that its discharge is pollution free, otherwise it ends up polluting its own processes.

Data Quality?

Very interesting, but what does this have to do with Data Quality? Exactly the same type of thinking can be applied.

What you need to do is to think of ways in which all data input by anybody will end up effecting them in some way or other. It could be that they themselves are going to be using the data. When this is the case they would very obviously want to make sure that they enter high-quality data.

The real challenge comes when all of the data that is being entered is going to be used by someone else. What is the incentive to get it right?

This is when you have to put on your ‘Lateral Thinking Hat’ and ask, ‘How can we make all low quality data entry the equivalent of having the person responsible poo on the own doorstep?’

One approach I came across that worked very well was where data entry staff were paid a low basic wage, plus a high bonus based on volume of data entered.  Each item of data entered added to the bonus, each identified error subtracted from it. This encouraged the data entry staff to have high productivity and low error rate.  It was up to them to decide how much they earned.

What approaches would you suggest? Remember, in order to get the most innovative results, you will need to abandon ‘straight-line thinking’ and take a completely sideways or Lateral step. Viewing the problem from an entirely new perspective will, if you keep an open mind, enable you to come up with ideas that could be termed ‘crazy’, ‘off the wall’, ‘bizarre’ and, if you persist, ‘genius’.

Share the Love

If  you enjoyed this post, then please share this page with a friend or colleague by clicking on one of the Social Media buttons below. Thank you.


4 Responses to “Can Pooping on Your Own Doorstep Improve Data Quality?”

  1. Jim Hems March 4, 2013 9:16 am #

    Just a couple of points to make from me having worked in the DQ arena for 14 years now.

    1. With regards to enhancing data quality at point of capture, there are a few things that can be done. Incentivising staff based on number of items captured is fine, but then you must ensure that information being entered is accurate. Certain tools and items of software can help accomplish this, as well as some basic rules for the formatting of supplied information (such as email addresses, telephone numbers etc.). A second option might be to seed the data going into a contact centre and introducing a bonus scheme for data entry staff based on picking up data innacuracies at point of entry, or on measuring the quality of these known seed contact details vs. what is actually entered by the user.

    2. I kind of agree with the zero error rate in data defects, although to actually get to zero would be very tricky. Even in manufacturing, I am not convinced that all sectors can boast zero defects? I’m thinking of various items that I have purchased which either broke very quickly or simply didnt work to start with, and this is presumably after some form of QA process! That said, regardless of how well you capture the data or information, the very next day it could become incorrect as people move house, change their mobile number, switch email providers, get married etc. So maybe you can have guaranteed zero defects for a day? Not sure I would want that as a guarantee on a new kettle!

    3. I would disagree about it being easier to fix problems than to prevent them – fixing issues can rely on infilling missing data from third parties and this information is never 100% accurate or comprehensive. It’s also normally more expensive – to add an email address to a customers could cost anywhere between 20-75p depending on where you go, but to just ask for it at time of capture is effectively free.

    • John Owens March 4, 2013 7:58 pm #

      Thanks for the input, Jim.

      Point 2.

      Zero defects is definitely achievable and many enterprises around the world do it on a daily basis, because that is their definition of Quality: Zero Defects.

      The items you purchased, which either broke or did not work, where manufactured by enterprises where the definition of Quality (presuming they have one) is something else, for example that it must cost no more than $30, or that it should last for no less than three months but no more than six. When you purchased the item, you also had your definition of quality. This may have been based on price, rather than the life expectancy of the product.

      Point 3.

      I agree that he is an illusion that fixing errors is cheaper than preventing them. Even when you can put a cost in dollars and cents on the act of rectifying the defect, this is nothing compared to the losses that the defect may have caused during the time that it existed, in terms of bad customer service or lost opportunities.


  2. Richard Ordowich February 27, 2013 10:39 am #

    One of the reasons it’s difficult to get people to get concerned about data quality is the fact that it is easier to just to fix the data rather than spending time preventing errors from occurring in the first place. Just go in and change the value. Besides, most organizations reward people for solving problems not preventing them from occurring. Prevention goes unrewarded.

    Organizations need a new reward system to motivate their employees to prevent errors. Rewards should be based on innovations on how to prevent errors from occurring. From designers, developers, operations, manufacturing, customer service, finance and sales, each department should be challenged and rewarded to come up with preventive quality solutions (not just data quality). Only then will data quality be successful.

    • John Owens February 27, 2013 11:49 pm #

      Hi Richard

      Thank you for this. You’re right. Total data quality is all about creating zero data defects, and rewarding people for doing so.

      Although zero defects is accepted around the globe my (nearly) all manufacturing industries, people working with data are still making the same lame excuses about zero defects ‘not being possible here’ as manufacturing was 20 years ago.

      The reality is that, ‘It’s not possible with data’ actually means, ‘I don’t know how!’ My advice to these people is to look to those who have achieved it and learn from them. Their problems are not new or unique. They have already been solved by many enterprises out there. Go out an learn.

      Thanks once again for your input.


Leave a Reply