Warning: Creating default object from empty value in /home/johnnz/public_html/wp-content/themes/simplicity/functions/admin-hooks.php on line 160

Improve Data Quality or Raise the Titanic?

In a recent post Dylan Jones of DataQualityPro, commented on a job description for a Data Quality Director that cited 17 key areas of responsibility across 4 functional areas and not one mention of the enormous support team that would be required to help this leader, no matter how energetic he/she might be, to complete these tasks.  Without this team, what this person would require, as Dylan commented, would be “Powers of Invincibility”.

I fear that the current approach to data quality will call for, either ever larger teams, or individuals with superhuman powers.  After all, what we are asking them to do is the equivalent of raising the Titanic!

In far too many enterprises, their data is indeed the equivalent of a massive wreck laying on the bottom of the ocean. How did it sink? Everybody is too busy trying to access the sunken data to even think about that.

Because nobody has bothered finding out why the data “sank”, all data newly created in the enterprise has the same flaws as the original and it too sinks to the bottom to join what has gone before.

Amazingly, instead of eliminating these flaws, the majority of data quality effort in enterprises, and the DQ industry as a whole, goes into developing ever more sophisticated dredging methods.  Many of the developers of these techniques are so pleased with their creations and have marketed them so well, that many enterprises now believe that a) it is not possible to create data without flaws that will keep it “afloat” or b) that it is cheaper to let it sink and then use the “amazing” new tools to recover it, than it is to remove the design flaws.

As long as enterprises perceive it to be impossible to keep their data ship afloat, then any person taking on the roll of Data Quality Director will either have to keep on developing or purchasing ever more sophisticated (and expensive) dredging tools or possess superhuman powers sufficient to raise the Titanic!


2 Responses to “Improve Data Quality or Raise the Titanic?”

  1. Tom July 5, 2010 9:48 pm #

    Same thing with documentation. Companies let the task of writing comprehensive developer, project and end-user documentation sink to the bottom of the priority list, from where it ends up to be non-existent. Then some fine day call an underpaid intern to “write documentation” since: “how hard can that be?” Of course this job cannot be done. Which eventually results in either a pile of non-sense, or nothing at all.

  2. Wes Baird July 3, 2010 1:31 am #


    I think that businesses allow data quality to sink during the capture of it because the departments in a company create data that’s core to them as a number I priority and then data that’s important to the company as a whole as a number VII priority,

    To explain further, a department (from the head on down) gets asked to do a job/task to the best of their ability. This results in a solid mission statement like: ‘produce me xxx widgets a month’. I think that the ask should be ‘produce me xxx widgets a month where you use the same supplier list as accounting’.

    The benefit to the company is that you can now do cross discipline analysis, you don’t maintain two lists of vendors and all of the effort that is required to a) maintain the two lists in two departments and then b) create a cross-reference system between the two. Instead, the widget department integrates with the accounting department and does not have to maintain their own supplier (business associate) list. They rely on the work that the other department does and they keep their costs down, to boot. More importantly, each department is owning up as a data owner to the C-level.

    So how do we get there?

    Here’s an idea; assign corporate responsibility (again at the C-level) of data types to individual department. Align bonuses with departmental re-use of data. Use accounting’s vendor list, get an extra $50/month in your pocket. Don’t use it, get nothing. Nothing motivates people like $$$ in their jeans.

    Where to get the extra $$$ from? I think that’s easy. From the system support/maintenance that you don’t have to do of your duplicated vendor list. And for that, you need metrics or ROI numbers you can trust…


Leave a Reply