Warning: Creating default object from empty value in /home/johnnz/public_html/wp-content/themes/simplicity/functions/admin-hooks.php on line 160

There Are No Such Things as Data Rules!

This may come as a surprise to many involved in Data Quality but, wait for it, there are no such things as data rules!

By looking at data in a standalone mode you can never deduce or define a rule that dictates that one entity should be related to another or that a value of an attribute of an entity has to be, for example, between 1 and 5000, or less than 250 characters in length, or boolean, or anything else.  In fact, there will be no rule to say that the attribute, or the entity to which it might belong, has even to exist!

Why No Data Rules?

It’s elementary, my Dear Watson!  Data, of itself, has no intrinsic value nor any required structure. The only reason that data exists in an enterprise is to support the Business Functions** of that enterprise.

I call this Data Quality Rule #1 and from it we can deduce two further key data rules:

  1. the only reason that any data entity, or any attribute of an entity, should exist would be to support a known, documented Business Function.
  2. the only reason that any attribute would have rules or constraints on its allowable values would also be to support a known, documented Business Function.

Both 1. and 2. above clearly show that the the allowable types, formats and value ranges of attributes are defined, not by “data rules”, but by Business Function Rules or, more simply, Business Rules!

Ergo, there are no such things as Data Rules! QED

Hair Splitting?

Am I really just splitting hairs and playing with semantics?  Actually, no.  The major Achilles Heel of the current approach to Data Quality is that far too many practitioners look at data in a standalone manner and try to deduce meaning from the data itself.  It does not have any! Data only has meaning and is only of value when viewed in the context of supporting the Business Functions of the enterprise.

If you want to understand the data you must first understand the Business Functions.

If you want to know what “rules” the data must obey, then you need to know the Business Function Rules.

What do you think?  Please feel free to leave a comment in the box below.  I would love to hear from you. Also feel free to Tweet this post. :-)

**Note: Business Functions are the core activities of an enterprise – they are NOT its departments.  Business Functions can be concerned with current operations, historical analysis or future strategy.  They are the core activities required to enable the enterprise to meet is objectives and continue in existence.

Tags:

12 Responses to “There Are No Such Things as Data Rules!”

  1. Robert Suzic April 29, 2013 10:47 am #

    How do you see on business rules originating from reference information models?
    A reference information model may contain business rules constraining data types of many organisations….

    • John Owens April 29, 2013 7:12 pm #

      Hello Robert

      In the normal course of events, the term ‘reference’ or ‘domain’ data would normally be used to a define or refer to set of data that would be used to define valid values to be used when creating instances of transactional or master data entities, e.g. a valid list of states or countries, etc.

      The business rule would not normally lie within the reference data but in the structure of the data within the enterprise. For example, the Logical Data Model (LDM) for the enterprise would show that every Location (e.g. the address of a customer) must be linked to a valid country and a valid state.

      The value from the country and state night come from and external source, but the rule is contained within the LDM of the enterprise in the structure of the entities concerned.

      I hope that this helps.

      Regards
      John

  2. John Owens October 28, 2010 10:39 pm #

    @Robert

    If a Business Function requires that the telephone number and e-mail address of (say) a customer needs to be collected as part of a transaction, then it must enforce all applicable data rules at the point of creation.

    An e-mail address without the @ symbol is not an e-mail address. A telephone number with letters instead of numbers is not a telephone number.

    But why do they need to be valid? Because they are going to be used during the execution of some Business Function. So, the need for them to have a particular format or values is defined by the needs of that Function that will use them.

    Data can only be said to be erroneous is it fails to meet the needs of the Business Function(s) that utilise it. “Valid” formats and values for data are always defined by those Business Functions that use the data.

    I hope that this helps.

    Regards
    John

  3. Robert October 19, 2010 8:53 pm #

    Dear John,

    Although I support your statement, it is in my optinion not always true.
    You are in general right with your statement that data does not have any meaning without a certain context.
    But data rules are important in many instances. What about an e-mail adresse of a customer that does not consists of an “@, , (at)” or a phone number with letters instead of numbers?
    Or is your optinion based on the fact, that every data recording front-end already implements all rules that where derived from a consistent business model? In this way, I would agree with you, but this is unfortunatly not the fact in reality.
    In data quality management you have to define rules or otherwise you will have many problem in everyday business applications and especially in the data integration process for BI applications.

    Regards

    Robert

  4. Adriana B. October 15, 2010 1:32 pm #

    Heh. I’m finding it funny that you had to write a post stating that “there are no such things as data rules!”. In many years working as a business analysis consultant, I never thought about the existence of “data rules”. I’m used to business rules statements that use connections between noun concepts (e.g., “a shipment must be approved by at least two employees”). Perhaps my surprise comes from the fact that I don’t know many “data quality” professionals myself :-).

    • John Owens October 15, 2010 10:21 pm #

      @Adriana

      Thank you for the feedback. If you follow the discussions on Twitter under #dataquality and #mdm (Master Data Management) and on blog posts on data quality, you will see a lot of reference to “data rules”, hence the reason for my article.

      John

  5. Gordon Hamilton October 14, 2010 6:06 am #

    Hi John,

    That’s a great way of “awakening” folks to the dichotomy between the business and IT. The home of data quality lies somewhere in between those two solitudes and thus sometimes gets orphaned by one or the other.

    Your provocative hook caught my attention because I have been re-reading John Morris’s “Practical Data Migration”. In his book John calls the central tool “Data Quality Rules” but his 4 Golden Rules of Data Migration make it obvious that he really means “Business Rules”.
    1. Data Migration is a business issue
    2. Business knows best and is the decision maker
    3. No organisation will pay for perfect data.
    4. If you can’t count it, it isn’t real.

    If a giant of data quality like John Morris can stress that business owns the Quality Rules but still neglect to name them Business Quality Rules, it probably means that you have an uphill battle changing the term from DQR to BQR.

    It might be interesting to see what definitions the different data quality gods and goddesses use such as English, Loshin, Maydanchik, McGilvray, Redman, Thomas, etc. For example, on a recent healthcare organization DW project we started with Jack Olson’s “Data Quality: The Accuracy Dimension” and used the term Business Rules.

    • John Owens October 14, 2010 8:51 am #

      Thanks, Gordon

      How did the DW project turn out?

      John

  6. Richard Ordowich October 14, 2010 1:35 am #

    I think this is an narrow perspective. Looking at the data to extract meaning without a known business fucntion is useful in two situations I know of. Data profiling of data, apart from the applications and business processes helps to identify patterns and relationships that are not evident when looking at data from the business process and application perspective. This may help identify flawed business processes that result in data anomalies.

    Data mining is another opportunity for examining data apart from business processes. Data mining algorithms help identify clusters and patterns of data that are not revealed using other techniques. As a result not all data exists to “support a known, documented Business Function”. Sometimes the business function is the “unknown” and looking at the data, exposes this.

    • John Owens October 14, 2010 8:48 am #

      Thanks for the feedback, Richard

      However, I think that both the examples that you give actually confirm that you must first know the Business Functions. When looking at data in isolation it is challenging, indeed can be fun, to try and spot patterns. But the question is, “are they relevant?”. “Relevant” means, “Are these patterns needed to support any Business Functions in our enterprise?”. If not, they are just patterns.

      Data anomalies are data structures or values that fail to support an enterprise Function. In order to identity the data as being anomalous you must first know when it ought to be. To know this you must know what Function it is required to support.

      Data mining is all about Function! It is trying to extract information from data in order to find out or achieve something. Unless you know what you are trying to do it is impossible to ascertain whether or not the data can be used to achieve it.

      It all boils down to the fact that data has no intrinsic value!

      Function puts it in context, gives it meaning.

      Regards
      John

  7. Gombos Pál October 13, 2010 11:53 pm #

    Dear John!

    Excellent insight!!! I share the same view and I am really happy to read it from you as well!!

    Regards

    Pal

    • John Owens October 14, 2010 8:51 am #

      Thanks for the feedback, Pal. Good to hear from you.

Leave a Reply