NOTE- This template is part of the paid Data Quality Package- the most comprehensive online data quality management tool-Kit. This page provides a sample view of the usage of this template. For Buying this template and the entire Data Quality Package, please Click Here

Overall Usage Guide

Purpose of this work-tool

This work-tool provides the template, guidelines and the text examples to create a universal and central reference for all data related definitions and standards. For example having a universal standard data-model for 'customer entity' or a domain value standard set for locations etc...This reference document is the foundation to bring in the consistency on the way we store, interpret and define our data. An organization needs to provide the universal definition and standards to people building systems and processes.

When to use this tool?

There is no specific check-point to use this document. The DM standards for data entities is an evolving document is not accomplished in one single initiative for any medium to large organization. A system initiative, an audit comment, a data quality program, an ongoing initiative under Data Quality council or a process- re-engineering initiative can trigger the creation, addition or updation in this document.

Developing this document is a highly collaborative exercise. It will involve various system owners and business owners to agree to common definitions. These common definitions will result in the teams to change their procedures and mind-set.

How to use this tool?

The creation of Data Management Standards For Data Entities, should be owned by the data steward. The initiation of this document is generally a project in a Data Quality Program. Once this document is established, its change management should be managed by the Data Steward.

Linked Template:

• Data Quality Assurance and Controls Guidelines- This tool provides the template, guidelines and text examples to create a central reference document for DQ guidelines.
• Data-Group Master File- This is another central reference point for the logical data-groups within the organization environment. Each data-group is assigned a business owner and data custodian.

 
----------------------------------

Help Guide

Back-ground

State the back-ground on what has led to the creation of this document For example:

• Business and IT following different rules, leading to inconsistency on the way we manage and process our business.
• As we become customer-centric company, we realized that we need to have a standard and consistent ways in which we manage our customer information.
• As we move to integrate our business processes, we needed common bridge and language.

Purpose and Objective

State the purpose of this document. For example:

• To establish a comprehensive universal reference for data elements and data entities.
• To establish common standards for domains, Data Standards, Data Models and Data Quality Business Rules for all possible data elements, data entities and data groups.
• To help identify the standardization gaps in the existing state of data.
• To help identify the actions and specifications to fix the data quality issues.

After this high level purpose, one can go into the next level details. For example

• Domain and Data Standards will enable the data within the organization to be consistent in terms of value and structure.
• Data Model Standards will help us to ensure that we have standard set of attributes for any entity. For example- having a universal data model for the customer entity.
• Universal business rules will ensure that we have a common language on what makes data complete, consistent and valid. For example, when we say that for every customer record, the address data is considered complete as long as either home or office address is provided AND within the provided address, as long as Address Line 1, State, City and PIN are provided, it will be considered as complete.

Document Description

State the description of the document. It provides the description of various categories along with the description of each field. The reason this kind of description is not given in typical documents, as most of them are self explanatory. However, when it comes to the Data Quality Management Standards kind of document, many of the audience may not be able to understand it especially people in business community. (You can refer Data Domain and Standards in ExecutionMiH.com site..)

Some examples of what you can write out here (you can get all the material from Data Quality Assurance Chapter)-

Section – Domain and Data Standards:

This section provides the universal domains and Data Standards for various attributes and data elements within the organization. For example- When we say ‘Major-Age’ as a domain, we can state that the Major-Age should be:

• Within 18-99 years
• Should not be in fractions or decimals
• Should be only in term of years and not in Years + Months...

When and how to Use?

State the events and triggers, when this document will be used. For example-

• Investigating of Production issues.
• Designing ETL for Data Warehouse.
• Driving Data Integration initiatives.
• Doing Data Quality Assurance Assessment in an initiative.
• Doing Data Quality Assurance Assessment of the current state.

For each of the above event, one will need to state on how to use this document. For example-

• Investigating the production issue- If you have a problem with certain set of data, you can check if the input form is following the universal standards. It might be possible that the data out of the defined domain value is being entered. For example, the system may not have the check of allowing age between 18-99 to be entered in the system.
• Designing ETL for data warehouse- As you design your ETL for Data Warehouse, these standards will help you for dimensional modeling and also the kind of transformation you need to do on the extracted data (as all the production data may not be following these standards).

Universal Domain-Data Standards

One needs to refer Data Domain and Standards to understand the concept of Universal Domain and Data Standards. This section will be in a tabular for with each page assigned to one domain and Data Standard. Each page for a given domain and Data standard will have the following fields:

• Domain and Data Element Name
• Domain and Data Element Description
• Domain Value range 
• Domain Discrete Value List
• Domain Skip over Rules
• Text Column Rules
• Domain Character Patterns
• Constraints for the domain
• Null/Optional value range
• Data Element Data Type
• Format of the Data Element
• Character pattern +coding for the Data element
• Data Element Length

The description for the above fields in given in Data domain and Standards

Let us take an example for Office- Location Domain and Data Element

• Domain and Data Element Name: Office- City
• Domain and Data Element Description: This is the cities in which our offices could exist.
• Domain Value range – Nil
• Domain Discrete Value List- Provide the list of all cities where the offices are existing as of today.
• Domain Skip over Rules- NIL
• Text Column Rules- NIL
• Domain Character Patterns- NIL
• Constraints for the domain- NIL
• Null/Optional value range- Mandatory
• Data Element Data Type- Character
• Format of the Data Element- First Letter always Capital
• Character pattern +coding for the Data element- NIL
• Data Element Length- No constraint

Universal Data Models

One can refer Data Model Controls to understand on what all you need to define as part of universal data models. This could be little different from a typical data model definition, as we also recommend all the insert, update and delete rules to be included in the data model definition. You will ideally have one page (or more if needed) assigned for each universal data model. Following are the fields for the Data Model-

• Data Model Name
• Data Model Description
• Data Model ID
• Entity Attribute 1
  • Entity Attribute ID
  • Attribute Description
  • Domain and Data Element Code
  • Part of Primary Key
  • Foreign key to Entity
• Entity Attribute 2
  • Entity Attribute ID
  • Attribute description
  • Domain and Data Element Code
  • Part of Primary Key
  • Foreign key to Entity
• Primary key (Combination of a single or multiple attributes)
• Cardinality 1
  • Linked Entity
  • Cardinality Rule
  • Optional/Mandatory Rule
  • Referential Integrity Attributes
• Cardinality 2
  • Linked Entity
  • Cardinality Rule
  • Optional/Mandatory Rule
  • Referential Integrity Attributes
• Insert Rules
  • Rule 1
  • Rule 2
  • Rule 3
• Update Rules
  • Rule 1
  • Rule 2
  • Rule 3
• Delete Rules
  • Rule 1
  • Rule 2
  • Rule 3
• Entity Hierarchy
  • Child Entities
  • Parent Entities
  • Type of Hierarchy (Refer Business Hierarchy)

• Data Model Name- Customer-City
• Data Model Description- This is the city, to which a customer can belong. This includes the City for Customer Home Address, Office Address, Billing Address and Shipment Address.
• Data Model ID- LOC02CUST03CITY
• Entity Attribute 1
  • Entity Attribute name- Customer-City-Name
  • Entity Attribute Code- LOC02CUST03CITYNAME
  • Attribute Description- The name of the city
  • Domain and Data Element Code- DOMCUSTCITYNAME
  • Part of Primary Key- No
  • Foreign key to Entity- No
• Entity Attribute 2
  • Entity Attribute ID- Customer-City-Code
  • Attribute description- Standard and Unique Code for the City.
  • Domain and Data Element Code- DOMCUSTCITYCODE
  • Part of Primary Key- Yes
  • Foreign key to Entity- Customer-Home-Address, Customer-Office Address, Customer-Billing-Address, Customer-Shipping Address.
  • Primary key (Combination of a single or multiple attributes)- City-Code
• Cardinality 1
  • Linked Entity- Customer-Home Address
  • Cardinality Rule- One to Many
  • Optional/Mandatory Rule- Every Customer-Home address should have a Mandatory City Field, which should link up to the City-Code in customer-City Attribute.
  • Referential Integrity Attributes- City Code
• Cardinality 2
  • Linked Entity- Customer-State
  • Cardinality Rule- Many to One
  • Optional/Mandatory Rule- Every Customer-City should have a Mandatory Customer-State Field, which should link up to the customer-state-Code in Customer-State-Entity
  • Referential Integrity Attributes- Customer-State-code
• Insert Rules
  • Rule 1- A new City, should have a customer-state-code existing in the customer-state master table. If not it should not insert a new city.
  • Rule 2- A new City should be belonging to the valid value range defined in the universal domain.
• Update Rules
  • Rule 1- City Code cannot be updated
• Delete Rules
  • Rule 1- A customer-city cannot be deleted, if the city is existing in the customer-Address.
• Entity Hierarchy
  • Child Entities- Customer-Address
  • Parent Entities- Customer-State
  • Type of Hierarchy (Refer Business Hierarchy)- Simple Hierarchy.

TIP- It is possible that you system landscape will have a repository of all the data models- in that case, you may refer to that repository and add only the additional elements. You do not need to duplicate the work already done on documentation of  data-models. However, if the existing data model documentation is not universal(for example three different customer data-models existing in the functional specifications of three different systems), one will need to create a complete definition.              

Universal Business Rules

Refer Business Rules Controls in Data Quality Assurance to understand the concept. State the universal business Rule, which provides:

• Completion Criteria and Rules
• Consistency Rules
• Tolerance Ranges
• Override Rules
• Exception Handling Rules
• Validity Rules

• Data Entity Name
• Data Entity Code
  • Completion Criteria
  • Completion Criteria 1
  • Completion Criteria 2
  • Completion Criteria 3...
• Consistency Rules
  • Consistency Rule 1
  • Consistency Rule 2
  • Consistency Rule 3...
• Tolerance Ranges
• Override Rules
  • Override Rule 1
  • Override Rule 2
  • Override Rule 3...
• Exception Handling Rules
  • Exception handling Rule 1
  • Exception handling Rule 2
  • Exception Handling Rule 3...
• Validity Rules
  • Validity Rule 1
  • Validity Rule 2
  • Validity Rule 3...
Each business rule will be linked to a data entity and not to a data element. example of a business rule page
• Data Entity Name- Customer Master
• Data Entity Code- Cust01Mast
• Data Group Name- Customer Data Group (This includes the customer master and all the associated entities, like customer city, customer state, customer country, customer category...Customer Master typically has the core data on the customer and the other entities combine with it to provide a comprehensive customer data)
• Data Group Code- CUST-DG
• Completion Criteria
  • Both the home, office and Billing address is filled up along with the city. If billing address is not filled-up it should be tagged ‘same as office addresses. Or ‘same as home address’.
  • Customer First Name and Last Name should be available
  • At least one customer preference should be populated
• Consistency Rules
  • Every customer record should be having it congruent copy in the ERP, Field and Customer Service System.
  • The Customer Status should be same in all customer masters, by the end of the day (not necessarily online).
  • Tolerance Ranges-NIL
  • Override Rules-NIL
  • Exception Handling Rules- NIL
  • Validity Rules
  • Customer Data is valid if there are at least one order shipped and received by the customer  OR one unreturned mail has been posted.
  • Customer DOB is valid if there has been a date of birth proof address is given