OVERALL USAGE GUIDE

Purpose and Objective

Data Quality Assurance Management Guidelines, is the central reference point to understand the Data Quality Assurance mechanisms and controls. These are the mechanisms,  one needs to incorporate, while designing the IT systems and business processes. This document forms and annexure to the Data Quality Policy.

Who uses it?

The DQ Assurance Guidelines documents is owned by the enterprise data steward. Ideally it should be only updated by the Data Steward or a designate. The users of this document are the analysts and designers of the systems and process initiatives. The document is reviewed by the CIO, CFO and other executive stakeholders.

When is it used?

The document has not specific check-point or phased based usage. It is a central reference (like Data-Group Master, Data Management Standards for Data Entities, Data Quality Policy.

Linked Practice Tools

• Data Quality Policy-  DQ Assurance Guidelines is an annexure to Data Quality Policy.
• Data Management Standards for Data Entities- This document contains the repository of the domain and Data standards, Data Model Standards and Business Rule standards for data entities and data elements.

----------------------------------
Help Guide on DQ Assurance Guidelines Document

• Back-ground: State the back-ground on what has led to the creation of this document. For example:
  • Business and IT following different control guidelines, leading to inconsistency on the way we manage and process our systems.
  • We needed to enhance the efficiency of our system and IT development.
  • We have been having production issues, most of which were avoidable..
  • We want to be pro-active in creating a capability to enable us to integrate better in future etc...
• Purpose and Objective: State the purpose of this document. For example:
  • To establish a comprehensive reference for data quality assurance controls to be placed in our systems and processes.
  • To help identify the standardization gaps in the existing state of data.
  • To help identify the actions and specifications to fix the data quality issues.

After this high level purpose, one can go into the next level details. For example

• Interface and Data Exchange control guidelines will enable an integral and reliable data transmission across systems and process.
• Input controls will help us to achieve the data quality at source.
• Batch-Process Controls will ensure that the massive data crunching happening in the back-end is being done in a reliable fashion
• Business Partner Interface Controls will ensure that we are maintaining a good data quality in our data exchange with our business partner..

• Document Description: State the description of the document, in terms of what it contains and how is it organized. The reason this kind of description is not given in typical documents, as most of them are self explanatory. However, when it comes to the Data Quality Management Control Guidelines kind of document, many of the audience may not be able to understand it especially people in business community. (You can refer Data Quality Assurance chapter in ExecutionMiH.com encyclopedia to understand the subject matter)

Some examples of what you can write out here -

• Section – Interface Controls

This section will provide you the list of control guidelines, which one needs to apply while designing any interface across the systems. This interface could be point to point interface or through a middle ware, messaging technology of loose coupling. This section will be having interface control, its description, the situations in which this control needs to be used.

• Section- Input Controls

This section will provide the list of control guidelines, which one needs, when one is creating an input form on an IT system. This section will be having input control, its description, the situations in which this control needs to be used etc...

• When and how to Use?: State the events and triggers, when this document will be used. For example-
  • Investigating of Production issues.
  • Designing ETL for Data Warehouse.
  • Driving Data Integration initiatives.
  • Doing Data Quality Assurance Assessment in an initiative.
  • Doing Data Quality Assurance Assessment of the current state.

For each of the above event, one will need to state on how to use this document. For example-

Investigating the production issue- If you have a problem with certain set of data, you can check if the input form is following the input controls. It might be possible that location data is not correct because one is not providing the drop down selection for state and city..

Interface Controls

One needs to refer Interface Controls to understand the concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Interface Control Name.
• Interface Control Code
• Interface Control Description
• Interface Control example
• Critical/Optional
• Exceptions

example is as follows

• Interface Control Name- Duplicate File Check.
• Interface Control Code- ICDUP01
• Interface Control Description-

The receiving system should be able to detect, if the interface file has been already up-loaded.

If a file has been already uploaded and populated the database of the destination system, it is possible that some further online activity has happened on that data. A duplicate file upload may overwrite the changes in the database since last file was uploaded.

An interface file should have an ID number placed in the header OR footer of the file. The destination system should maintain a table where it maintains the statuses of the files uploaded. Before uploading a file, it should check if it has not been already uploaded.

There are many other ways of checking, if the file has been already uploaded. This can be done by checking on the date and time stamp of destination tables, OR other business rules, which can validate if a file has not already been uploaded.

• Interface Control example-

The customer Master File, which is send from ERP to Field Services System, has a duplicate file check control. Each file has an ID, which is included in the header piece of the file. The sending and receiving system maintain tables which track the file on the basis of their IDs and if the received file in the receiving system, as an existing ID, it will reject it.

• Critical/Optional- Critical
• Exceptions- NIL

Batch-Process Control Guidelines

One needs to refer Batch-Process Controls to understand the concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Batch-Process Control Name.
• Batch-Process Control Code
• Batch-Process Control Description
• Batch-Process Control example
• Critical/Optional
• Exceptions

Here is an example to fill this section

• Batch-Process Control Name- Time Cut-off Control
• Batch-Process Control Code- BPTC01
• Batch-Process Control Description-

These controls assure that the batches do not start OR end before OR after the prescribed time limit. This is to ensure that the batch-processing window is synched-up with other processing.

• Batch-Process Control example: Credit Card billing cycle batch to start by 9.00 P.M. (after most of the transactions are done), post which the card statement generation process to begin by 2.00 A.M. so that the spool files can be sent to the card statement printing batch by 5.00 so that the collation and stamping etc. is done by 8.00 A.M. to be picked up by the postage and shipping company.

• Critical/Optional- Critical
• Exceptions- NIL

Input Control Guidelines

One needs to refer Input Controls to understand the concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Input Control Name.
• Input Control Code
• Input Control Description
• Input Control example
• Critical/Optional
• Exceptions

Here is an example to fill this section

• Input Control Name- Master Data Drop Down
• Input Control Code- IFDD01
• Input Control Description-

Provide the drop down options for master tables. Instead of asking the operators to enter, provide a drop down of the locations, states, existing customer, job codes and product codes etc.. Alternatively ensure that the entry is checked against the master tables. Many a times the location codes etc. are not important to processing, but they will be important for analytics (For example location-wise customer preferences).

Try to have the codes or descriptions in the drop down from a standard as much as possible. For example, I would use the locations list from US postal address system standard. It makes your coding as well as the description easy to create and handle.

• Input Control example

When a user is selecting the city of the customer, one can provide the drop down of country, followed by drop down of the states within the country, followed by the drop down of the city within the state. Another way to do it is to have context sensitive drop down. For example, if an operator enters ‘New’ within a state, it should automatically provide New York and New Jersey as the options to select.

• Critical/Optional- Critical
• Exceptions- NIL- When the location on the data entry form does not exist in the system master, mention others along with a free form field to enter the location.

Domain and Data Standards Controls

One needs to refer Data Domain and Standards Controls to understand the concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Domain and Data Standards Control Name.
• Domain and Data Standards Control Code
• Domain and Data Standards Control Description
• Domain and Data Standards Control example
• Critical/Optional
• Exceptions

Here is an example to fill a section

• Domain and Data Standards Control Name: Range Value Control.
• Domain and Data Standards Control Code: DDRV01
• Domain and Data Standards Control Description:

When one is defining the attribute or a field in our systems and processes, one needs to check if the valid values of that attribute should fall within a certain range of values. One can define multiple ranges if needed.

• Domain and Data Standards Control example: Shipping_order_size cannot be less than 300 and more than 2000 pieces

• Critical/Optional: This is a critical control
• Exceptions: Case to case basis. No general rule.

Data-Model Controls

One needs to refer Data Model Controls to understand the concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Data Model Control Name
• Data Model Control Code
• Data Model Control Description
• Data Model Control example
• Critical/Optional
• Exceptions

Here is an example:

• Data Model Control Name: Referential Integrity
• Data Model Control Code: DMRI01
• Data Model Control Description:

Any two tables relate with each other through the foreign keys. This ensures that the link exists across the two tables. As we design out systems, one needs to ensure that the referential integrity is maintained depending upon the mandatory or optional linkage.

• Data Model Control example:

One cannot have the order details added in the order table, if the supplier ID (Foreign key for the order table) mentioned in the records is not existing in the supplier master table (where supplier ID is the primary key)

• Critical/Optional: Critical
• Exceptions: Nil

Business Process Controls

One needs to refer Business Process Controls to understand high level concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Business Process Control Name
• Business Process Control Code
• Business Process Control Description
• Business Process Control example
• Critical/Optional
• Exceptions

Here is an example:

• Business Process Control Name: Data Entry Form Data standards
• Business Process Control Code: BPDE01
• Business Process Control Description:

As we design data entry forms or forms to be filled by stakeholders like customers, we should adhere to the universally defined data standards.

• Business Process Control example:

If the data standard for PIN-Code is ZIP+4+2, the data entry form should have the same format, with separators.

• Critical/Optional: Critical
• Exceptions: Nil

Business Partner Controls

One needs to refer Business Partner Controls to understand high level concept and subject matter. This section will be in a tabular form with each page assigned to one control. Each page for a given control will have the following fields:

• Business Partner Interface Control Name
• Business Partner Interface Control Code
• Business Partner Interface Control Description
• Business Partner Interface Control  example
• Critical/Optional
• Exceptions

Here is an example:

• Business Partner Interface  Control Name: Master Table Synch
• Business Partner Interface  Control Code: BPIMT01
• Business Partner Interface  Control Description:

If our business partner needs to use our master tables in an offline mode, there should be tight controls to ensure that there is a congruence between the contents.

• Business Partner Interface  Control example:

The sales distribution partner will need the product pricing table to ensure the right invoicing and publishing product catalogues. We need to ensure that the product pricing table with our business partner is updated as soon as a change in the product price is made.

• Critical/Optional: Critical
• Exceptions: Nil