Establishing 'Making it Happen' as a 'Formal & Predictable' Discipline

Data Quality Organization Roles

ENCYCLOPEDIA→ Enterprise Intelligence → - Data Quality → - Data Quality Program →

Data Quality Control Procedures

These are the procedures to implement data quality assurance, data monitoring and data correction. If Data Quality Policy and Data Quality Organization are the enablers of Data Quality, these procedures drive its success.

Data Quality Assurance Procedures

Data Quality Assurance methods are listed in a separate chapter. This page provides the way one can ensure that they are implemented. It is not possible to implement all the methods in one go. Typically one starts doing the same in all the new projects. For some very critical areas, as identified in the data quality approach and plan, one may go for changing the existing set-up to implant these controls. One can implement the methods by following ways:

Include it in the project management handbook (s)

Any enterprise has single OR multiple project management procedures being used by diverse project managers for managing projects ranging from IT to business operations to sales. The data quality policy should prescribe

Data quality assurance methods as part of the appendix.
Data quality assurance checklist as part of the appendix.
‘Confirming Adherence’ to the data quality checklist should be made as part of the standard ‘work break down structure’ for any project in the end of analysis/design and project completion phase.

For example

Say, a company is implementing a cash management system, which is sitting in front of a core production system. During business analysis phase, when the analysts refer the DQ assurance checklist, they will ensure that all the business needs related to DQ (i.e., data standards, front end capture validations..) are also included.

Similarly, when they are designing the solution, they will again ensure that the technology related data quality assurance methods (batch processing, database check, interface checks etc..) are included.

Refer Data Quality Assurance Checklist in the Execution-MiHPractice Tools area of Data Quality Section..

Make Data Quality Assurance methods as part of the business process re-engineering team

A typical enterprise has business process re-engineering OR change management teams. One can make sure that these quality assurance methods are part of their methodologies.

DQ Program Deliverables- Data Quality Monitoring Procedures

Data Quality Monitoring methods are listed in a separate chapter. This part covers on how to implement the same.

A Data Monitoring request

This is equivalent to a business need. It is generated by the business owner/technology owner, which includes the following:

Data to be monitored.
The frequency with which it is to be monitored (Daily, alternate days, weekly…).
The online (while the processing is going on) OR offline (After the processing is done).
The quality benchmarks and ranges.
Alerts, which need to be generated. (for example - red alert , if the currency value set-up data is not completely populated).
Distribution list of alerts.
Format of the monitoring report.
Actions to be taken.

Data monitoring solution design

This is the solution designed by the technology team. This includes the following options:

Include the monitoring constructs in the batch process.
Acquire/implement a data quality monitoring tool.
Build queries to be run by the DCO.
Create data monitoring reports as part of schedule production reports.
Create data monitoring reports as part of ad-hoc reports.

Data Quality Monitoring Review

This is done with the business owner and technology owner. The deviations are taken through a root-cause analysis, which feed the Data Mapping & Assessment documentation.

PLEASE REFER Execution-MiHPractice Tool Data Monitoring Request

DQ Program Deliverables- Data Correction Procedure

This is one of the biggest headaches of IT, and We have seen many hours in the boardrooms being spent by CIO in trying to explain on why Data Fix is taking so much time. Following is the ideal set of procedures recommended for data fix. This subject will be expanded to a separate chapter at a later stage. For example of Data Corrections, you can refer to Customer Data Correcting Techniques

Data correction should ideally be done from the front end., but sometimes it is not possible (like vouchers, which are already posted with wrong reference numbers..) OR when it is too voluminous (over one hundred thousand customer are tagged with a wrong product..)

In case there is a need to do a data correction through a back-end, high level imperatives are given here:

Data Fix Request has to be formal

The data quality fix request can be generated by any party, but has to be signed off by no one less than a CIO, head of the function and if possible, head of internal control as well. It should list down:

The data, which needs to be fixed.
Why the fix cannot be done from the front end?
What is the urgency of the fix and what is the impact?

Technology should list out the impact of the fix. examples are:

System down time.
Spoiling of audit trail. (the changed records may will not be reflected in the audit trail maintained in the system).
The duplicate communication going to the customers. (When you correct a billing record, a database trigger automatically populates the billing communication table).

NEVER run the queries directly on to the production data-base

The data correction instructions should be carefully analyzed. The set of instructions should be created in form of a correction procedure, which should include the core correction queries, checks to be performed before & after the corrections, alerts in case something is wrong etc..

Always test the correction queries on the test environment before running them on production environment

Its commonsensical to test the correction procedure in test environment to make sure that it works.