Information quality is the level of accuracy, completeness, auditability and consistency of data (you may refer what is data quality for some more perspective). However, from the data warehouse perspective it has more connotations than the traditional data quality. Refer Assuring data-warehouse quality
Your BI platform Supports queries and reports with few data errors
A good data-warehouse is considered as the system of record, with any quality issues documented and formally accepted. Ideally Data-Warehouse testing and quality assurance should throw-up any issues. However, it’s not possible to check every permutation & combination. At the same time, you cannot rely upon end-user testing to trap all data quality errors. One will need to do the data monitoring scripts to identify the issues. As you assess performance on this factor, one needs to look at the following fine points, where the performance shortfall is seen with a more positive view:
- Is the error consistent? If the error is following pattern in terms of occurrence and level of deviation, it is possible to 'counter-act' your calculations and come out with nearly correct answer.
- Is the error predictable?: If the error can be predicted in terms of its occurrence, one can know on which report, or data output it will happen and you can place appropriate annotation and level of confidence
- Is the error within the acceptable range? If the error is acceptable, it does not/minimally impact the objective behind the information generated.
- Is the error-fix part of the plan?: If the error is trapped, its root cause is done and the fix is under progress, the error will soon be eliminated
- What is the criticality of the error and what is the frequency of occurrence?: If the error is coming in the data, which is used infrequently and is not related to critical information need, it has lesser impact.
Your BI platform supports the level of data correctness needed for its intended purposes:
The data quality assessment has to be calibrated vis-à-vis the expected level of the quality. Therefore, the DW performance on data quality should be a combination of absolute errors and expected level of errors. It’s not only a question of business specifying the expected levels of data quality. As the program manager of BI initiative, you may have to facilitate a discussion on the cost-benefit of the expected data quality in BI. Business would typically look for perfect data quality. The cost of ensuring data quality includes the fixing of data in the source system as well as using transformation for the same. Business will need to know the cost-benefit equation, before they can decide on the level of data/information quality acceptable to them.
TIP- One should not stop the implementation of DW, if there are persistent errors in spite of all the efforts on source systems and though ETL. One needs to be able to articulate the impact of these errors and the area where they will manifest themselves. The other part is that you simply do not include those attributes or entities within your data warehouse scope. The issue in that approach is that if you do not include the 'erroneous' fields, you may need to spend some effort to include the same in ETL and Data Warehouse/OLAP model at later stage. Which option to select? is a question of situational judgment.
The data values in the BI environment represent the real-world objects and events being described:
There are many manifestations of this performance parameter. Overall it points to how well we have been able to extract and process the data as per the defined business rules. The success on this parameter means that you are:
- Extracting correct data from the correct systems
- Transform the data so not to loose the completeness and accuracy of the data
- The derived data fields from the 'base' extracted data is as per the correct business logic
- The aggregation of the data is as per the permitted additivity
- Correct business logic applied in the end user tools (for example 'residual commission' is calculated in the same way in BI as it does in the production systems)
Essentially, one has to ensure that apart from data quality, the business rules and logic behind all the information is constant throughout the source systems and BI environment.
All necessary decision support data is available within the data warehouse:
This goes beyond the defined user requirements. As mentioned in the DW modeling should not rely only on the business requirements, one needs to apply a more holistic thought process (for example foundation dimensions..) to ensure that one can cater to a more thorough information, even if users are not asking for it. In other words, unless it has a visible impact on the time cost and effort, I would like to pick up detailed data and as much logically-grouped data as possible. For example, I would pick up all important customer data while extracting customer master, and not only limit myself to the 15 (say) data fields as mentioned in the business requirements.
Your BI has data consistency:
This one talks about the integration of the data and also standardizing it. If you refer to the topic of data transformation, there are many ways to check and correct the inconsistencies of the data. This also means that you are extracting data from the systems which have consistent data and most reliable data. For example- you will pick-up the data related to the receipts not from the collections system but from the core accounting system, where the revenue has been realized with crediting of your bank account.
Your BI architecture provides a single system of record for decision-support data:
This means that all the data and information used is coming from the data warehouse. There are no parallel channels of information, for the scope of a given data warehouse. For example, if your data warehouse is having the sales management related information and still users are creating reports/MIS through ad-hoc extracts from the distribution system, it will be considered as a failure.
The %age of data, which is actually used: This is the reverse view of the fulfillment. Data warehouse is expected to provide what is needed. On the other hand users are expected to use the full potential of the data warehouse. We strongly recommend that one works on maximizing the usage and diagnose the under-usage before progressing on further initiatives.
Sources of data for measuring BI performance on above parameters
- Production Support reports on data quality issues and errors
- User feedback: On consistency, data quality, decision support capability
- Change requests by the users: Provides an indication on decision support capability
- The reports on the data used from DW and OLAP
|
