Building Making It Happen
Building Making It Happen
  Sign-in         Register
    
Principles and Rules Listing Page
Maintain a trail of the key dimensional elements from source system to loaded
Sometime due to specific query needs and for audit-ability of your information, you would need to maintain a link between source to end information.
 
This page of 'Principles and Rules' is linked to:  Data Warehousing, Data Analysis/OLAP, BI platform Tools Evaluation, BI business intelligence end-to-end view, Metadata Management, Core Data Management Tools,


This is a relatively interesting and thoughtful tip. The typical use of this ETL and DW combo is to provide the information needs (enterprise reporting, Query and analysis and Performance reporting) for a given set of dimensions, data elements. It also provides you information to maximum granularity. However mostly the information on the 'source' or 'source system' of the information is lost once the data is taken through the ETL. That is fine as long as you have the above conventional needs. However, it becomes a problem, when reporting and querying is linked to a particular source system and not only for a particular information theme. For example:

  • Number of sales calls made to active customers v/s how many sales calls were made to active customer through the "Jupiter" (say a name of customer contact system)?
  • How much revenue was booked through the 'Saber' system?
  • Mismatch between the financial information reported through the data warehouse and the transaction systems. If this needs to be investigated, one needs to have an audit trail on how the information from source system got transformed to the loading stage.

Therefore it is advisable to have the key master fields (source system customer ID, location ID) to be retained as attributed in the dimension tables.

The other key benefit for the source system trail to be maintained is to change manage the dimensional model. There can be two types of changes in the DW environment from design perspective:

  • Source system undergoing a change and you want to do an impact analysis on the downstream dimensional model.
  • You have additional information requirement. Due to this, you are adding new attributes and facts to the existing dimensional model. In this case you will need to change your ETL programs.

In both the above scenarios, you will have to do an impact analysis and drive those changes. If you have a source system trail, you will be much better equipped on the impact analysis and doing the change management. For example, let us say that you are extracting travel booking information from three different travel systems. If one travel system undergoes a change in the 'travel_agent_ID' field code (which means that agent code is now 8 characters instead of 6 characters) and you have been tracking it in your dimensional model in the data-warehouse, you can make the change much easily.

   Access more details on this page   

Quick Feedback- Was this information helpful ?
Relevant Links to this page
Principles & Rules → Dimensional model has to be aligned to the Entity-Relationship → Principles & Rules → Always Use Conformed Dimensions → Principles & Rules → You may not be a able to have a perfect ETL → Practice Techniques → Handling Sparse Dimensional tables → Principles & Rules → Do not separate the parent and child line item data → Practice Techniques → Managing time-stamps across multiple time-zones → Practice Techniques → Recording events in multiple currencies → Practice Techniques → Handle different units of measure in the same fact table → Principles & Rules → Handling of Null foreign Keys in fact tables → Principles & Rules → Dimension Attributes as NULL → Principles & Rules → Don't rely too much on Meta Data Tools to enforce Business Intelligence → Principles & Rules → Don't wait for universal models for Data Marting → Principles & Rules → Add extra buffer for ETL phase → Principles & Rules → Homework before interviews is must (Business Requirements Phase in Data Warehouse) → Principles & Rules → Excel is the competition, which should be challenged → Principles & Rules → Avoid Pure MOLAP → Practice Techniques → Field Tips Series- Streamlining & Cost-Reduction in Business Intelligence- Consolidate Data-Marts → Practice Techniques → Field Tips Series- Streamlining & Cost-Reduction in Business Intelligence- Licensing & Maintenance Contracts → Practice Techniques → Field Tips Series- Streamlining & Cost-Reduction in Business Intelligence- Governance & Standards → Practice Techniques → Field Tips Series- Streamlining & reducing cost of Business Intelligence- Evaluate Open Source → Principles & Rules → Master Data Management- Making a Right Start → Practice Techniques → How to integrate stand-alone BI environments- Gradual Approach → Principles & Rules → Business owned applications are a reality- Manage it → Principles & Rules → New Data Standards- What about existing data and applications? → Principles & Rules → Handle Each Time-stamp in the Fact Table as a separate dimension → Principles & Rules → Keep Aggregates and Details data in different Fact tables → Principles & Rules → Some considerations for Infrastructure in Data Warehouse → Principles & Rules → For Core BI platform go for a single, established and robust player → Principles & Rules → Don't be guided only by the business requirements for your Business Intelligence → Practice Techniques → Using Synonyms and Views → 
 
Back
Featured Pages
Give & Take for Source systems
Business Intelligence Project Management Success Metrics
Source to Target Table
Data Warehouse ETL Loading

Make 'Executable' Strategy
Maximize Results
Maximize People
Manage Execution

Featured Pages
Avoid Pure MOLAP
Master Data Management vs. BI vs. Metadata
Rationalization of Information
Dimensional Attributes+ Facts + Source Matrix