Sales Management Customer Relationship Human Resources Business Performance BI & Data Quality IT Tools & Vendors

Sign-in   Register
Establishing 'Making it Happen' as a 'Formal & Predictable' Discipline
   Data Warehouse Purpose and Objective- Why is Data Warehouse Needed? Data Warehouse Challenges and Issues  

ENCYCLOPEDIA→   Enterprise Intelligence  →   -  Data-Warehouse/Mart  →   -  Data Warehouse Overview  → 

Data Warehouse Components and Framework

Data Warehouse framework starts from extracting data from source systems, transforming and cleansing it, before loading into the repository. It ends with the data being accesses, analyzed, mined and dashboarded using end user tools.

This page presents a high level listing of the components linked to Data Warehouse. PLEASE REFER TO Business Intelligence Architecture for a complete big picture on the relevance and positioning of Data Warehouse.

Source systems and Databases

Source Systems are all those 'transaction/Production' raw data providers, from where the details are pulled out for making it suitable for Data Warehousing. The sources can be quite diverse:

· Production Databases like Oracle, Sybase, SQL.
· Excel Sheets.
· Database of small time applications like in MS Access.
· ASCII/Data flat files.

Data Staging 'Area'

The data staging area is the place where all 'grooming' is done on data after it is pulled from the Source Systems. The end point of grooming is for the Data to be loaded into the 'Analysis OR Presentation Server'. Data staging covers most of the 'back-bone' activities of a Data-Warehouse, which typically are also the biggest analytical and technical challenge of a project. These activities are 'Extraction' and 'Transformation'

ETL-Data Extraction

Data Extraction is an activity, which pulls the data from various data sources. Most of these sources are production systems OR are used for transaction level work.

ETL-Data Transformation

If Data Extraction is mining the iron ore, Transformation is to create the steel billets. The Transformation makes sure that the transaction level raw data is transformed into a form (while still being detailed) so that it can be loaded into the 'presentation/Loaded' area.

ETL-Presentation/Loaded 'Area'

This is the repository where the data is finally loaded after going through all the works of Extraction and Transformation. This becomes the ultimate source for information for various reasons ranging from queries to advanced data modeling.

Dimensional Model

The presentation area has data model, which is different from that of production system. This is called Dimensional Model. It is the way data is organized in data-warehouse. This concept has been dealt with fair degree of detail as this is the engine of Data Warehouse.

Meta Data

Meta Data subject is covered in a separate section. It contains all the business and technical designs, rules and locations etc. of all the data starting from the Extraction to final data usage.

End User Tools and Applications.

Data is cooked for consumption. There is a long list of applications to which the data can be put to and the tools, which can make it happen. This includes the reporting, publishing, analysis, modeling and mining tools.

Data-Warehouse Administration and Tools

Data warehouse is a large platform, which has large number of users, data sources and data targets. Just like production systems, it has to be administered in terms of performance, timelines and availability. This also includes activity logging, data security, backing-up and archiving.

Data- Marts

The entire section of Data Warehouse is equally applicable to a Data-Mart. A Data-Mart is a Data repository with a more restricted and short-term perspective. Please refer to De-Normalized Data Warehouse/Data Mart for similarities and differences between a Data Warehouse and a Data Mart.

OLAP Servers & Data Marts

While Data Warehouse can be accessed for any end-user tools application, it also feeds to the downstream OLAP Layer. For example, HR wants to have its own data mart in their separate servers due to confidential reasons. Similarly people who are traveling may need to have their own offline data Mart.

 

   Data Warehouse Purpose and Objective- Why is Data Warehouse Needed? Data Warehouse Challenges and Issues  
 
 
All Topics in: "Data Warehouse Overview" Chapter
 Data Warehouse vs. Data Mart | Data Warehouse vs. ODS →  Data Warehouse purpose Objectives | Integration | consistency | quality →  Data Warehouse Components and Framework →  Data Warehouse Challenges and Issues | Data Warehouse vs. OLTP → 
 
 
More on Data Warehouse Overview
Definition- What is DW?
Purpose and Objective- Why Data Warehouse?
Data Warehouse Challenges and Issues
BUY BI & Data Management Vendors & Tools Evaluation Kit
Read more...
BUY largest on-line Data-Quality Management Kit
Read more...
Additional Channels
Principles & Rules
Free Templates
Glossary
Key Performance Indicators



Most Popular Zones with list of pages crossing 25000 hits  →→→ 
Maximising Sales Performance
Sales Leads Management Concept
Sales Revenue SWOT
Sales Campaign Infrastructure
Sales Channel SWOT
Sales Behavior
Read more...
  Customer Relationship Management
Customer Satisfaction and Retention- Overview
Customer Segmentation approach
Customer Satisfaction & Retention- Data Management
Customer Segmentation Data Management
Customer Value and Profitability Data Management
Read more...
  Human Resources & Leadership
Give feedback closer to the observation
Develop Self and Others
Leadership Development- Setting the Context
People become the way you treat them
What is Leadership?
Read more...
 
 
Business Performance & Planning
Creating Strategy Blueprint
Never design performance systems for specific KPI
Strategic Planning Business Themes
Strategy Map to Strategic theme
Strategy Map Objectives Measures and Initiatives
Read more...
  Business Intelligence & Data Quality
Non-Additive Measures-Facts
Master-Data-Management CDI Objectives
Business Process Controls
Derived Dimension Attributes Table
BI Cost-Reduction- Open Source
Read more...
  IT Vendors & Tools Management
OLAP Server administration
Single point vendor needs to be cost-effective
Collaboration and Administration Support
End User Reporting Features
Vendor Commercial Evaluation- Billing structure
Read more...