Unless your data-mart is very simple, ETL is one of the most unpredictable areas
in any Data Warehouse design and development work. The surprises are faced on the following
areas, which are difficult to estimate in the initial phases when
you are preparing a business-case:
The data quality of the data in the source system
Typically you check the data quality of some key data elements and not all
the elements. When you get into designing ETL and start preliminary testing,
you some across many data elements, which you took for granted, but they come
out to have horrendous quality.
Complexity of ETL routines: When you run them on the system, they take more
system load and time, than you anticipated and you have to re-work the same.
Change in requirements during ETL design: This is a conventional issue, and you
cannot totally eliminate it. |
