|
One of the key differences between BI and MDM is the level of variability and directions in which data can flow. BI is simpler as Data flow is typically unidirectional, whereby the source systems send data to a BI repository (Data Warehouse), from where the data flows to OLAP and finally to the end user tools (refer BI architecture). Data generally does not flow back from a BI platform to source system (unless BI produces some classification or derived information to be written back to some of the source systems). End-user tools can write-back to OLAP
, but from OLAP, there is no back-ward flow to Data-Warehouse, as Data Warehouse is read-only.
MDM is a single point reference environment for wide ranging applications ranging from BI to Real-time transactional operations. By this sheer 'infrastructure' role of MDM, there are many types of architecture and usage patterns. This list will be useful, when you design your own MDM architecture.
MDM transaction interception pattern
This is true-blue EAI application. This is how it works:
There is a centralized physical repository in MDM-CDI Hub. As soon as any of the transactional systems linked with MDM-CDI hub gets into an addition or updation of Master data, the transaction is intercepted, sent to the MDM-CDI hub. MDM performs various operations, and sends a response back. Based on the response, the transaction system 'commits' the transaction to its database, so that it can be viewed and used by the user of that system.
Here is the list of operations by MDM-CDI hub, response sent back to the transaction system and action taken by the transaction system
- Operation: MDM-CDI hub Cleanses, transforms or standardizes the master data record if it is a new record. For example, the intercepted record might be having the first name as 'Joe' and the MDM-CDI hub changes it to 'Joseph'.
Response: The changed (if needed) record is sent to the transaction system.
Action by transaction System: The transaction system commits the changed record to its database.
- Operation: MDM-CDI hub checks if there is an existing record, which match the search criteria for the new record to be added. In other words, if you are adding a new customer record, the MDM-CDI hub may come out with an existing record, which carries the same first name, date of birth, telephone number and PIN-Code.
Response: MDM-CDI hub sends a message that there is a matching record. It also sends back the existing matched record in MDM-CDI hub as well as the new record, which it had received from the transaction system.
Action taken by the transaction system: The transaction system can either change the new record, and send back to the MDM-CDI hub for reprocessing or it will accept the existing record, by making some changes (if needed) for MDM-CDI hub to reprocess.
- Operation: If information on the existing record is changing (for example changing the marital status of an existing customer), the MDM-CDI hub will be updating the information in its MDM-CDI hub.
Response: Send the message to the transaction system to commit the updation of the record.
Action taken by the transaction System: Transaction system commits the updation.
The essence of the above is that every transaction before being committed to the transaction system is intercepted by the MDM-CDI hub and only after the requisite operation to ensure the MDM objectives are met, is it committed to the transaction system. One cannot get more real-time from MDM perspective. One needs an extensive use of real-time EAI technology. SOA architecture is a good enabler, but is not compulsory.
In this kind of pattern, the MDM-CDI is the final reference point for master data.
MDM publish/subscribe pattern
This is a one-way data flow pattern. In this case, MDM-CDI hub provides the MDM to the systems like customer statements, billing, product catalogue printing etc... These systems, themselves are not generating data which flows back to MDM-CDI. Typically, these downstream systems subscribe to the 'master data feeds' from MDM-CDI hub and the data flows as per the feed settings in terms of the:
- The type of data.
- Any filters
- The frequency of the feed...
This kind of pattern can use standard EAI messaging technique to batch-replication.
MDM message-based integration pattern
This is different from the transaction-interception pattern in the following ways:
- The transaction system is the master system for the master data and the MDM-CDI hub is more for a reference point to that master data.
- The master data is committed in the transaction system, before it is sent to the MDM-CDI hub.
Given that the transaction system is the master reference for master data, the MDM-CDI hub is simply the recipient of that data. In this scenario, all the standardization, cleansing, searching and matching is done in the transaction system itself.
This scenario happen typically if the transaction system is a core production system like SAP, Oracle etc.., which is sophisticated enough to ensure the Master data quality.
This kind of pattern is advised for registry style architecture scenario, whereby the MDM-CDI hub is containing only the unique identifier fields, and the master data is lying in the source systems. This is because the MDM-CDI hub is not the master system for the master data, but only a reference point. Therefore a client system, when it is looking for the master data, will be routed by the MDM-CDI hub to the transaction system for the details of the data.
MDM information synchronization pattern
This kind of pattern is used when you cannot have a true persistent (or transactional) MDM-CDI hub. When we say persistent style, it assumes that MDM-CDI hub is the master reference point, and all the changes done in the transaction system are driven by MDM-CDI. In the information synchronization pattern, one can have a multiplicity of synchronization relations like:
- MDM-CDI hub as master and transaction system as a Slave.
- MDM-CDI hub as the slave and transaction system as the Hub.
- MDM-CDI and Transaction system have peer-to-peer relationship.
In an information synchronization pattern, the MDM-CDI hub can have different relationship with different transaction systems. For example, it can be a slave to SAP, master of in-house developed field system and peer-to-peer with channel management system.
The synchronization routines are run as per the relationship. It can be real-time or nearly real-time synchronization. In case of master-slave, it is a simple rule as the data from master takes precedence over the data in the slave system. In case of peer-to-peer, it will be driven by the business-rules which may vary.
One can use any EAI technique to make it happen.
MDM business intelligence (BI) analytical pattern
This talks of the two way data-flow across the MDM and BI platform. This is how the two-way flow works:
- MDM to BI: MDM-CDI Hub can be the best source system of master data to the BI platform. Therefore the extraction of master data from be done from MDM-CDI hub and taken through the standard transformation and loading process in Data Warehouse. Given that MDM does a significant transformation and integration job on master data, the BI platform will not need to do much of data transformation
- BI to MDM: BI platform, through end-user tools like analytics and data mining, can perform customer segmentation, ranking, risk rating etc... This information can be crucial for the operational systems (like risk management systems, sales compensation systems, customer relationship systems...). The BI platform can send back this information to the MDM-CDI hub which can provide it to the client systems.
MDM data warehouse pattern
This is the most traditional MDM to BI linkage. In this pattern, the MDM-CDI hub provides the master data to the data warehouse through ETL. There is no write back or back-word flow from the BI platform to the MDM (unlike BI analytical pattern, where there is a two-way flow).
MDM multiple system pattern
In this pattern, one has a situation, where there are more than one MDM-CDI hubs. This happens when organization is large, and there are insulated IT landscapes. It can also happen during mergers and acquisitions.
In this scenario, instead of merging the MDM-CDI hubs, it makes better sense to integrate these MDM-CDI hubs, without physically bringing them to-gather. |
