|
Just like we have a concept of foundation OR conformed dimensions, you will need to have standards at various levels of metadata design. Here is the list of different standardizations you will need. A great deal of standardization can be enforced by your metadata tool. Most of these standards are typical and are equally applicable for any IT application OR a Data Warehouse. We will give examples specific to metadata.
Standard Metadata Models:
Metadata Models are like dimensional models in a data warehouse. The metadata models define the structure of database in which the metadata is stored. For example, you would have a metadata model for a physical database table. In a standardized metadata world, this model will store information about all physical tables existing across all the applications. You will have different models for different type and level of technical and business metadata. This is just like having different structure for different entities existing in your data model.
Standard Metadata naming conventions:
The metadata models will have standard naming conventions. Primarily the standard naming convention is linked to naming of fields (OR attributes) in metadata model (for example the column_name field could be a standard name. You would not like to have fields like col_name, c_name..).
Data standards within Metadata:
One needs to enforce common data-standards like any robust application environments.
Taking a parallel from Data Integration- Let us say that you have a data standard for customer ID, whereby we have a structure of 2 alpha characters, followed by 5 numeric characters, with no upper case and no 0 in numeric. Your current applications might have been following different structures to store customer ID. As you integrate the data in a data warehouse, you may assign a standard customer ID (with standard data structure). You will then map it to the actual customer ID existing in the source system.
Taking this concept to Metadata environment- Let us say that you have a data standard for the Table_Name field in the table, which stores the details on physical database tables. One example of that standard could be the XX (alpha characters signifying the application to which this table belongs) + XXXX (the logical entity this table belongs to) + XX (version of the table). As you find that applications are not using this naming standard for the table name, you will create a separate field carrying the table name as per the data standard. You will then map it to the actual table name as it exists in the applications.
TIP- Use these standardized names as the main reference points for your queries, as the actual names could be in duplicate and will make your queries complex.
TIP- You may not like to use these standard names as primary keys in our metadata tables. This is because these physical tables may undergo a structure change in their application. To track this change, a new record will be added with a time stamp, in the same metadata table. This is similar to slowly changing dimensions in a dimensional model of a data warehouse.
Standard data exchange schemas:
An XML metadata exchange standard will work, if there are standard schemas for metadata exchange.
Taking a parallel from application interfaces, two systems will be able to exchange data, if they are aware of the schema in which the data is going to be transferred. XML provides a common language which two metadata repositories, can read and write in. However, if the structure of the metadata sent from one repository to another is not aligned, this common language will not work.
For example – Let us say that you are exchanging metadata on the physical tables across Data Warehouse metadata repository and ERP metadata repository. If the ERP system expects the table_name field to have a standard structure of 10 alpha characters, and it finds nine characters, it will reject the XML file. This is a direct implication of metadata model standards.
TIP- Include data exchange standards as part of the responsibilities of the data steward.
Standard semantic Metadata Layer:
Semantic Metadata layer is a universal concept, and not linked only with metadata. Semantic layer (in its simplistic definition) enables a more user friendly view of the data and other objects lying in an application. For example, while in database the table name is XZ-CUST_MASTER, the semantic layer will map it to – ‘Customer Master Table from CRM system’.
As you use metadata, apart from having data standards on what you actually store in metadata tables, you also need to have standard semantics. Extending upon previous example – for a table name, one can gave a semantic standard like - Name of the entity (like customer)+ type of entity ( master) + Location of entity (CRM system).
|
