- By David Nolting
Denali CTP was released in July. You can find it at Microsoft Denali
We’ve recently finished a Proof of Concept using Denali to model the organizational structure for an international energy provider based in France. Denali introduces major improvements to MDS through data steward management capability with Excel. Excel is the tool most used and mastered by the user profiles who will be defining MDS entities. Denali marks Microsoft’s foray into the MDM space with its superior interface with Excel and its very nice Silverlight improvements to the MDS interface.
We could go on about the functional improvements introduced in Denali but much of that has been already described on the web site. An excellent source is this channel 9 video TechEd 2011 Denali
Real World MDM
In France, MDM is becoming a topic of discussion in IT urbanization. As data needs to be published universally across the enterprise, definition of entities and aggregation of data from disparate sources seems to be the best approach to accomplish this. Our client has been involved in many middleware projects using BizTalk for inter-application integration; however, the motivation to implement an MDM POC for universal data publishing surpassed anything I’d seen with the middleware projects. MDM was deemed critical because the current data integration environments for centralized data management had become difficult to maintain and unreliable. MDM, as a business concept is much easier for the business group to get its head around than say middleware integration.
This project pushed MDS features to the limits. Key POC criteria consisted in the following:
· Ability for the Data Steward profile to Model Data and control the Data Entry process
· Ability to Integrate Tight Business Rules
· Ability to Create Work Flow around the data approval process
· Ability to notify appropriate data owners when Business Rule anomalies occur
· Ability to Partially expose Hierarchy Information ( nodes ) to defined users and user groups
· Ability to subscribe to multiple versions of the data and the current version of data
· Ability to import Data Subscriptions into Excel
· Ability to Roadmap SOA Architecture for system and portal integration
MDS has all of these features but getting things to work as expected is another matter. The majority of time was spent solving Business Rules, Workflow, and Security aspects of the POC.
User Profiles
As with most projects and POCS, it is important to understand who will be doing what to achieve successful MDM. Our client defined two principal roles:
The Data Administrator
The Data Administrator would be working with modeling ( building hierarchies ), Business Rule definition, Security assignment, Version Management. This profile would manage the MDS artifacts associated with a model version.
The Data Steward
The Data Steward defines entities, manages the quality of data, and participates in the workflow process used to integrate data in the MDS model. The data steward receives notifications of data anomalies.
MDS vs. the Competition
Our proof of concept pitted SAP MDM against SQL Server R2 MDM with Denali. Our customer currently uses Business Objects for B.I. and SAP for ERP so MDM solutions from SAP where politically advantaged.
Key business and technical drivers defining the POC included a budget concerns, implementation complexity, and ability to meet basic design criteria – Modeling, Hierarchies, and Security. This was a first time, real world exercise in MDM for the client. Motivation was very high. The project was driven from the IT dept. but the business group quickly became stakeholders in the project. A comparison of SAP MDM and Denali MDS was made early on which led to the decision to select Denali MDS as the more appropriate technology for the POC. SAP MDM had more features and was deemed more complete. But, if Denali could meet the baseline requirements of the POC, it would be the technology of choice for MDM for at least 2 projects within the organization. Why was this decision made? Basically because the SAP MDM solution was considered to be too complex to implement for the POC and too expensive for the scope and budget of the project. Denali was pitched as an MDM lite solution that was much easier to implement and could meet the baseline requirements of the project. ReachSOA was called in to help pick up where Microsoft training left off. We worked on building architecture around MDS to round out the Denali offering so that it matched some of the “off the shelf” capability touted in the SAP MDM solution.
Before we get to that, here are some key functional takeaways from the POC:
Data Modeling
The Data Administrator modeled and remodeled Derived Hierarchies using the Excel plug in. What’s important is that CSV data dumps from various systems were rapidly modeled in Excel. This phase was drastically easier to perform than in the previous version of MDS.
The Data Steward easily defined and modeled about 10 different entities that reflected real corporate data. The exercise in MDM Entity identification and definition was the first step in helping business formalize data relationships and organize data more strategically. Entities prior to this exercise had not been formally defined.
Data Modeling using the Excel Plugin proved to be easy and intuitive enough to allow the Data Steward to define entities and established derived relationships between entities. The excel plugin reduced the time necessary to import and stage the data drastically from the previous version of MDS where SSIS or other means were required to import the data. The plugin adds high business value to MDS by transferring entity definition and data import capabilities to the functional Data Steward. Version Management
POC expectations required us to focus on version management capabilities built into MDS. The POC had to successfully demonstrate how the Data Administrator could define and manage versions of the data.
Version Management requirements for the POC included: Version creation, Version Backups, Assignment of the Current Version, Fall Back to a previous versions, Version Data Subscription management, and Version Delta Reporting. MDS was capable of meeting most of these requirements. The only area where it seemed to fall a little short was in the Delta reporting. We showed Transaction Reporting to detail modifications made to models, entities, and data which seemed to satisfy the client. We showed that transaction data could be mined to create simplified delta reporting but a one touch solution – like a database diff report – was not available off the shelf in MDS. Other version management features were very close to what the client expected. The Version Flag can be used to guarantee that subscriptions retrieve data from the most recent version as shown below:
Figure 1 - Subscription Export based on CURRENT Version of Data
This helps with Change Management control for subscribing services. These subscriptions can be mapped to Service Data Transformation Objects to assure that connected applications move seamlessly from one version to the next of the MDM data.
- Versioning is intuitive and flexible enough to meet the demands of the POC. We showed how the Data Manager can transition from open to committed states and report ancestry:
Figure 2 - Version Ancestry
Security
Security was a critical aspect of the POC. MDS had to show how security could be managed at the attribute level of an Entity. Hierarchies structured by business units had to assure that managers could only visualize and modify entities in the business units they managed. Basic security features were easy to modify; however, we ran into challenges trying to configure MDS to block out Hierarchy Members for unauthorized data managers. We had to contact Microsoft who, in turn, contacted experts in the U.S. to configure our Beta copy for Node level security in Hierarchies.
Once we got over the hurdle of security at the Hierarchy level, we successfully showed security requirements for the POC application. However, without this support, we would have been required to resort to collections to expose partial hierarchies. The built in security features worked perfectly. Business Rules
The Data Administrator managed to define a set of business rules that validated data. Since data is imported from disparate sources, basic data cleansing and normalization was easily performed through the Business Rules layer.
Email notifications were assigned to Business Rule anomalies. MDS configuration was simple enough and worked 100 % for the POC.
Work Flow
Workflow was built into the POC. Workflow did not consist in WF modules linked to Business Rules. The Data Manager assigned a simple workflow leaf attribute to the principal entity. When the entity was initially imported, the flag was set to “Needs Approval” and a notification was fired to the data steward. A link to the Data Entity Entry screen in the email allowed the Steward to review, and approve or reject the Entity. This was a somewhat acceptable approach for the POC; however, true WF integration with access to a SharePoint portal will probably be the preferred approach for the actual implementation. The advantage of the WF triggered business rule is that the data is not integrated in the MDS database and will, therefore, not be visible in export subscriptions.
SOA
Exposing services that consume MDS entity and aggregated data is an important requirement long term for an MDM solution. Without getting into too many technical details we showed ODATA services modeling of MDS entities. This approach will work well for read only consumption of data from MDS. Services should be decoupled from the MDS structure through versioned DTO ( Data Transformation Objects ). This will allow for change management of underlying MDS data without directly impacting services. We look forward to working with the MDS WCF services that allow for entity creation, data publishing, and basically the features available in the Excel plugin.
Change Management
Change management should be built into MDS and the outlying SOA solution from the beginning. To reassure Business, we addressed Change Management and Versioning strategies at every stage. This included Model versioning and versioning of services
POC Results
As mentioned earlier, the POC was sponsored and managed by IT. The Business Group did not see the results until the final presentation. MDS and our presentation fared well in front of the Enterprise Architects and Business Process Owners. The BP Owners locked into the potential early on looking at the Excel Plug in and the Excel export as a practical means of managing data. Seeing real data restructured in MDS hierarchies spoke volumes. The lead EA appropriately challenged roles, infrastructure requirements, and who would be in charge of managing the new data source. IT’s involvement in the design and functional application modeling will provide strong backup and support for MDM moving forward.
Fortunately, a project will be launched in early September, after “les vacances bien sur”!
