Back Again
Jumping in a little late in the game for year-end recaps but the good news is we’ve been too busy to post. The past 6 months have been nonstop, round the clock, project implementation. Who ever said anything about the 35 hour work week? Here are some of the types of projects we’ve gotten are feet wet with this year:
Core Model Integration
We’ve had success getting companies to apply strategy towards integration rather than just using BizTalk as a tactical link between systems. One project involves developing a core model to absorb regional EAI and B2B with a core, corporate infrastructure. This company is worldwide and is harmonizing all regional field service transactions behind a single, multi-tenant instance of the corporate software. The regions – Europe, North America, South America, - all have different ways of handling repair transactions and communicating with their B2B client base. BizTalk would be used to transfer data from the regions to the back end repair system. So naturally we heard questions and comments like “ who will support the systems?”, “we don’t want to get involved in troubleshooting regional data”, “who will develop the flows for the regions?”. When we looked at one of the first regions, we found pure spaghetti: Out of 50 B2B clients, we found just as many file formats. So the B2B partner dictates all system integration specifications such as file format, mode of transfer, and SLA response time. We didn’t try to force a change here. Enterprise Architecture should allow for localization and we want to let the regions handle their customers in their own way. However, many of the regions IT guys said things like “ just give us the SQL tables and we’ll write our own queries to the back end”, etc. This is what they had done in the past with local instances of software product pushing the spaghetti theory to the max. So what we came up with was this : a corporate instance of BizTalk and a Regional instance of BizTalk. The corporate instance would expose standard field request services – RMA, INSTALLATION, CLOSING – that were corporate SOA services. The Regional instance of BizTalk would pick up the files and map Flat File schemas to the corporate services and transfer via WCF to the corporate instance. We started with a BizTalk branch model but quickly moved to a BizTalk Enterprise instance that would house all the regional flows. Regional flows would be limited to file pickups, Flat file conversion, and mapping to the regional services. Pub/Sub is used to route information back to the proper region. Corporate was happy with this solution because they could control the infra structure and functional monitoring of their corporate instance while giving the region our functional dash to monitor their flows for each region. The European guys could log on and see their flows, the USA guys, their flows, etc. Regions could decide whether they wanted to develop the BizTalk flows themselves our use the corporate 3rd party developers to do the work.
Functional Monitoring Projects
We’ve seem to have struck a chord here. Our functional monitoring pattern based on a BAM mash up with ESB, DTA, and external application sources for BizTalk and AppFabric is in high demand. This is not a technical monitoring solution based on BizTalk Message box but really focuses on delivering a business point of view to integration data stewards. This is a pattern that we’ve evolved over several projects since BizTalk 2006 but it is now used in as part of our core integration model described in a future post. This pattern is as non-intrusive as possible. For example, we’ve implemented it in one of the largest BizTalk installations in France for a multi-billion euro manufacturing company in the energy space.
The pattern is gaining momentum because if fills a void in the BizTalk and SOA ‘functional’ monitoring space. We have at least 5 new clients lined up for the pattern and we insist on its usage when defining a core integration model for clients. We adapt the pattern and deliver the source code for each new client which differentiates us from those selling licensed products.
Referential MDM Projects
Without looking for it, we have worked on at least three MDM projects this year one of which uses SQL MDS. One thing is for sure, MDM is starting to interest CIOs in France. Orchestra Networks MDM is a strong contender and especially suited for large scale projects. But we’ve found that with the right guidance, SQL MDS can be effective for mid-sized to large scale MDM implementations. SQL MDS fits nicely into the Microsoft integration stack and gives service companies the chance to build value around it with SOA, WF workflow, and EAI integration with BizTalk, AppFabric Connect, or the cloud.
We’ve worked on other, more implicit, MDM projects that required remodeling ERP information to support referential data and front ending it with AppFabric Cache and WCF to provide SOA MDM data. Since the ERP had been remodeled to support referential data, ( not our choice but one that worked out well for the customer ), a light weight solution to front end the data for high performance web services was required. AppFabric cache fit the bill perfectly. This architecture will be now employed by other clients who use the same ERP.
Portal Optimization
One of our largest projects of the year required high performance access to referential data. AppFabric Cache was used authenticate and recover client referential information ( name, address, fidelity info, etc. ). The web site needed to support 5000 simultaneous connections with an SLA of 15 000 in the years to come. The cache barely flinched as we bench marked it around 3000 transactions / sec using a simple Soap UI tool.
Other interesting things came out of this experience such as cache patterns for persist cache, reboot cache, BizTalk Asynchronous integration with the Cache, delta calculation between the referential and the cache, etc. Account creation and validation happened much quicker with the cache + business rules than if we had tried to link up synchronously to the back end ERP. As a result, we’ve strengthened our core integration model for synchronous and asynchronous behavior using AppFabric, WF, and BizTalk.
Alternative EAI
Where our initial reflex would have been to install BizTalk for EAI, we revamped a project ( with guidance from MS ) to use SQL brokered services to meet performance requirements: hundreds of satellite applications that push to and poll data from a central SQL based application. The solution, still under development, requires something like 3000 transfers bi-directional / sec. SQL brokering was chosen over BizTalk because tight coupling between systems was permitted. All systems connect via the same web service and push data ready transactions to the host. So things like mapping, pub/sub, and connectivity to a range of adapter types were not necessary. Even if BizTalk could be scaled up to meet performance, the price would have been exorbitant.
Conclusion
It’s been a ride this first year working as a coherent group @ ReachSOA. Things are snowballing and like any company, we need to figure out growth. We remain focused in the Microsoft integration space and have expanded our portfolio of Microsoft technologies to build out a coherent reference architecture for integration. Fortunately, we find ourselves dialoging more and more with core model, business architects. We’ll be posting in more detail on some of the technology related to these projects. A+
