BizTalk Summit 2014 was a fully packed event with many interesting speakers. BizTalk 360 and Saravana Kumar did a good job putting this together. After two intense days, we arrived back in Sweden with new contacts in our network and more knowledge regarding new information and techniques.
No less than 12 integration MVP’s were represented at the sessions. On top of that, Harish and Guru from Microsoft initiated the summit by presenting some of the news regarding BizTalk 2013 R2 and BizTalk Services. At the summit as a whole, the subject focus was as broad as it can get within the narrow integration area. Amongst the covered areas we can find BizTalk 360 (of course), WABS and Azure Mobile Services but also a fair share of sessions covering softer subjects on a higher abstract level.
First of all, we are all happy to once again see a united front from Microsoft that BizTalk is a product to rely on and will be around for many years ahead. Microsoft describes it as Reality and points out that it is a vital part in the hybrid solutions that now also is reality. They also press that it still is an important product as an on premise platform, which is how it often is used by many of the customers. The release plan is a regular one release per year where they will be alternating between major and minor (R2) releases. Guru Venkataraman (Senior Program Manager - BizTalk Product Team, Microsoft) also revealed that just for testing the BizTalk engine there will be around 7800 tests available. The large amount of tests may sound insane but as Guru explained, it is necessary since they need to maintain and support BizTalks heavy backpack that has been building up since back in 2000.
With that said, let us move on to the fun parts; a wrap-up of the sessions!
The first session opening the summit was held by Guru who presented new features covered in BizTalk 2013 R2. The largest effort has been put in to optimizing compatibility but some new features will be released:
The long awaited native JSON support is very welcome by the BizTalk community which now is mostly using third party components to achieve this. Although, I clearly agree with Richard Seroter’s (integration MVP, when talking about A look at choosing integration technology) statement that the idea of using XSD schemas for JSON is somewhat absurd since it really destroys the whole idea of JSON. However, with the architecture BizTalk is built on today, there is no choice but to implement it this way to be able to support transformation between messages in a btm mapping. Another new feature regarding REST is the processing of empty messages using an empty schema. You heard me, an empty message! It may sound strange but it is in fact a very powerful feature. Guru had a great demo showing that this schema solves the whole problem that the REST adapter had earlier when no message body was sent with a REST GET request witch just is a simple URL.
Some other features added in BizTalk 2013 R2 are requests from American Health Care. When he talked about this, the thing that got to me most was the new xml field data type FreeText. This data type literally makes BizTalk stop parsing that field and says “Hey BizTalk, what’s in this field is none of your business”, even though it could be some xml or invalid characters in the field. I love loose coupling so let’s hope this field is as useful as I think it could be. Planned release is June 2014.
The latest news about WABS and Service Bus features were presented by Harish Kumar Agarwal (Program Manager - BizTalk Product Team, Microsoft). Some of the most useful new feature they presented is support for EDI and that WABS now can receive messages from Service bus queues and topics.
As a short and neat presentation on this, they simply took the EDI parties from BizTalk and put them in the cloud. They have tried to simplify the process and kept the most used parts visible and close together, the rest went under the “Advance” section to not interfere in the interface. Harish showed us how the setup is done and he demonstrated how easy it is to add and manage parties. Since this is done in the portal, it is a lot easier to distribute the management of the parties to the users who really should administer these things.
How about maintaining solutions in WABS? Steef-Jaan Wiggers (Integration MVP, Author) had a lot of nice input to this and as with solutions overall, the keys is good source control management and good administration options. So if we take the scenario that the source control part is covered and we started to investigate how to maintain the deployed solutions. How it works today, the main tool that you actually would use is Visual Studio which might seem a bit odd since we are used to have some sort of administration tool added as well. At the moment we cannot edit, but we can read the settings of adapters for example in the Azure Portal which might not be the best solution but it works. The REST API and PowerShell could be used for managing the solutions, stopping, starting, deploying etc.
So what if you would like to start moving some of your current integrations in BizTalk Server to the cloud using WABS?
Jon Fancey (Integration MVP, Author) showed us in a simple demo that migrating EDI parties and agreements where quite trivial. Migrating of ports is fairly easily doable (as long as they are of the adapter types supported by WABS) as well as pipeline components to bridges and map migration, except for some minor necessary changes. How about orchestrations then? First of all, orchestrations are not part of WABS which is replaced by workflow. You may question this, what, I cannot do these awesome cool Orchestrations anymore? And the answer is: NO! Though, that is not entirely true since the new workflow has been added instead. Jon was not entirely pleased with this as it means it will be hard to motivate a customer not migrating an orchestration to a workflow. WABS does not come with any tool for migrating this so he wanted to find a solution for this by actually making a converter from orchestration to workflow.
The same technique enabled him to also fully migrate pipeline to bridges in WABS. The workflow converter is not fully developed yet but it will be in the future and his demo clearly showed that it will be possible to convert a really advanced orchestration into a workflow and still have almost the same functionality. Even though I might not be the biggest fan of workflows, this approach might become one of the most common ways to solve migration problems. Thus, it will probably have a high possibility to bring the most value in the sense of cost per migration. Still, my opinion is that it is better to use the new platform on the way it is supposed to be used and in the most optimal way. Always use and take advantage of the framework don’t fight it!
Tord Glad Nordahl (Integration MVP) had a memorable session on the first day, where he in a clear way stated that all developers are evil and lazy using real examples in a humoristic approach. Personally, I can’t agree to that, not all the way at least. But I do agree with him on the areas such as keeping good strategy of host instances and a good level of logging is making life easier and the solutions much more maintainable. A great admin that knows what he is doing will make the BizTalk server so much happier and healthier. He clearly pointed out that he often sees problem with the disaster recovery (DR) model that might be “forgotten” or just handled poorly. He presented examples like for instance that someone backup their message box instance then two hours later they backup the configuration database. What are the consequences of this? Well you do backup, congrats! But have they actually tested that the backup can be restored? Probably not since this setup is doomed to fail, the consistency is lost and you will run into problem when you try to run a DR. So basically you’re screwed. His real life examples really lightened up the room and everyone had a good laugh, although almost everyone there were developers.
We keep on talking about BizTalk Server 2013, we saw a pretty amazing demo by Stephen W. Thomas (Integration MVP) of how to automate creation of complete BizTalk environments, complete with all complex setups done automatically! I mean, who hasn’t dreamt of creating a complete environment with two clustered SQL servers and three BizTalk nodes with one button click, with the exception of a configuration file to change values in. Azure brings these great possibilities to the area of easy virtualization and powerful API’s. Both PowerShell and the REST API have some really great strengths.
With this, Stephen demonstrated the strengths of Azure in a clear and simple way and that really powerful and useful things can be created with small means. Just think about how hard some complex test cases are to create, where you need multiple BizTalk servers and a fully functional Clustered SQL server, just as you have in production. Before Azure I would say almost impossible (unless you have all the time in the world) and even with the Azure portal it still takes a lot of time creating all VM’s, doing the configuration and so on.
So Stephen’s work ended up in a couple of PowerShell scripts and one config file with three parameters and he could create all of this in one click! With this we could actually do these tests. Just use a script to create the environment, create the tests, run them and verify the result. Kill the machines, delete them and go on with it just like it was a simple test case! Well, true not all that simple but, still now it’s doable. This is some really awesome stuff and I look forward to see more of it and be able to create all kinds of messed up test scenarios.
At the Summit a Nino Crudele (Integration MVP) on fire had a session on a Visual Studio tool for BizTalk he developed himself. He was literally so exited he could not stand still. He has created some extended tools for Visual Studio with focus on speeding up the developing process. I especially liked the tools that he created for integration solution overview and automated documentation. You could inside Visual Studio right click on an orchestration file or project to generate some real nice documents that described the process and a lot of metrics. There were also tuning of build process, dependent tracking and extraction of code out of BizTalk Server using a dll file! Awesome! He marked an orchestration and said “compare it” and the tool compared the installed orchestration in BizTalk with the one in Visual Studio and if there were differences it downloaded the dll file and extracted the source code of the orchestration. From this, you could manually check the differences and make corrections. This was actually possible to do with all artifacts in BizTalk. Talk about useful tool, in all those lost projects cases where several developers are involved and everyone thinks “uhm, I think this is the source code”. With this we could actually compare the source code with the installed code and if there is a mismatch decompile it out from BizTalk to continue developing from the decompiled code. Another interesting part was the automated documentation which was actually quite good, useful metrics and readable documentation, either for entire projects or just for a specific orchestrations. Guru stated that the tool is wanted and he want to add this to the Developer Toolkit distributed by Microsoft later this year, hopefully together with the R2 release. Another useful feature was possibility of testing of pipelines and pipeline components in the same way you test maps inside VS today. This might be really useful as a complement to other tools but I would still recommend using the unit test solution with WinterDom in addition as you build tests that are long-lived and can be really useful during the whole lifecycle of the pipeline or pipeline component.
We also saw a great presentation of the BizTalk360 product which is an enhancement of the BizTalk Administration tool. They did not only improve the user experience but have also added a lot of nice features like audit for all changes, improved security and authorization model so you could give user only read permissions for specific applications. Saravana Kumar (Integration MVP) also demonstrated the monitor and alarm functions available but the most impressive part were that by minimal effort you could access the portal directly via Azure. Using the Live-ID logging into the cloud, from where you could access multiple BizTalk360 instances that technically can be hosted anywhere in the world. That is pretty amazing and could be very useful!
Dan Rosanova (Integration MVP, Author) had a great presentation of BAM and how this could be used to give the business insight and status report of what’s happening in the integration flows. Obviously, this is not news, but he gave some very useful examples and tips how to use BAM in a way so it is understandable by administrative staff. I have seen and tried BAM before but Dan made it look cool and interesting, he pointed out some obvious but still very nice areas where this could be very useful and give more insight into the area of integration. He also pointed out that for most people in an organization integration is just a black box and with the right tools we could enlighten them about what is happening or even make them understand integration a bit more. In this way, integration can be brought to the table again with a richer life, and as he said “get the business people of our backs living happy with their user friendly tracking tool”.
Kent Weare (Integration MVP, Author) talked about this topic. As we all know mobility is a smoking hot thing on the market and everyone is talking about it, but what has it to do with BizTalk, WABS and Service Bus? Easy! They need integration as a part of the infrastructure otherwise they would be quite boring, living there alone in the mobile device. Admittedly, it is not that easy, they do know how to communicate with a server so they can get information. But still there is a gap area, from an integration point of view, especially when integrating a corporate app with an ERP system. We need to help the mobile developers so that they can work against a single API or endpoint and then let integration specialists do the work of connecting all the systems together. The coolest and greatest part however is that the Service Bus really helps us to do this in an easy way. Now they have added C# support for programming Mobile Services which means you are not forced to learn node.js. They have now given back the power to us! As we all know by now the Service Bus Relay helps enabling communication with the LOB’s and legacy systems and makes communication go seamless in and out of firewalls in an extremely secure and controlled manner. Thus, integration will be a key and play a bigger role in the future regarding mobility. We might need to speed up the process and make changes on the fly but we need the people understanding how things work.
Richard Seroter (Integration MVP, Author) had a session about management and different integration technologies focusing on when and how to use them. His focus where Microsoft core stuff as Service Bus, BizTalk 2013 ,WABS etc. The perspective in which he presented this was “Is this right for me?”, “Do I have people who understand this technique?”, “Is this technique mature enough?”, “How can we maintain the solutions we create?” and other important issues to address before choosing technology. Since a lot of the arguing can start like this “This technique is cool let’s try it!” and later on when you have 15 different technologies and platforms, I bet no one wants to be responsible for maintaining all that. So from what I see, it is very important to understand your organization and your current products and use that has the most value for the organization. Another key question is how steep the learning curve is, how long time does it take for my team to learn these new technologies? Is this technology strong enough to survive the next 5-10 years? These questions should today be addressed before asking whether we should use BizTalk or WABS. Or both? How effective would it be if we had five C++ developers that suddenly had to learn node.js?
The best plan, as Richard told, is to see the potential in your organization, what kind of developers do we have and make sure you believe in the product and not be afraid of mixing a few of them. Just make sure to have good reasons for bringing new ones in and not just because they are new and cool. For example, the Service Bus greatly makes the life a lot easier when communicating in and out over firewalls, so it’s a great add on to BizTalk.
Johan Hedberg’s (Integration MVP) session was about a very common and hard to solve topic, master data. It doesn’t matter if it is customer data or employee data etc, I think we all have faced this problem in some way. It is always hard to solve master data problems where often very complex and hard maintainable solutions are developed. Johan demonstrated the Master Data Services which is a service on SQL Server for keeping track of and manage Master Data. Basically the idea is to create a data type and an entity where the data type is the name of it and the entity represents how the data looks like. The service then creates tables and views for the entities and on top of this, there could be data validation rules added for automatic validation. When adding data we could just populate the staging table of the entity and it will be imported and later validated. For pulling or sending data we read from a view with some filter for only pulling the new, updated or data we are interested in. In this way, we could setup event pushing or just pull the whole table for full push to other systems. This is an easy job for BizTalk 2013! Simple, neat and surprisingly powerful solution.
This blog post is written by Mattias Lögdberg, Therese Axelsson and Robin Hultman.