Early heads up in Integration projects

Naming conventions, application structure, canonical formats.

I have worked on systems integration for a number of years and have encountered a number of strange names in both projects and applications. Below I try to explain some tips and tricks that can be useful. Of course, the choices you make depend on the situation, future demands, Biztalk license etc etc…

Naming and application structure

I stumbled on this a few years ago:

“Agresso.ABWTransaction.xxx.TransactionABW.”

What can you get from this? That there is a message from Agresso named ABWtransaction? Yes, but it’s also the only thing you get. Direction? Mapping? What is xxx?

Wikipedia:

  A pornographic integration. Ok! :-)

Try to have a common way to name the integrations. And try to plan ahead! Think about the fact that a helpdesk will be responsible for the artifacts. If it´s possible and you have some kind of Integration number to identify integrations, use it in the naming.

Another black hole where you tend to put artifacts are “Common” libraries, “all that should be shared”, but it is a truth with modification. The artifacts should be very static, for example, messaging standards such as EDIFACT, papiNet, OAGIS, etc…  Unfortunately, it is common practice to also add the canonical schemas there with specific pipeline components that are only used by a specific integration. If you are forced to do so, I recommend that you have just one xsd per project. A problem that can arise is that you have to tear down all the applications to update these “shared” libraries. All systems are affected by an update and a service window for the server is forced to be much longer than necessary. The production units could be affected by an update in a human resources system, that’s not easy to tell the customer or production staff.

One should therefore think about the application structure and naming conventions before starting to develop integration artifacts. It is possible to divide the applications in system groups, geographical and domain areas. Several times, I have had to take over an environment where naming was “outdated” and not prepared for the integration environment and business needs. It is common to install number of pilot projects before a full-scale project is started and unfortunately the naming is often forgotten. Then you must consider that there is a big difference between the administration of 10 BizTalk integrations than of 200 BizTalk integrations for a variety of systems. My advice is that you should think about the operation and management at an early stage. Is it easy to debug? Can you help someone from the business who calls and shouts “We can´t create the documents for the lorry in the warehouse”? Where do you start to look? The answer is that you should think about this early on and have an application structure based on domain areas, such as Logistics, Finance, Sales, BI, HR and so on.

You check the applications in the logistics domain and find a port that is stopped and you start it? Truck driver happy! Developers/administrators have a penchant for system naming, this can also be applied to a domain structure and everyone is happy:

CompanyName.Logistics.SystemName

The important part is the domain. You could also have the geographical area if you want:

CompanyName.Logistics.Europe.SystemName

CompanyName.Logistics.NorthAmerica.SystemName

Canonical formats

A canonical format is a common format for converting / mapping to and from. It is used to create loose connections and minimize dependencies between systems. A tip for you to start using canonical schemas is to open it up. I.e. remove constraints, cardinality (that is not justifiable) and in some cases even data types. Set the data type to “string” rather than DateTime in the date element. Why is that then? Well, because you do not know how the schema will be used or the direction in which the messages are sent. You do not know how the data will look. In other words, it is in the mappings from your canonical format instead, you must ensure that the mappings are correct and that the data is in the correct format. The only thing you know is your endpoints and which format to use before the message is allowed to take off. What reasons exist to validate a canonical format? None at all in my opinion! Canonical format is a transport format. The canonical schema should be based on what you know. If you work in for example the paper industry there is common format named papiNet. That’s an accepted standard that has existed for several years and thus it is smart to copy it (if possible) and use it as a canonical format.

If you don’t have a standard to copy, then take a copy of the sending systems schema format. Add an envelope schema part that is also well thought and start your integrations!

Unfortunately, there is no best practice for these types of projects because there are a number of parameters to take into account. But this is some of my thoughts and ideas! Hope it interests some of you.

Over and Out!

Posted in: •Integration  | Tagged: