Message queuing

Message queuing is an often forgotten technique to provide asynchronous communication between systems or services where the server and client do not need to interact with the message at the same time. MSMQ is Microsoft’s message queue implementation.

Some of the advantages of message queuing are:

  • Load leveling Most systems do not have an even load. If you design your system for peak load, you are wasting resources. If you design your system for average load, you will not be able to handle peak load. When using message queuing the calls get queued up and the service can process them in its own pace.
  • Asynchronous communication The server and the client do not need to interact with the message at the same time. The client does not need to wait for the server to process the message.
  • Reliability A client may need to work against a service even when it is disconnected. By utilizing MSMQ the client can queue up messages to a local queue and then send them to the service when it is connected.

A MSMQ queue can be configured to be either a transactional queue or a non-transactional queue. Messages sent to a transactional queue are transferred exactly once.

There are typically three transactions involved in a message transfer (if the client and the server are not on the same machine).

 

 

Client transaction The client transaction takes place when the client pushes a message to the client queue. If the client commits the transaction, the message is pushed to the queue. If the client aborts the transaction the message is rejected from the queue.

Delivery transaction MSMQ is responsible for delivering the message from the client queue to the service queue. If the network is down, or the service machine has crashed, message removal from the client queue is rolled back. MSMQ will eventually deliver the message when the service is up.

Destination transaction The service tries to remove the message from the service queue. If the transaction aborts, message removal is rolled back. If the transaction is committed, the message is removed from the queue.

SOA’izing the message queue Working directly with MSMQ is hard. The developer must handle transaction commits and roll backs explicitly. Retry functionality must be implemented on the client and on the service. The messages must be hand-crafted and throttling behavior must be implemented manually. In the SOA world (Service Oriented Architecture) we are used to not think in terms of messages on a queue, but transport neutral business operations.

WCF to the rescue WCF can act as an abstraction layer on top of MSMQ to shield you from many of the pains of using MSMQ directly. In the SOA world we are used to communicate via contracts, often exposed as WSDL documents. The service contract tells the client how it is supposed to communicate with the service, how transactions are implemented, which security settings should be used, throttling behavior etc. The data contract tells the client what data needs to be sent. A proxy is generated on the client side based on the contracts. The proxy does all the heavy lifting of converting WCF-messages to MSMQ messages, enlisting in transactions etc. There is not a direct mapping of WCF messages to MSMQ messages. The client posts messages to a queue, not a service which means that all calls are asynchronous and disconnected.

 

Another important aspect is throttling. If there are 15 messages on the queue the service may not want to have 15 concurrent instances and worker threads. The throttling behavior in WCF controls how many instances and sessions can be handled at once. Maxed-out messages stay in the queue.

This has been a brief introduction to what the benefits of message queuing is and how WCF makes life easier when it comes to message queuing. For a more technical reference on leveraging MSMQ in WCF I can really recommend Programming WCF Services by Juval Löwy.

Posted in: •Integration  | Tagged: •Message Queuing  •MSMQ  •WCF