You might notice that queues seem to play a large part in MSMQ code. Intuitively, you might imagine that all messages must travel through a queue to reach their destination; it is more precise to realize that the queue is the destination. Much like a database table, a queue is a persistent repository for messages that can be read from any MSMQ recipient in an FIFO (First-In-First-Out) fashion. You will see such a recipient in the section "MSMQ Receiver Implementation," but let's continue our queue discussion for the moment.

There can be as many queues as you care to create, and they can be created by a sender, receiver, or any application. In Listing 10.1, it is the sender who creates a queue called MSMQTest. The following lines describe the location of, and then create, a queue:

Dim qInfo As New  MSMQQueueInfo
qInfo.PathName = "darjeeling private$ MSMQTest"
qInfo.Create

It is important to realize that a queue is not a connection nor is it an asynchronous path between two points. Queues do not have a sense of direction relative to the clients and servers that use them in the same way (for example, as TCP/IP sockets do). One of the queue can be used by any client anywhere to push messages into the queue, and the other can be read by any server anywhere to pop messages off the queue and interpret them in some way. A message queue is simply a persistent entity that acts as a repository and FIFO-style distribution center for messages. It is probably best to think of MSMQ as the RM it is. The MSMQ RM transactionally handles its own proprietary data store and provides methods allowing for the storing and retrieving of your messages to that store.

Now, back to queues. After a queue is created, it exists completely independently of the entity that created it and can (security permitting) be used by any client and server pair at any time for any purpose. If you want to see message queues that have been created and are currently in existence, you can open the Computer Management Microsoft Management Console (MMC) snap-in and look under the Message Queuing tree-view item, as shown in Figure 10.1.

Figure 10.1. A list of message queues currently in existence, including the queue for a COM+ example application we will be using in this chapter, FirstQCApp.

After a queue has been created, it can be opened for reading or writing. A sender of a message typicallyopens a queue for writing (though technically, it can open and read from the same queue if it wants) with the following line:

Set qDest = qInfo.Open(MQ_SEND_ACCESS, MQ_DENY_NONE)

The arguments used in the preceding method call are typical, and we'll hold off on a more detailed discussion of these arguments; interested readers should consult Appendix E, "Queue Moniker Parameters." For now, note that theOpen method of an MSMQQueueInfo object returns an MSMQQueue object. The MSMQQueue object represents a connection to a queue and is used to sand retrieve messages from the queue.

Not surprisingly, the message itself is represented by an object, one of type MSMQMessage. The following code demonstrates how to set the simplest of an MSMQMessage object's properties and then send the message:

msgSend.Body = "Hello World"
msgSend.Label = "Test Message"
msgSend.SqDest

In this example, you are setting the Body property equal to the string "Hello World". The Body property is really a variant, and so it can hold any type of Visual Basic data type or array. The question arises as to whether you can set the Body of a message to an instance of an object and sthat object through a queue. The short answer is yes, provided the object supports theIPersistStream interface. We discuss this in more detail in the subsequent section "Persistence: Passing Objects Through Messages."

You might notice that the MSMQMessage object has aSend() method that takes the current queue connection (represented by the MSMQQueue object returned from an MSMQQueueInfo's Open method) as an argument. Thus the following line of code sends the message represented by the myMessage object to the myQueue queue:

myMessage.SmyQueue

When this method returns, the message is sent. All the sender needs to do isclose its connection to the queue. You do this by using the following code:

myQueue.Close

Remember, the preceding line does not destroy the queue. When created, a message queue stands on its own and can outlive its creating object or process. Queues must be explicitly deleted as shown in the next section.

All the basic objects, MSMQQueueInfo, MSMQQueueMessage, and MSMQQueue, are used in a similar fashion to the sender—the location of a queue is specified, a queue is opened, and a message is popped from the queue with the following line:

Set msgRec =  qSource.Receive

By default, theReceive method is a blocking call. This means that if there are no messages in the queue, the calling thread freezes when making this method call. You can, however, specify a timeout period using one ofReceive's optional arguments, or you can ask MSMQ to notify you when a message comes in. In the C API, you can enable this feature by giving a callback function to MSMQ; in VB, you can implement the MSMQEvent interface in a form or class and pass it as an argument to the EnableNotification() method of the MSMQQueue object. See the companion source (http://www.newriders.com/complus) for greater detail.

When a receiving application no longer wants to process messages, like the sender, it can simply call the following:

qSource.Close

Of course, the preceding line does not delete the queue, it only closes the receiver's connection to it. If you want to delete the queue, do so with the following line:

qInfo.Delete

This assumes, of course, that the caller's myQueueInfo is describing an existing queue and that the caller has the permissions to delete it. The ability to delete a queue, as well as privileges to send, receive, and peek (that is, look but don't remove) messages can be set on a per-queue basis.

Let's assume that you want to instantiate and obtain an interface (one designated as queued) from an object called FirstQueue. To create this object and obtain this interface, you need to go through the queue object via a moniker. This is demonstrated in Visual Basic in Listing 10.5.

//VB

Dim QC as IFirstQueue
Set QC = GetObject("queue:/new:FirstQC.FirstQueue")

//C++

IFirstQueue* pFirstQueue;
CoGetObject (L"queue:/new:FirstQC.FirstQueue", NULL, IID_IFirstQueue, (void**) &pFirstQueue);

In the event you are not entirely comfortable with monikers, I will offer a brief explanation. A moniker string can allow you to create a COM object and pass it arguments in the process. The ordinary COM creation mechanisms (mainly CoCreateInstance, New) do not provide a way to actually send construction arguments to an object. Although monikers are certainly useful in helping you bypass this restriction, they don't seem especially impressive until you take into account that the arguments they let you send can be other COM objects. And if this still isn't compelling, the COM objects specified in a moniker string can coordinate with one another and, in effect, cooperate to produce the object ultimately returned by a call to VB's GetObject (CoGetObject in C++). By way of example, examine the following moniker string:

queue:/new:FirstQC.FirstQueue

This moniker string has two COM classes in it, queue and new. If you look in the Registry, you will find both are COM classes with the ProgIDs of queue and new.

Both queue and new are ordinary COM objects, but they are also written to support certain interfaces (IMoniker being one) that make them monikers. An object wants to be a moniker if it wants to have its creation influenced by other monikers specified in a moniker string. The implementation details of a moniker are complex to describe, but from the client's perspective, it is pretty simple. When the preceding string is given to VB'sGetObject() method, COM+ processes the string from left to right. It creates the queue object and feeds it the rest of the string. The queue object processes as much of the string as it can, but then gives the remainder back to COM, usually after detecting and chopping off some kind of character delimiter, a / in this case. COM then repeats this process with the remainder of the string by creating the new object and giving the portion of the string remaining to that object. This can be thought of as the parsing process, but another process, the binding process, remains.

As moniker objects are created left to right, a common context (called a BindContext) groups them together and relates them to one another. A moniker object is given an interface (IMoniker) to the moniker at its left (queue in this case); thus, the new moniker knows that a queue moniker is at the left of it and can interact with it in any fashion. This negotiation between monikers from right to left is often called binding. The binding process is like a snowball rolling down a hill, right to left, getting bigger as more creation information is amassed, ultimately resulting in a single object being created. By way of example, the moniker string queue:/new:FirstQC.FirstQueue can be described in the following conversation:

The queue object mentioned in the moniker string ultimately produces an IFirstQueue interface. VB places a reference to this interface in the QC variable (as shown in Listing 10.5), and the client can call methods on it in a normal fashion. It is always true of moniker strings that no matter how complex, they always result in one single interface to one particular object.

Although creating objects via monikers might seem like an unnecessary step (after all, why not just create a queue object directly?), remember that monikers allow for arguments to be passed to objects. Although the moniker string queue:/new:FirstQC.FirstQueue is relatively simple, it is more common to pass additional information to the queue object in strings like the following:

queue:  Priority=6,ComputerName=Darjeeling/new:FirstQC.FirstQueue

If you cannot pass arguments in the moniker string, the author of the queue object is forced to write the queue object such that it has properties corresponding to every possible argument. You end up with code like the following:

'We wouldn't want to do this:
Dim q as new Queue
q.Priority=6
q.ComputerName="Darjeeling"
'And every other possible property

Keep in mind that, although you are creating a queue object in the moniker, the queue object is ultimately returning IFirstQueue, an interface normally implemented by the FirstQueue object. Ultimately then, the queue object is merely a way station, a staging area for the creation of another object. As you see in the next section "The Recorder," the queue moniker does not return an actual RPC interface pointer to an instantiated FirstQueue object, it returns an MSMQ-based entity called a Recorder.

Before we move on to the next section, note that some other variations of the queue moniker are detailed and explained in Appendix E.

The Recorder is a COM+ system object that pretends to be a given, queueable object. When you create a queueable object via the queue moniker shown in the previous section, you get back an interface to a Recorder, not an interface to the object itself (which COM+ does not even bother to instantiate). The Recorder is a kind of proxy that stands in for the requested object and implements the interfaces of this object (those marked as Queued), such that the client is unaware of the deception. To be fair, the Recorder cannot deceive the client completely; attempts by a client to QueryInterface (QI) for a non-queueable interface that the real object would ordinarily support is met with an error code. A Recorder is an imperfectproxy then, supporting only an object's queueable interfaces.

Proxies are certainly nothing new. Remember, a client application on Machine A can only have an interface proxy to an object on Machine B—that is, the client side of an RPC channel—not a pointer to the object itself. In this traditional case, there is a synchronous RPC connection between the two. The client can rely on the fact that when it calls a method on the interface proxy it holds, the call is immediately, synchronously delivered to a living object.

This is not the case with the Recorder. Think of the Recorder as exactly that, a recorder. The same way a tape recorder translates your spoken word to magnetic imprints and stores them on a tape to be played back to another party at some later time, the COM+ Recorder records method calls, translates them into messages, and stores them in a queue. When you record your voice on a tape recorder, you don't need to be around when the tape is played for someone else, and you certainly don't need to be around while someone is listening (processing) what you said on tape. To be fair, the person listening can play the tape recording whenever he is ready and prepared to, and the tape itself can outlive both you and the listener. If you think of the tape as an MSMQ queue and the listener and recorder as the COM+ Player and Recorder, you begin to get the idea.

The Player performs actions opposite the Recorder. It reads messages from a queue, turns them back into method calls, and invokes these method calls (plays them) on the queueable object. The Player can only play messages if the COM+ application in which the queueable object resides is running and listening for messages.

The next section demonstrates how to make a COM+ application listen, but basically an application listens to its queues through an instance of theListenerHelper COM class. When a message arrives, theListenerHelper object instantiates an instance of the Player and gives it the message(s) received. The Player, in turn, instantiates an instance of the recipient object, translates the message back into method invocations, and invokes them on the recipient object, security settings permitting.

As with the client's interaction with the Recorder, the recipient queueable object is unaware that the method invocations it is receiving on its interfaces are coming from the Player. They have the security context, privileges, and so on as though they are arriving via direct RPC invocation.

Queueable objects are ordinary COM objects. Aside from the fact that the methods of their interfaces cannot have output parameters and their method arguments can only take OLE automation compatible data types, they are not written any differently. Such objects can be used in the ordinary COM fashion, and it is only when they are instantiated via the queue moniker that their special nature becomes apparent. Keep in mind also that their queueableness can be taken away at any time by simply removing the check from the Queued check box for its interface(s), as illustrated in Figure 10.2. An attempt to instantiate a non-queueable object via this moniker results in an error.

Writing a queueable object is straightforward; anyActiveX Dynamic Link Library (DLL) will do (again provided that its interface methods do not have output parameters and it uses only OLE automation compatible types). InVB, after a new ActiveX DLL project has been created, adding even one subroutine, as in the following, is all you need:

In VB ActiveX class FirstQC

Sub DisplayNumber(ByVal Number As Integer)
    MsgBox Number
End Sub

But in VB, beware! The following method renders the class unqueueable:

'Can't be queued
Sub  DisplayNumber(Number As Integer)
     MsgBox Number
End Sub

The reason is that, in VB, all arguments are passed by reference (implicitly ByRef) unless otherwise stated by using theByVal keyword. In VB, ByRef translates to an [in,out] parameter in the underlying type library, which is not allowed for a queueable interface.

Assuming our object is named FirstQC, after you compile this object (VB registers it during the compilation process), you then need to import it into COM+ through mechanisms described in Chapter 6, "The COM+ Catalog." At this point, you designate the _FirstQC interface as queued. (Note that VB prepends its default interfaces with a _.) One additional step is required. Even though FirstQC is now, technically, a queueable object, the COM+ application that FirstQC lives in must also have the appropriate settings. These settings will be discussed next.

One of the property pages that appear when you right-click on an application using Component Services is the Queuing page, show in Figure 10.3.

If you find that these boxes are grayed out, you might check to make sure that the application is a Server Application and not a Library Application. QC-based applications must always be Server Applications.

Figure 10.3. The Queuing tab for an application.

If you click theQueued check box, by default, COM+ creates seven queues. You can see them using the Computer Management snap-in, shown in Figure 10.4.

The first queue has the same name as the application, and it is followed by five similarly named queues numbered 0 to 4. There is one last queue, known as thedead queue. Basically, a message works its way through all these queues as COM+ repeatedly attempts to deliver the message to the receiver in the context of a transaction.

Figure 10.4. The Message Queuing branch of the Computer Management snap-in.

The fact that the message delivery is attempted as part of a transaction should not surprise you, given that MSMQ is a bona fide RM. To MSMQ, delivering a message to or from a queue is akin to a more traditional RM, such as a database receiving a SQL update. As far as MSMQ is concerned, the queue is the data store, and the main purpose of its transaction is to ensure that the message successfully gets placed in the queue or removed from it completely. It is also possible to use solitary non-transactional queues with QC, but we will delay the discussion of these until the section "Transactional and Non-Transactional Queues."

After the Queued check box is clicked and the queues are created, FirstQC can be instantiated via a queue moniker and can begin receiving messages. Keep in mind, however, that it is the Recorder standing in as a proxy for the object that is getting the method calls from the client, turning them into messages, and storing them in the queue for later retrieval.

This brings us to the other check box shown in Figure 10.3. The second check box,Listen, seems simple enough, especially with the helpful sentence imprinted on the dialog box. This check box causes theListenerHelper to monitor the queues for the application. If theListenerHelper detects a message, it hands it to theListenerHelper object, which gives it to the Player, which creates a new instance of FirstQC and replays the method invocations made by the client.

One final, critical point to make is that FirstQC's parent application must be running for FirstQC toreceive any messages. To make certain that your application is running, simply right-click on it in the Component Services and select Start. Note that clients can still send messages via the Recorder even if the application is not running—they just aren't picked up and played to a new instance of the FirstQC until the application is next run. It might seem odd that the application needs to be explicitly run by the user or otherwise started using the COM+ administrative objects. After all, COM+ applications are normally run automatically when a client calls on one of its components. It is the asynchronous, monitoring nature of QC and MSMQ that make explicit startups a necessity.

Security settings paired with your specific NT domain topology and other administrative issues add another, more complicated layer to this process, which we'll explore in Chapter 12, "Security." For the examples in this chapter, let's explore the simplest possible QC scenario first, where security and other network considerations are removed. To cut out the security layer altogether, go to the Properties page for the FirstQCApp application and click on the Security property tab. Adjust the settings such that Authentication Level for Calls is set to None, as shown in Figure 10.5.

Figure 10.5. The Security settings tab for FirstQCApp.

You know that the Recorder translates method calls into MSMQ messages and puts them in an application's queue. If you are wondering exactly when the method calls are put into the queue by the Recorder, always remember that MSMQ is an RM. Therefore, by the rules outlined in Chapter 8, "Transactions," the message is placed in the queue when the transaction is committed—that is, when the root object is deactivated with its Happy bit set to TRUE. Given that your base client executable does not have a context and, therefore, does not have a transaction, COM+ automatically creates a new transaction when the Recorder is instantiated. If you check the transactional settings for the COM+ Recorder, you'll notice that it is set to Required. As you learned in Chapter 8, this setting means that the object inherits a transaction if its creator had one, or COM+ creates a new transaction for the object if it did not. Thus, all QC method calls are transactional, and the method calls are committed to the queue as messages when the Recorder object, the root object in this case, is deactivated and Happy.

Let's take a look at how method calls turn into messages. Imagine the following code exists in a simple VB client:

Dim QC As  FirstQC.FirstQueue
Sub Form_Load()
Set QC =  GetObject("queue:/new:FirstQC.FirstQueue") 
End Sub

Sub Command1_Click
QC.DisplayNumber 4
End Sub

Further, imagine that you perform the following steps:

Keeping in mind that because FirstQCApp is not running (you haven't explicitly started it), you can go to this application's queues and take a look at the messages. Using the Computer Management Console, you can navigate to FirstQCApp's queues, as shown in Figure 10.6.

Figure 10.6. Two messages in the FirstQCApp queue.

You might have expected 11 messages because you made 11 method calls. But, if you think about it, you'll find it makes perfect sense. As an RM, MSMQ attempts to commit a transaction when the root object completes its work. In our example, our base client made one method call and was closed, thus deactivating the Recorder object (the root object of the transaction) and forcing the transaction to commit. The result of this operation was one message.

Next we reran the application, made 10 method calls, and again forced the transaction to complete. It seems logical and efficient then that the 10 method calls should be bundled into a singular message.

There is one large caveat, however. If any single method call of the 10 in the message causes the recipient to crash or call SetAbort, the message isretried (according to the rules laid out in the upcoming sidebar "Timeout Periods and Retry Attempts for Failed QC Messages"). This message, however, contains 10 method invocations, and they are all played to the recipient again, not just the one that failed. This implies that your method calls should beidempotent, or at the very least, any data modifications performed by queued methods should be under the umbrella of the recipient's transaction, which given that the modifications are undone in the event of an aborted transaction, amounts to the same thing, more or less. These messages can, of course, be viewed. Double-click on the first one (or solitary method call), and you will see something like Figure 10.7.

Figure 10.7. The contents of the message resulting from a method call to FirstQC.

You don't need to understand the contents of the message, only that this is the binary representation of your queued method call. In the section, "Manually Reading the Message," I demonstrate how to take the contents of this message and put it into a queue directly via an ordinary, non-COM+ MSMQ client. For the moment, however, let's continue our exploration.

Because FirstQCApp is not running, method calls made by clients requesting queueable objects from this application simply pile up in the queues. So, by the simple action of right-clicking the application in the Component Services Console and selecting the Start menu item, you are ultimately greeted by 11 message boxes, some of which are minimized, as shown in Figure 10.8.

If you dismiss the 11 message boxes and look into FirstQCApp's queues, you find them empty. The messages have been detected by the Listener; and the Player was created, which then instantiated a FirstQC object and played the messages as method calls for the object. Messages are removed from the queue after they are successfully played for the recipient.

Figure 10.8. The methods are executed by the Player when FirstQCApp is run.

The algorithm used by the Recorder to translate method calls to messages is not documented. Assume that the following code is executed by a QC client:

Dim QC As  FirstQC.FirstQueue
Set QC = GetObject("queue:/new:FirstQC.FirstQueue")
QC.DisplayNumber 4

You can see the resulting message in FirstQCApp's queue by navigating to FirstQCApp's primary queue, viewing its messages section, and double-clicking the resulting message. You see a message similar to that shown in Figure 10.7.

We are going to simulate the preceding client-side code using an ordinary, non-transactional MSMQ client. For simplicity, I am assuming you are in Workgroup mode (that is, not using Active Directory). Follow these steps:

Now examine the contents of the queue as shown in Figure 10.10.

Figure 10.10. Contents of the new SecondQCApp queue.

Predictably, there is one message in the queue that corresponds to the method call you just made. But this queue can be read from or written to by MSMQ-aware applications, so it is legal to read and remove a QC-generated message using a pure MSMQ application. The code in Listing 10.6 demonstrates how this message can be read by an ordinary MSMQ application.

Dim qInfo As New MSMQQueueInfo
Dim qSource As MSMQQueue
Dim msgRec As MSMQMessage
Dim vntBody As Variant

'Specify the queue we will be reading from, in this case
'the non-transactional queue we created before. Note that
'if this queue was transactional, an attempt to open it
'would fail since a client application has no transaction
'associated with it.

'Note that this code assumes you are in Workgroup mode
'in other words authentication via Active Directory is not being used

qInfo.PathName = ". private$ MSMQTest"

'Open the queue and try to retrieve the message
Set qSource = qInfo.Open(MQ_RECEIVE_ACCESS, MQ_DENY_NONE)
Set msgRec = qSource.Receive(ReceiveTimeout:=1000)

If msgRec Is Nothing Then
    MsgBox "Nothing in Queue"
Else

    'dump the message body contents to a binary file:
    vntBody = msgRec.body
    Open "body.dat" For Binary As #1
    Put #1, , vntBody
    Close #1

    'dump the message extension contents to a binary file:
    vntBody = msgRec.Extension
    Open "extension.dat" For Binary As #1
    Put #1, , vntBody
    Close #1

End If

qSource.Close

The code in Listing 10.6 reads the message from the queue (thus removing it) and dumps the message body and extension into two binary files. The message extension is a new property in MSMQ 2.0 (the version that comes with Windows 2000) and is used to specify additional application-defined information that is associated with the message.

These two binary files completely describe the message that results in the method call you want. To illustrate this, you can construct a program that uses the binary files to package a message and put it back in the queue. Listing 10.7 demonstrates this process.

Dim qInfo As New MSMQQueueInfo
Dim qDest As MSMQQueue
Dim msgSend As New MSMQMessage
Dim vntBody As Variant

qInfo.PathName = ". private$ MSMQTest"
Set qDest = qInfo.Open(MQ_SEND_ACCESS, MQ_DENY_NONE)

'Construct the message by reading the binary files:

Open "body.dat" For Binary As #1
Get #1, , vntBody
Close #1
msgSend.body = vntBody

Open "extension.DAT" For Binary As #1
Get #1, , vntBody
Close #1
msgSend.Extension = vntBody

'Send the same message 5 times:
For k = 1 To 5
    msgSend.Send qDest
Next

qDest.Close

If the code in Listing 10.7 is run, an examination of SecondQCApp's queue reveals five new messages. If you were expecting one large message, remember that you do not have a transactional queue, and this is not a transactional client, so each Send corresponds to exactly one message. That said, it should come as no surprise that when SecondQCApp is started, five message boxes appear as the FirstQC object is instantiated and played its messages.

Although it is possible to examine the contents of body.dat and try to understand the binary layout of QC messages, such an effort would probably bear little fruit. The layout can always change, and if authentication were on, the contents would be encrypted.

Callbacks and acknowledgements are one thing, but failure is another. If you want to be explicitly notified when a queued method call fails to be delivered, you need only declare an exception class. The Properties dialog box of any COM+ component has a tab labeled Advanced. It looks like the one shown in Figure 10.12.

Type the ProgID (or ClassID) of an object that you want COM+ to call in the event of a failed message delivery. Most commonly, you set this property on both the sending and receiving applications on different machines. This is because COM+ triggers the exception class on the sending machine if it cannot reach the destination machine at all (perhaps the network is down); it triggers the exception on the destination machine if it can reach the destination, but ultimately it cannot play the message for the recipient.

In the event of a failed delivery, COM+ instantiates your exception class and then QIs for an interface calledIPlayBackControl. If your object implements this interface, COM+ calls one of its two methods, which follow:

Figure 10.12. The Advanced tab.

If you don't feel like peeking into a queue and analyzing the failed message, you can request that COM+ play the failed method call to your exception class. All your exception class needs to do is simply implement the queued interface containing the failed method call in question. In other words, you can write your exception class such that it is, in a sense, the recipient object's evil twin; if the method call fails to be played to the recipient object, then your exception class will receive the method call instead, just as if it were the originally targeted object. Of course, by receiving method calls originally slated for the recipient object, your implementation of these method calls would probably do little more than announce "This function was attempted but failed!"

For example, imagine some interface, IExample, is declared queued in some recipient object, and it has one method called Test(). If a client calls IExample->Test(), and the message resulting from that call fails to successfully play for the recipient, COM+ instantiates the event class you established for that object, and after QIing for IPlaybackControl, it then QIs for IExample .Predictably, the Player for your exception class invokes the Test() method on this interface of your exception class. So, when authoring such an event class and implementing IExample, you can write code to take whatever actions you deem appropriate given that you know exactly which method(s) failed to be delivered to the recipient.

An event class might support the IPlaybackControl, the queueable interface of the recipient, or both.

At this point, you might be envisioning schemes that allow the client executable to receive acknowledgements while it is running. Maybe you're thinking about writing a trade-entry front that allows the user to submit a trade via QC and soon after get back an acknowledgement that perhaps populates some control on the currently running form. For this type of application, however, COM+ events are better suited. As you see in the next chapter, COM+ events can use either synchronous Distributed COM (DCOM) or MSMQ as an asynchronous transport but provide a generic, extensible architecture for real-time acknowledgements.