Asynchronous queries in J2EE
by Kyle BrownSenior Technical Staff Member
IBM Software Services for WebSphere
Introduction
There's a servlet design problem so common that I hear it asked at least once a week, if not more often, in customer meetings, on Java newsgroups, or on forums like the JavaRanch or jGuru. The question, as it's often phrased is:
"How do I get my servlet to stop timing out on a really long database query?"
Regardless of how much database tuning effort you put into a project, there usually are one or two queries that end up taking much, much longer than you hope. In many cases, these especially complex queries may take up to a minute or longer to execute. The issue is that if you call a long query directly from a Servlet or JSP, the browser may time out before the call completes. When this happens, the user of your application will not see the result. There are commercial solutions to this problem (such as Async Beans in WebSphere Application Server, Enterprise Edition) but if you do not have access to those proprietary solutions, or need a solution that will be portable across multiple application servers, you are left to your own devices.
So, given this problem, developers often start casting about for a solution and usually end up finding out about using the <META> tag for server polling. This tag tells the client it must refresh the page after a number of seconds[1]. The number of seconds and the URL of the new page are specified in the CONTENT attribute of the tag:
<META http-equiv="Refresh" content="10; url=New-page.html">
So, once a developer determines that he can have the browser query the servlet on a regular basis to re-fetch a page, they quickly determine that the servlet can check for the value of a variable (perhaps in the HttpSession) to determine if the page returned will contain either the that the user requested, or a <META> tag that will display a "Please wait…" message and try fetching the page again later.
What goes along with this solution is usually a decision to use a separate thread to handle the long query so that the main thread can return the HTTP response as quickly as possible. While this has been a perfectly acceptable solution in the past, it introduces complications, both from a programming perspective, and from a specification perspective today. For instance, in WebSphere Application Server 5.0, if you attempt to access a database on a thread you have spawned yourself, you will see the following errors appear in the log:
4/11/03 11:53:41:711 PDT] 891a0
ConnectionMan W J2CA0075W: An active transaction should be present while
processing method allocateMCWrapper.
[4/11/03 11:53:41:815 PDT] 891a0 ConnectionMan W J2CA0075W: An active
transaction should be present while processing method initializeForUOW.[2]
According to a Technical note from WebSphere support "If a Servlet is spinning its own threads and accessing a database, the J2EE specification is not clear on this, so WebSphere Application Server 5.0 will allow it at this time. IBM is working with Sun to clarify this in the specification, so eventually (i.e. J2EE 1.4) spun threads from a Servlet accessing a database outside of a transaction will not be supported either". Later the same tech note states "Customers should consider changing their application to comply with the J2EE specification."
However, this is not the only problem that comes up with starting your own threads from a servlet. Not only will the transaction context not be defined, but the security context will not be defined either. Also, if the JVM running this thread dies, then the query is lost; even if your HttpSession is persistent, no other application server JVM in the cluster will know about the query that was running on the thread. In general, given the direction in which the J2EE specifications are heading, toward becoming more restrictive regarding thread creation, it's probably not a good idea to create your own threads from a Servlet in any case.
A J2EE-compliant solution for Queries
So, how can we manage to execute a long-running query asynchronously from a servlet that is intended to return the result of that query, and yet remain firmly within the bounds of good J2EE design? What you should examine instead of creating your own threads is using your application server's built-in internal JMS messaging (such as WebSphere's based on WebSphere MQ) to do the database access within the context of a Message-Driven Bean, and then have the MDB post a response to another queue which the Servlet would then query for the particular response. The way it works is that your servlet posts a "request" on a queue and returns to the user a page that then checks back later, either using a <META> tag or a Javascript script. When "later" happens, the servlet looks for the response in the queue and shows the result. The following diagram illustrates this process.
This diagram shows four main activities:
1. Requesting Servlet places the query on the request queue.
2. The Browser polls the servlet at a refresh interval.
3. Displaying Servlet checks the reply queue for the query response.
4. Query Processor MDB dequeues the query, performs it, and queues the result.
The pages returned by both the Requesting servlet and the Displaying Servlet (prior to the final display of the results) use the META tag to automatically request a refresh of the Displaying Servlet every few seconds. The browser polls the Displaying Servlet; each time it is invoked, the servlet checks for a result inside the JMS Queue.
The Query Processor then takes in the requests, executes the appropriate long-running database code, and then returns the response to the servlet through another queue. Query Processor should be implemented as a Message-Driven Bean, which has several advantages – first, since MDBs are EJBs, they tie together in a single transaction the receipt of the message on the queue and the processing of any database updates that execute within the onMessage() method. This means that if a database failure occurs that prevents processing of a request, that the request message will be placed back on the queue so that it may be tried again later; this helps with resolving transient errors like a database connection not being available. Second, MDB's within a cluster listening on the same queue act as "Competing Consumers" [Hohpe]. This means that all of the members of the cluster balance work of processing messages; if messages are not being processed fast enough, you can add more JVM's to the cluster in order to increase the processing power available.
Implementing multiple Queries with Commands
An important point to examine here is how you can implement your MDB in such a way that you don't need to have a different queue or MDB for each potential query that you run. For example, the simplest implementation of this pattern would simply involve pulling parameters off of a message, and then directly invoking a DAO with the parameters thus obtained. So, your implementation might look something like this:
public void onMessage(javax.jms.Message msg) {
TextMessage tMsg = (TextMessage) msg;
String customerNumber;
try {
customerNumber = tMsg.getText();
} catch (JMSException e) {
System.out.println("Exception caught: " + e);
fMessageDrivenCtx.setRollbackOnly();
return;
}
OrderDAO orderDAO = new OrderDAO();
Collection orders = orderDAO.findOrdersBy(customerNumber); // now send the response back by looking up the
// response queue and returning the collection...
}
However, the problem with this approach is that if we have another long running query (say, finding all Orders by the items they contain) then we'd need to write and deploy another MDB to invoke a different method on the DAO; something that would quickly become tedious as the number of queries increased. So, instead, a better approach is to use the Command pattern [Gamma]. In fact, this is also the "Command Message" pattern [Hohpe]. Our solution is as follows;
(1) Create a Command object in your Requesting Servlet that represents the particular query that you want to execute
(2) Send the Command object as an object message to the command processor MDB
(3) Invoke the execute() method on the command object, letting it execute its query inside the context of the MDB
(4) Obtain the result from the Command object and package it as a result on a response queue.
So, a more sophisticated example of our MDB, one that deals with as many query types as you may care to support, would look like the following:
public void onMessage(javax.jms.Message msg) {
Response response;
try {
ObjectMessage message = (ObjectMessage) msg;
ProcessingRequest request = (ProcessingRequest)
message.getObject();
response = request.execute();
} catch (Exception e) {
response = new ErrorResponse(e);
}
processResponse(response, msg);
}
To make this work, we need to implement our Commands so that they execute their long-running queries in response to the execute() message. This is a standard implementation of the Command pattern. In our case, let's consider just one of many possible Commands, an OrdersForCustomerRequest, as shown below:
Creating Commands in a Servlet
In order to understand how this Command will be used, let's begin at the Servlet that creates it, the OrderQueryRequestServlet:
public class OrderQueryRequestServlet
extends
DeferredDatabaseAccessServlet {
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
String
customerNumber =
(String)
req.getParameter("customerNumber");
OrdersForCustomerRequest
retrievalRequest =
new OrdersForCustomerRequest();
retrievalRequest.setCustomerNumber(customerNumber);
packageRequest(retrievalRequest, req);
RequestDispatcher disp = getServletContext().getRequestDispatcher("/PleaseWait.html");
disp.forward(req, resp);
}
}
This servlet would run in response to a PST request form an HTML page that includes one form parameter, the customerNumber that we are querying for. The flow of the servlet is simple; it obtains the parameter, creates a Command object (the OrdersForCustomerRequest) that will use that parameter, and then calls the packageRequest() method to manage the request. The packageRequest() method is the one that actually does the heavy lifting of placing that command on the request queue. Its implementation (from the superclass DeferredDatabaseAccessServlet) is shown below:
public void packageRequest(ProcessingRequest reqObj,
HttpServletRequest servletReq) {
try {
//Create a correlation id
String
correlationId =
Long.toString(System.currentTimeMillis());
//Look up the QueueConnectionFactory in JNDI
javax.jms.QueueSession session =
conn.createQueueSession(
false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
javax.jms.QueueSender sender =
session.createSender((Queue) requestQueue);
javax.jms.ObjectMessage replyMsg =
session.createObjectMessage(reqObj);
// Now need to set the correlation ID
replyMsg.setJMSCorrelationID(correlationId);
replyMsg.setJMSReplyTo(responseQueue);
// Finally send the message
sender.send(replyMsg);
session.close();
// Now set the correlation id in the session so
// the response servlet can find the message
HttpSession servletSession = servletReq.getSession();
servletSession.setAttribute("MessageCorrelationId",
correlationId);
} catch (Exception e) {
System.out.println("Could not process Response " + e);
}
}
Now we begin to get to some of the more interesting design points about implementing this pattern. Note that in this method we have used a JMS correlation id to uniquely identify this message. Correlation identifiers (as described in [Hohpe] and [Monson-Haefel]) are used to link related messages together. The most common use is exactly what we are doing here: to link a response with an earlier request, so that you can locate a particular response out of many unrelated responses on a shared queue. We'll return to this later after we trace the path of this particular command object through the rest of the system. Note that in this case we are simply using the current system time as our correlation identifier. If this application runs in a cluster (or just runs very quickly) it is possible, but improbable, that the same correlation identifier might be used for two messages. In a real system using this pattern, you should probably also append a random number to your correlation id to prevent this.
At this point, the Command object for our query has been placed on the outgoing (request) queue and awaits processing. After it has been placed on the queue, the requesting servlet places the message's correlation id in the HttpSession (we'll see why later) and then returns a page to the user that asks him to wait. This page is shown below:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<META HTTP-EQUIV="Refresh" CONTENT="3; URL=http://localhost:9080/DeferredExecutionMessageWeb/ResponseServlet">
<TITLE>PleaseWait.html</TITLE>
</HEAD>
<BODY>
<P>We are attempting to process your request. Please wait...</P>
</BODY>
</HTML>
As you can see, this page uses the meta tag described above to check in on the Servlet named ResponseServlet every 3 seconds. We'll visit this servlet later, but for now let's follow the trail of our Command message.
Processing Commands in an MDB
The next stop for our Command object is the MDB that processes all of the commands placed on the request queue. In our example, this is the DatabaseAccessProcessorBean. As with all MDB's, the action begins in the onMessage() method, where the message is received:
public void onMessage(javax.jms.Message msg) {
Response response;
try {
ObjectMessage message = (ObjectMessage) msg;
ProcessingRequest
request = (ProcessingRequest)
message.getObject();
response = request.execute();
} catch (Exception e) {
response = new ErrorResponse(e);
}
processResponse(response, msg);
}
Here the first thing that happens is that the Command object is extracted from the ObjectMessage created in the servlet. Once the Command object has been obtained, then the Command's execute() method can be invoked. Remember that execute() is polymorphic; in our case since this Command is an instance of the OrdersForCustomerRequest class, then this is the execute() method that runs:
public Response execute() {
// Note – this sleep() is ONLY to simulate a long running query!
// Do NOT do this in your own code!
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
}
// The method as it should be implemented really starts here.
Response response;
try {
OrdersForCustomerResponse
aResponse =
new OrdersForCustomerResponse();
aResponse.setCustomerNumber(customerNumber);
Collection results = getOrdersFor(customerNumber);
aResponse.setOrders(results);
response = aResponse;
} catch (OrderException e) {
ErrorResponse aResponse = new ErrorResponse(e);
response = aResponse;
}
return response;
}
Before moving much farther into this method, I have a confession to make. It's hard to simulate a long-running query in an example like this. Usually queries take a long time to execute because they are examining a large number of tables, or a large number of rows. In any case, this article is about how to deal with long running queries when they occur, not how to create a long-running query. So, in this example, I slowed down the execution of the Command by adding a Thread.sleep() for 10 seconds. This is not something you should normally do; in fact, doing this (or any sort of thread manipulation) is frowned upon by the EJB specification, even though it will execute in WAS 5.0.
Now that we're past that nastiness, though, we get to how the Command really runs. In this case, we construct a Response object (of the type OrdersForCustomerResponse), and then run our query. I'll not show the JDBC code to execute the query here, but if you are interested, you can download the example and look at the code. The point is that this is where you execute the long running query. The advantage of doing it here is that the EJB container is managing the thread in which this query runs, and what's more, you have control over whether or not you choose to execute the query within an EJB transaction (in which case the transaction could time out before the query completes) or outside of an EJB transaction within a Resource Manager Local Transaction (see the InfoCenter or Chapter 29 of [Brown] for information on RMLT's).
After the query completes, the collection of results is placed in the Response object, and is returned back to the onMessage() method, which then calls the processResponse() method, shown below:
private void processResponse(Response response, Message inMsg) {
try {
InitialContext ctx = new InitialContext();
Object o = ctx.lookup(
"java:comp/env/jms/QueueConnectionFactory");
qcf =
(QueueConnectionFactory)
javax.rmi.PortableRemoteObject.narrow(
o,
QueueConnectionFactory.class);
//Create a QueueConnection on the Connection Factory
//and this queue
qConn = qcf.createQueueConnection();
Destination replyQueue = inMsg.getJMSReplyTo();
//Look up the QueueConnectionFactory in JNDI
javax.jms.QueueSession session =
qConn.createQueueSession(
false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
javax.jms.QueueSender sender =
session.createSender((Queue) replyQueue);
javax.jms.ObjectMessage replyMsg =
session.createObjectMessage(response);
// Now need to set the correlation ID
replyMsg.setJMSCorrelationID(
inMsg.getJMSCorrelationID());
// Finally send the message
sender.send(replyMsg);
session.close();
qConn.close();
} catch (Exception e) {
System.out.println("Could not process Response " + e);
}
}
There isn't much new here, since this is nearly the same as the corresponding packageRequest() method in the DeferredDatabaseAccessServlet we saw earlier. Basically this method exists to place the response object in an ObjectMessage, and then place that ObjectMessage back on the response queue. The only two points of interest in this method are:
(1) Instead of hard-coding the queue name we've used the JMS reply-to queue feature, which, you'll recall, we set up in the packageRequest() method earlier. (By the way, this is called the "Return Address" pattern in [Hohpe].
(2) Also, we set the correlation id of the response message to be the same as the correlation id of the request message. This is what will enable our Response servlet to locate the right response method from the queue out of all the responses placed on the queue.
So, now that the response has been placed on the response queue, we're ready to examine our Response servlet, and determine how it handles looking for the response.
Checking for Responses
You'll remember that the <META> tag will poll the ResponseServlet every three seconds to see if a response has been returned. That polling invokes the following doGet() method:
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
try {
Response processingResponse = checkForMessages(req, resp);
req.setAttribute("receivedResponse",processingResponse);
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
getResponseURL(processingResponse));
disp.forward(req, resp);
} catch (ProcessingException e) {
log("ProcessingException + " + e);
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
"/Error.html");
disp.forward(req, resp);
} catch (NoRequestException e) {
log("NoReplyException + " + e);
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
"/Error.html");
disp.forward(req, resp);
} catch (ReplyTimeoutException e) {
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
"/PleaseWait.html");
disp.forward(req, resp);
}
}
This method begins by checking for messages from the queue. We'll look at that method in a moment, but for the time being, let's just examine what happens after the method checkForMessages() runs. That method will either return a response, or throw one of several possible exceptions. Obviously, if a problem occurs (indicated by a ProcessingException) then that must be handled appropriately by displaying an error page. The more interesting case is where the checkForMessages() method throws a ReplyTimeoutException, which is what happens when the reply message has not been received on the response queue. In that case, it redisplays the Please Wait page shown above, and the browser will then re-query the response servlet 3 seconds later. If a Response is returned, it will locate the appropriate URL of the JSP or Servlet to display those results and forward to that URL.
So, now that you understand how responses are handled, let's look at the implementation of the checkForMessages() method and see how it checks the queue for responses.
public Response checkForMessages(HttpServletRequest req,
HttpServletResponse resp)
throws
ProcessingException, NoReplyException,
ReplyTimeoutException {
Response responseObj = null;
try {
HttpSession httpSess = req.getSession(true);
String id = (String)
httpSess.getAttribute("MessageCorrelationId");
if (id == null)
throw new NoRequestException(
"Not Waiting on a Message");
QueueSession session =
conn.createQueueSession(
false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
String selector = "JMSCorrelationID = '" + id + "'";
QueueReceiver rcvr =
session.createReceiver(responseQueue, selector);
Message receivedMsg = rcvr.receiveNoWait();
if (receivedMsg == null) {
throw new ReplyTimeoutException(
"No Reply Message Received");
} else {
ObjectMessage inMsg = (ObjectMessage)
receivedMsg;
responseObj = (Response) inMsg.getObject();
}
session.close();
return responseObj;
} catch (JMSException e) {
throw new ProcessingException(e);
}
}
The first possible thing that can go wrong in this method is that the ResponseServlet has somehow been invoked when this user does not have any outstanding requests. The first few lines look in the User's HttpSession for an identifier named "CorrelationID", which should have been placed there by the servlet that originally made the request. If it is absent, then this is an error, and the NoRequestException is thrown.
After guaranteeing that there is an outstanding request for this user, then what happens is that the method creates a QueueReceiver on the response queue and then sets a message selector on the Queue. The message selector in JMS (see [Monson-Haefel]) is an implementation of the "Selective Consumer" pattern from [Hohpe]. What it does is to only return back those messages on the queue that have headers that match the pattern entered as the message selector. In this case, we are scanning for messages that have a correlation identifier matching the one held in the HttpSession (e.g. only responses for the last request made by this user). If there is a message on the queue matching that query, it will be returned from the receiveNoWait() method. If not, that method will immediately return null. A null return value means that the request is still being processed, so the method then throws the ReplyTimeoutException.
This is the last method of our implementation to review, and it brings up the last JMS design point to consider. In our design we've chosen to use a fixed request queue and a fixed response queue combined with a message selector on the response's correlation id to find only those messages that belong to a particular user. Another option would have been to use JMS temporary queues. In that case, we could have created a temporary queue for the response back in the packageRequest() method and then placed that temporary queue in the HttpSession. The problem with that approach is that it won't survive failure of the JVM in which the message was received. Since JMS temporary queues are not serializable, if the HttpSession is lost then the queue (and all of its messages) are lost as well. The message selector approach will work regardless of whether or not the JVM that receives the response is the same as the one that originally placed the request, since the message selector (being only a string) is easily serializable.[3] One thing that your designs should deal with, however, is what to do in case of VERY long queries, where the time it takes to execute the query is greater than the session timeout value, or when the user loses interest and closes the browser prior to the display servlet being able to fetch back the response message from the queue. [Hohpe] discusses strategies for clearing queues in situations like this.
Finally, there is an additional downside to the message selector approach. Many JMS providers (such WAS embedded messaging) do not have indexing capabilities. For a large number of queries, the use of a message selector could lead to poor performance due to the inefficiency of selecting out the required message. Thus, there's a potential tradeoff between performance and scalability that you need to carefully evaluate.
An Alternate User Interface Choice
What we have shown in this article is a mechanism for allowing each user to have a single outstanding request that is placed on a shared queue. However, what happens if you have not one, but several very long queries? And what if your users would like these queries to be able to run simultaneously instead of one at a time? Luckily, our basic Command-driven design can be adapted to solve that problem as well. In our design we've chosen to place a single correlation id in the HttpSession. However, it's not much of a stretch to imagine that you could instead place a collection of correlation id's in the HttpSession, with each one representing an independent query. In that case, what you would need to implement (instead of a single "please wait…" page) would be a page that shows the status of several queries at once. As the results are returned on the queue, the polled servlet that generates that page would add the completed query results into the HttpSession as well. The servlet would then indicate by a link which queries are completed, and thus available for review, and which ones are still in progress.
I will warn you, however, that if your problems are getting this sophisticated, that you should probably look into a commercial workflow tool like the WebSphere Process Choreographer (part of WebSphere Enterprise Edition) to solve this problem. At this point this is no longer a simple issue of avoiding threads; it's become a more complex problem.
Summary
In this article we've covered a lot of ground, but in essence only conveyed one simple idea. Rather than spinning off your own threads to handle long-running queries, it's best to let the container manage the threads by taking advantage of JMS messaging and Message-Driven Beans. Hopefully you now have the tools necessary to build your own systems using this approach, and can take advantage of the benefits of letting containers manage your threads.
Acknowledgements
Thanks to Bobby Woolf for his insightful comments in improving this article, and for catching a couple of technical errors as well. Also, thanks to Roland Barcia and Wayne Beaton for their suggestions for improvement to the article.
Bibliography
[Barcia] Roland Barcia, "JMS Application Architectures", The ServerSide, available at: http://www.theserverside.com/user/login.jsp?nextpage=/articles/content/JMSArchitecture/JMSApplicationArchitectures.pdf
[Brown] Kyle Brown et. al., "Enterprise Java Programming with IBM WebSphere, 2nd Edition", IBM Press, 2004
[Brown 2003] Kyle Brown and Keys Botzum, "Improving HttpSession Performance through Smart Serialization", WebSphere Developer's Domain, December 2003
[Gamma] Erich Gamma, et. al., "Design Patterns, Elements of Reusable Object-Oriented Software", Addison-Wesley, 1995
[Hohpe] Gregor Hohpe and Bobby Woolf, "Enterprise Integration Patterns: Designing, Buidling and Debugging Messaging Systems", Addison-Wesley, 2003
[Monson-Haefel] Richard Monson-Haefel, "Java Message Service", O'Reilly & Associates, 2001