Asynchronous queries in J2EE
by Kyle BrownSenior Technical Staff Member
IBM Software Services for WebSphere
Introduction
There's a servlet design problem so common that I hear it
asked at least once a week, if not more often, in customer meetings, on Java
newsgroups, or on forums like the JavaRanch or jGuru. The question, as it's
often phrased is:
"How do I get my servlet to stop timing out on a really long
database query?"
Regardless of how much database tuning effort you put into a
project, there usually are one or two queries that end up taking much, much
longer than you hope. In many cases, these especially complex queries may take
up to a minute or longer to execute. The issue is that if you call a long
query directly from a Servlet or JSP, the browser may time out before the call
completes. When this happens, the user of your application will not see the
result. There are commercial solutions to this problem (such as Async Beans in
WebSphere Application Server, Enterprise Edition) but if you do not have access
to those proprietary solutions, or need a solution that will be portable across
multiple application servers, you are left to your own devices.
So, given this problem, developers often start casting about
for a solution and usually end up finding out about using the <META> tag
for server polling. This tag tells the client it must refresh the page after a
number of seconds.
The number of seconds and the URL of the new page are specified in the CONTENT
attribute of the tag:
<META http-equiv="Refresh" content="10; url=New-page.html">
So, once a developer determines that he can have the browser
query the servlet on a regular basis to re-fetch a page, they quickly determine
that the servlet can check for the value of a variable (perhaps in the
HttpSession) to determine if the page returned will contain either the that the
user requested, or a <META> tag that will display a "Please wait?"
message and try fetching the page again later.
What goes along with this solution is usually a decision to use
a separate thread to handle the long query so that the main thread can return
the HTTP response as quickly as possible. While this has been a perfectly
acceptable solution in the past, it introduces complications, both from a
programming perspective, and from a specification perspective today. For
instance, in WebSphere Application Server 5.0, if you attempt to access a
database on a thread you have spawned yourself, you will see the following
errors appear in the log:
4/11/03 11:53:41:711 PDT] 891a0
ConnectionMan W J2CA0075W: An active transaction should be present while
processing method allocateMCWrapper.
[4/11/03 11:53:41:815 PDT] 891a0 ConnectionMan W J2CA0075W: An active
transaction should be present while processing method initializeForUOW.
According to a Technical note from WebSphere support "If a
Servlet is spinning its own threads and accessing a database, the J2EE
specification is not clear on this, so WebSphere Application Server 5.0 will
allow it at this time. IBM is working with Sun to clarify this in the
specification, so eventually (i.e. J2EE 1.4) spun threads from a Servlet
accessing a database outside of a transaction will not be supported either".
Later the same tech note states "Customers should consider changing their
application to comply with the J2EE specification."
However, this is not the only problem that comes up with
starting your own threads from a servlet. Not only will the transaction
context not be defined, but the security context will not be defined either. Also,
if the JVM running this thread dies, then the query is lost; even if your
HttpSession is persistent, no other application server JVM in the cluster will
know about the query that was running on the thread. In general, given the
direction in which the J2EE specifications are heading, toward becoming more
restrictive regarding thread creation, it's probably not a good idea to create
your own threads from a Servlet in any case.
A J2EE-compliant solution for Queries
So, how can we manage to execute a long-running query
asynchronously from a servlet that is intended to return the result of that
query, and yet remain firmly within the bounds of good J2EE design? What you
should examine instead of creating your own threads is using your application
server's built-in internal JMS messaging (such as WebSphere's based on
WebSphere MQ) to do the database access within the context of a Message-Driven
Bean, and then have the MDB post a response to another queue which the Servlet
would then query for the particular response. The way it works is that your
servlet posts a "request" on a queue and returns to the user a page
that then checks back later, either using a <META> tag or a Javascript
script. When "later" happens, the servlet looks for the response in
the queue and shows the result.
The following diagram illustrates this process.
This diagram shows four main activities:
1. Requesting
Servlet places the query on the request queue.
2. The
Browser polls the servlet at a refresh interval.
3. Displaying
Servlet checks the reply queue for the query response.
4. Query
Processor MDB dequeues the query, performs it, and queues the result.
The pages returned by both the Requesting servlet and the
Displaying Servlet (prior to the final display of the results) use the META tag to automatically request a refresh of the Displaying Servlet every few seconds.
The browser polls the Displaying Servlet; each time it is invoked, the servlet
checks for a result inside the JMS Queue.
The Query Processor then takes in the requests, executes the
appropriate long-running database code, and then returns the response to the
servlet through another queue. Query Processor should be implemented as a
Message-Driven Bean, which has several advantages ? first, since MDBs are EJBs,
they tie together in a single transaction the receipt of the message on the
queue and the processing of any database updates that execute within the onMessage() method. This means that
if a database failure occurs that prevents processing of a request, that the
request message will be placed back on the queue so that it may be tried again
later; this helps with resolving transient errors like a database connection
not being available. Second, MDB's within a cluster listening on the same
queue act as "Competing Consumers" [Hohpe]. This means that all of the
members of the cluster balance work of processing messages; if messages are not
being processed fast enough, you can add more JVM's to the cluster in order to
increase the processing power available.
Implementing multiple Queries with Commands
An important point to examine here is how you can implement your
MDB in such a way that you don't need to have a different queue or MDB for each
potential query that you run. For example, the simplest implementation of this
pattern would simply involve pulling parameters off of a message, and then
directly invoking a DAO with the parameters thus obtained. So, your
implementation might look something like this:
public void onMessage(javax.jms.Message
msg) {
TextMessage
tMsg = (TextMessage) msg;
String
customerNumber;
try
{
customerNumber
= tMsg.getText();
} catch (JMSException
e) {
System.out.println("Exception caught: " + e);
fMessageDrivenCtx.setRollbackOnly();
return;
}
OrderDAO
orderDAO = new OrderDAO();
Collection
orders = orderDAO.findOrdersBy(customerNumber); // now send the response back by looking up the
// response
queue and returning the collection...
}
However, the problem with this approach is that if we have
another long running query (say, finding all Orders by the items they contain)
then we'd need to write and deploy another MDB to invoke a different method on
the DAO; something that would quickly become tedious as the number of queries
increased. So, instead, a better approach is to use the Command pattern
[Gamma]. In fact, this is also the "Command Message" pattern [Hohpe]. Our
solution is as follows;
(1) Create a Command
object in your Requesting Servlet that represents the particular query that you
want to execute
(2) Send the Command
object as an object message to the command processor MDB
(3) Invoke the execute()
method on the command object, letting it execute its query inside the context
of the MDB
(4) Obtain the result from
the Command object and package it as a result on a response queue.
So, a more sophisticated example of our MDB, one that deals
with as many query types as you may care to support, would look like the
following:
public void onMessage(javax.jms.Message
msg) {
Response
response;
try
{
ObjectMessage
message = (ObjectMessage) msg;
ProcessingRequest
request = (ProcessingRequest)
message.getObject();
response
= request.execute();
} catch (Exception
e) {
response
= new
ErrorResponse(e);
}
processResponse(response,
msg);
}
To make this work, we need to implement our Commands so that
they execute their long-running queries in response to the execute() message. This is a standard
implementation of the Command pattern. In our case, let's consider just one
of many possible Commands, an OrdersForCustomerRequest,
as shown below:
Creating Commands in a Servlet
In order to understand how this Command will be used, let's
begin at the Servlet that creates it, the OrderQueryRequestServlet:
public class OrderQueryRequestServlet
extends
DeferredDatabaseAccessServlet {
public void doPost(HttpServletRequest
req, HttpServletResponse
resp)
throws
ServletException, IOException
{
String
customerNumber =
(String)
req.getParameter("customerNumber");
OrdersForCustomerRequest
retrievalRequest =
new OrdersForCustomerRequest();
retrievalRequest.setCustomerNumber(customerNumber);
packageRequest(retrievalRequest,
req);
RequestDispatcher disp = getServletContext().getRequestDispatcher("/PleaseWait.html");
disp.forward(req,
resp);
}
}
This servlet would run in response to a PST request form an
HTML page that includes one form parameter, the customerNumber that we are
querying for. The flow of the servlet is simple; it obtains the parameter,
creates a Command object (the OrdersForCustomerRequest)
that will use that parameter, and then calls the packageRequest()
method to manage the request. The packageRequest()
method is the one that actually does the heavy lifting of placing that
command on the request queue. Its implementation (from the superclass DeferredDatabaseAccessServlet) is
shown below:
public void packageRequest(ProcessingRequest
reqObj,
HttpServletRequest
servletReq) {
try
{
//Create a correlation id
String
correlationId =
Long.toString(System.currentTimeMillis());
//Look
up the QueueConnectionFactory in JNDI
javax.jms.QueueSession
session =
conn.createQueueSession(
false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
javax.jms.QueueSender
sender =
session.createSender((Queue) requestQueue);
javax.jms.ObjectMessage
replyMsg =
session.createObjectMessage(reqObj);
//
Now need to set the correlation ID
replyMsg.setJMSCorrelationID(correlationId);
replyMsg.setJMSReplyTo(responseQueue);
//
Finally send the message
sender.send(replyMsg);
session.close();
// Now set the correlation id in the session
so
// the response servlet can find the message
HttpSession
servletSession =
servletReq.getSession();
servletSession.setAttribute("MessageCorrelationId",
correlationId);
}
catch (Exception
e) {
System.out.println("Could not
process Response " + e);
}
}
Now we begin to get to some of the more interesting design
points about implementing this pattern. Note that in this method we have used
a JMS correlation id to uniquely identify this message. Correlation identifiers
(as described in [Hohpe] and [Monson-Haefel]) are used to link related messages
together. The most common use is exactly what we are doing here: to link a
response with an earlier request, so that you can locate a particular response
out of many unrelated responses on a shared queue. We'll return to this later
after we trace the path of this particular command object through the rest of
the system. Note that in this case we are simply using the current system time
as our correlation identifier. If this application runs in a cluster (or just
runs very quickly) it is possible, but improbable, that the same correlation
identifier might be used for two messages. In a real system using this
pattern, you should probably also append a random number to your correlation id
to prevent this.
At this point, the Command object for our query has been
placed on the outgoing (request) queue and awaits processing. After it has
been placed on the queue, the requesting servlet places the message's
correlation id in the HttpSession
(we'll see why later) and then returns a page to the user that asks him to
wait. This page is shown below:
<!DOCTYPE
HTML PUBLIC "-//W3C//DTD HTML 4.01
Transitional//EN">
<HTML>
<HEAD>
<META http-equiv="Content-Type"
content="text/html;
charset=ISO-8859-1">
<META
HTTP-EQUIV="Refresh" CONTENT="3;
URL=http://localhost:9080/DeferredExecutionMessageWeb/ResponseServlet">
<TITLE>PleaseWait.html</TITLE>
</HEAD>
<BODY>
<P>We
are attempting to process your request. Please wait...</P>
</BODY>
</HTML>
As you can see, this page uses the meta tag described above
to check in on the Servlet named ResponseServlet
every 3 seconds. We'll visit this servlet later, but for now let's follow the
trail of our Command message.
Processing Commands in an MDB
The next stop for our Command object is the MDB that
processes all of the commands placed on the request queue. In our example,
this is the DatabaseAccessProcessorBean.
As with all MDB's, the action begins in the onMessage()
method, where the message is received:
public void onMessage(javax.jms.Message
msg) {
Response
response;
try
{
ObjectMessage
message = (ObjectMessage) msg;
ProcessingRequest
request = (ProcessingRequest)
message.getObject();
response
= request.execute();
} catch (Exception
e) {
response
= new
ErrorResponse(e);
}
processResponse(response,
msg);
}
Here the first thing that happens is that the Command object
is extracted from the ObjectMessage
created in the servlet. Once the Command object has been obtained, then the
Command's execute() method can
be invoked. Remember that execute()
is polymorphic; in our case since this Command is an instance of the OrdersForCustomerRequest class, then
this is the execute() method
that runs:
public Response execute() {
// Note ? this sleep() is ONLY to simulate
a long running query!
// Do NOT do this in your own code!
try
{
Thread.sleep(10000);
} catch (InterruptedException
e) {
}
// The method as it should be implemented really
starts here.
Response
response;
try
{
OrdersForCustomerResponse
aResponse =
new OrdersForCustomerResponse();
aResponse.setCustomerNumber(customerNumber);
Collection
results = getOrdersFor(customerNumber);
aResponse.setOrders(results);
response
= aResponse;
} catch (OrderException
e) {
ErrorResponse
aResponse = new ErrorResponse(e);
response
= aResponse;
}
return
response;
}
Before moving much farther into this method, I have a
confession to make. It's hard to simulate a long-running query in an example
like this. Usually queries take a long time to execute because they are examining
a large number of tables, or a large number of rows. In any case, this article
is about how to deal with long running queries when they occur, not how to
create a long-running query. So, in this example, I slowed down the execution
of the Command by adding a Thread.sleep()
for 10 seconds. This is not something you should normally do; in fact, doing
this (or any sort of thread manipulation) is frowned upon by the EJB
specification, even though it will execute in WAS 5.0.
Now that we're past that nastiness, though, we get to how
the Command really runs. In this case, we construct a Response object (of the
type OrdersForCustomerResponse),
and then run our query. I'll not show the JDBC code to execute the query here,
but if you are interested, you can download the example and look at the code.
The point is that this is where you execute the long running query. The
advantage of doing it here is that the EJB container is managing the thread in
which this query runs, and what's more, you have control over whether or not
you choose to execute the query within an EJB transaction (in which case the
transaction could time out before the query completes) or outside of an EJB
transaction within a Resource Manager Local Transaction (see the InfoCenter or
Chapter 29 of [Brown] for information on RMLT's).
After the query completes, the collection of results is
placed in the Response object, and is returned back to the onMessage() method, which then calls
the processResponse() method,
shown below:
private void processResponse(Response
response, Message
inMsg) {
try
{
InitialContext
ctx = new InitialContext();
Object
o = ctx.lookup(
"java:comp/env/jms/QueueConnectionFactory");
qcf
=
(QueueConnectionFactory)
javax.rmi.PortableRemoteObject.narrow(
o,
QueueConnectionFactory.class);
//Create a QueueConnection on the Connection Factory
//and this queue
qConn
= qcf.createQueueConnection();
Destination
replyQueue = inMsg.getJMSReplyTo();
//Look
up the QueueConnectionFactory in JNDI
javax.jms.QueueSession
session =
qConn.createQueueSession(
false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
javax.jms.QueueSender
sender =
session.createSender((Queue) replyQueue);
javax.jms.ObjectMessage
replyMsg =
session.createObjectMessage(response);
//
Now need to set the correlation ID
replyMsg.setJMSCorrelationID(
inMsg.getJMSCorrelationID());
//
Finally send the message
sender.send(replyMsg);
session.close();
qConn.close();
} catch (Exception
e) {
System.out.println("Could not process Response " + e);
}
}
There isn't much new here, since this is nearly the same as
the corresponding packageRequest()
method in the DeferredDatabaseAccessServlet
we saw earlier. Basically this method exists to place the response object in
an ObjectMessage, and then place
that ObjectMessage back on the
response queue. The only two points of interest in this method are:
(1) Instead of hard-coding
the queue name we've used the JMS reply-to queue feature, which, you'll recall,
we set up in the packageRequest() method
earlier. (By the way, this is called the "Return Address" pattern in [Hohpe].
(2) Also, we set the
correlation id of the response message to be the same as the correlation id of
the request message. This is what will enable our Response servlet to locate
the right response method from the queue out of all the responses placed on the
queue.
So, now that the response has been placed on the response
queue, we're ready to examine our Response servlet, and determine how it
handles looking for the response.
Checking for Responses
You'll remember that the <META> tag will poll the ResponseServlet every three seconds to
see if a response has been returned. That polling invokes the following doGet() method:
public void doGet(HttpServletRequest
req, HttpServletResponse
resp)
throws
ServletException, IOException
{
try
{
Response
processingResponse =
checkForMessages(req, resp);
req.setAttribute("receivedResponse",processingResponse);
RequestDispatcher
disp =
getServletContext().getRequestDispatcher(
getResponseURL(processingResponse));
disp.forward(req,
resp);
}
catch (ProcessingException
e) {
log("ProcessingException +
" + e);
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
"/Error.html");
disp.forward(req, resp);
}
catch (NoRequestException
e) {
log("NoReplyException + "
+ e);
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
"/Error.html");
disp.forward(req, resp);
}
catch (ReplyTimeoutException
e) {
RequestDispatcher disp =
getServletContext().getRequestDispatcher(
"/PleaseWait.html");
disp.forward(req, resp);
}
}
This method begins by checking for messages from the queue.
We'll look at that method in a moment, but for the time being, let's just
examine what happens after the method checkForMessages()
runs. That method will either return a response, or throw one of several
possible exceptions. Obviously, if a problem occurs (indicated by a ProcessingException) then that must be
handled appropriately by displaying an error page. The more interesting case
is where the checkForMessages()
method throws a ReplyTimeoutException,
which is what happens when the reply message has not been received on the
response queue. In that case, it redisplays the Please Wait page shown above,
and the browser will then re-query the response servlet 3 seconds later. If a
Response is returned, it will locate the appropriate URL of the JSP or Servlet
to display those results and forward to that URL.
So, now that you understand how responses are handled, let's
look at the implementation of the checkForMessages()
method and see how it checks the queue for responses.
public Response checkForMessages(HttpServletRequest
req,
HttpServletResponse
resp)
throws
ProcessingException, NoReplyException,
ReplyTimeoutException {
Response
responseObj =
null;
try
{
HttpSession
httpSess = req.getSession(true);
String id = (String)
httpSess.getAttribute("MessageCorrelationId");
if
(id == null)
throw new
NoRequestException(
"Not Waiting on a Message");
QueueSession
session =
conn.createQueueSession(
false,
javax.jms.Session.AUTO_ACKNOWLEDGE);
String
selector = "JMSCorrelationID = '" + id + "'";
QueueReceiver
rcvr =
session.createReceiver(responseQueue, selector);
Message
receivedMsg =
rcvr.receiveNoWait();
if
(receivedMsg ==
null)
{
throw new
ReplyTimeoutException(
"No Reply Message Received");
}
else {
ObjectMessage inMsg = (ObjectMessage)
receivedMsg;
responseObj = (Response) inMsg.getObject();
}
session.close();
return
responseObj;
} catch (JMSException
e) {
throw
new ProcessingException(e);
}
}
The first possible thing that can go wrong in this method is
that the ResponseServlet has
somehow been invoked when this user does not have any outstanding requests.
The first few lines look in the User's HttpSession
for an identifier named "CorrelationID", which should have been placed there by
the servlet that originally made the request. If it is absent, then this is an
error, and the NoRequestException
is thrown.
After guaranteeing that there is an outstanding request for
this user, then what happens is that the method creates a QueueReceiver on the response queue
and then sets a message selector on the Queue. The message selector in JMS
(see [Monson-Haefel]) is an implementation of the "Selective Consumer" pattern
from [Hohpe]. What it does is to only return back those messages on the queue
that have headers that match the pattern entered as the message selector. In
this case, we are scanning for messages that have a correlation identifier
matching the one held in the HttpSession
(e.g. only responses for the last request made by this user). If there is a
message on the queue matching that query, it will be returned from the receiveNoWait() method. If not, that
method will immediately return null. A null return value means that the
request is still being processed, so the method then throws the ReplyTimeoutException.
This is the last method of our implementation to review, and
it brings up the last JMS design point to consider. In our design we've chosen
to use a fixed request queue and a fixed response queue combined with a message
selector on the response's correlation id to find only those messages that
belong to a particular user. Another option would have been to use JMS
temporary queues. In that case, we could have created a temporary queue for
the response back in the packageRequest()
method and then placed that temporary queue in the HttpSession. The problem with that approach is that it
won't survive failure of the JVM in which the message was received. Since JMS
temporary queues are not serializable, if the HttpSession
is lost then the queue (and all of its messages) are lost as well. The message
selector approach will work regardless of whether or not the JVM that receives
the response is the same as the one that originally placed the request, since
the message selector (being only a string) is easily serializable.
One thing that your designs should deal with, however, is what to do in case
of VERY long queries, where the time it takes to execute the query is greater
than the session timeout value, or when the user loses interest and closes the
browser prior to the display servlet being able to fetch back the response
message from the queue. [Hohpe] discusses strategies for clearing queues in
situations like this.
Finally, there is an additional downside to the message
selector approach. Many JMS providers (such WAS embedded messaging) do not
have indexing capabilities. For a large number of queries, the use of a
message selector could lead to poor performance due to the inefficiency of
selecting out the required message. Thus, there's a potential tradeoff between
performance and scalability that you need to carefully evaluate.
An Alternate User Interface Choice
What we have shown in this article is a mechanism for
allowing each user to have a single outstanding request that is placed on a shared
queue. However, what happens if you have not one, but several very long
queries? And what if your users would like these queries to be able to run
simultaneously instead of one at a time? Luckily, our basic Command-driven
design can be adapted to solve that problem as well. In our design we've
chosen to place a single correlation id in the HttpSession. However, it's not
much of a stretch to imagine that you could instead place a collection of
correlation id's in the HttpSession, with each one representing an independent
query. In that case, what you would need to implement (instead of a single
"please wait?" page) would be a page that shows the status of several queries
at once. As the results are returned on the queue, the polled servlet that generates
that page would add the completed query results into the HttpSession as well.
The servlet would then indicate by a link which queries are completed, and thus
available for review, and which ones are still in progress.
I will warn you, however, that if your problems are getting
this sophisticated, that you should probably look into a commercial workflow
tool like the WebSphere Process Choreographer (part of WebSphere Enterprise
Edition) to solve this problem. At this point this is no longer a simple issue
of avoiding threads; it's become a more complex problem.
Summary
In this article we've covered a lot of ground, but in
essence only conveyed one simple idea. Rather than spinning off your own
threads to handle long-running queries, it's best to let the container manage
the threads by taking advantage of JMS messaging and Message-Driven Beans.
Hopefully you now have the tools necessary to build your own systems using this
approach, and can take advantage of the benefits of letting containers manage
your threads.
Acknowledgements
Thanks to Bobby Woolf for his insightful comments in
improving this article, and for catching a couple of technical errors as well.
Also, thanks to Roland Barcia and Wayne Beaton for their suggestions for
improvement to the article.
Bibliography
[Barcia] Roland Barcia, "JMS Application Architectures", The
ServerSide, available at: http://www.theserverside.com/user/login.jsp?nextpage=/articles/content/JMSArchitecture/JMSApplicationArchitectures.pdf
[Brown] Kyle Brown et. al., "Enterprise Java Programming
with IBM WebSphere, 2nd Edition", IBM Press, 2004
[Brown 2003] Kyle Brown and Keys Botzum, "Improving
HttpSession Performance through Smart Serialization", WebSphere Developer's
Domain, December 2003
[Gamma] Erich Gamma, et. al., "Design Patterns, Elements of
Reusable Object-Oriented Software", Addison-Wesley, 1995
[Hohpe] Gregor Hohpe and Bobby Woolf, "Enterprise
Integration Patterns: Designing, Buidling and Debugging Messaging Systems",
Addison-Wesley, 2003
[Monson-Haefel] Richard Monson-Haefel, "Java Message
Service", O'Reilly & Associates, 2001