Alternate async proposal premise rebuttal

The Alternate Proposal For Async Servlets is based on several premises that are disputed and some have been demonstrated to be incorrect.

The most common use of suspend and resume is really to manipulate the response in a delayed fashion without blocking the container.

This is one of only three general use cases for asynchronous handling and it is not clear that it is the "most common use". The three general use-cases are to suspend (or threadlessly wait) BEFORE, DURING or AFTER normal servlet processing.

BEFORE The use-case to suspend before normal servlet processing is to wait for an available resource or to make other Quality of Service handling. A key example of this is avoiding many threads waiting on a JDBC connection pool. A request is suspended when there are no available JDBC connections and resumes when a JDBC connection becomes available. Once resume, the request can proceed through normal servlet processing and no framework/application code needs to be altered.

DURING The use-case to suspend during normal servlet processing is to call remote webservices. In the example linked below, normal JSF processing is used to handle a post request and initiate an asynchronous web services request to ebay. The request is then suspended until the web services response is received. The request is then resumed and the JSF phases are continued with normal handling to produce the response. Typically frameworks/applications need some minor modifications to allow state to be restored (see JSF ebay example below).

AFTER The use-cases to suspend after normal servlet processing is for new protocols like comet or for proxying remote servlets. After the request is handled by the servlet, the request is suspended so that it is not committed by the container. Asynchronous threads and services may then generate the response as events occur or remote data is received. Typically the content will be generated by newly written frameworks/applications. This is the only use-case supported by the alternate proposal.

Feedback received from Amazon, Ebay, LinkedIn, Paypal and Yahoo indicate that they are all very much interested in the BEFORE and DURING use cases. Such large websites as must deal with the scaling issues of limited resources (eg JDBC connections) and slow remote web services. All have implemented proprietary extensions for these use-cases, but are interested in a standards based solutions. Enterprises without the resources of such companies should not be forced to implement proprietary solutions to achieve scalability.

''Re-dispatching of request does not help application and framework authors. It is a distraction and adds more complexity to the application in addition to potentially breaking existing applications.
''

Re-dispatching is not a new feature for the servlet specification. Requests are already redispatched by the error page mechanism for exceptions that are caught by the container. Many frameworks (eg sitemesh) redispatch the request multiple times to down stream processing. RequestDispatchers also create the possibility that filters and servlets can see the same request more than once.

The asynchronous features may break some applications, but only if the developer chooses to use them. As described below, a large amount of code will be able to be used with suspend/resume without any changes.

If a component calls suspend on the request object and if JSF is in the process of building the component tree then it would be the responsibility of JSF in this case to store the state it is in before spawning of a separate thread. To achieve this the framework would have to wrap the Request and Response objects so it can capture the state so that when the request is resumed and redispatched it can restore the components and then continue processing.

Wrapping the request is not necessary to save state. A simple patch has been written for the mojarra JSF implementation. These 36 lines allow any phase of JSF processing to suspend a request and that phase will be re-executed when the request is resumed. The component tree is saved as a request attribute when the request is suspended and recovered when the request is resumed. This allows application code to suspend during phase processing or during rendering. The example provided with Jetty-7 uses this patch to suspend during an asynchronous REST call to the ebay webservices, so that a thread need not be held during the several hundred millisecond wait for a response.

The alternative proposal does not work at all with JSF.

Insufficient example for the Alternate async proposal

The example given for the alternate proposal show a CometProcessor that extends Thread that is started by a suspended request. This is a poor example!

If every suspending (auto-commit=false) servlet spawns a thread to handle the request, then there is no benefit from the async, as there is still a thread per request, it is just a different thread.

So assuming that the CometProcessor thread, is shared between multiple suspended requests then the example does not show the complexity of how that thread would handle an event. Consider if the comet app was a chat room with a 100 users in it,
then the CometProcessor would have references to 100 suspended requests. When an event happens, it must arrange for a response to be sent to all 100 suspended requests. If the thread sends the responses itself, it must handle all the IO issues of sending each response and each response may block causing delays in
other responses. Preferably, the CometProcessor will need to invoke 100 new threads
(or 100 new tasks to a thread pool), so that each response may be written without the potential to hold up other responses.

But the servlet container already provides a thread pool. One of the key benefits of the resume proposal is that it allows the existing thread pool to be used to handle the sending of responses for each suspended request rather than the application creating a wasteful second thread pool.

To truely evaluate the alternate proposal, fully working examples are needed to discover all the hidden complexities. The suspend/resume proposal has fully working examples available that are running real world applications.

It's not a servlet

A suspending (auto-commit=false) servlet is not fully functional servlet.

• It can't be filtered with filters such a GzipFilter that will attempt to complete the response on exit. The is no API for a filter to enquire if the response it is handling is suspended or not.

• It can't be dispatch to like a normal servlet. Because the content is produced asynchronously by another thread, forwards and includes will not work. The suspend/resume proposal can work with forwards and includes because content is produced normally within a servlet.

• Content cannot be generated with existing frameworks like JSP, JSF

The case for Resume

With the alternative proposal (suspend without resume),
once suspended, a request cannot access any existing content
generation framework. So after waiting for whatever event
(comet, webservice, JSR-237) a response still needs to be
generated.

Undeniably the alternative proposal can invoke an asynchronous
web service call (servlet 1.0 could do it - invocation is the easy
part). But once the asynchronous web service call(s) completes
the response content still needs to be generated.

The examples given simply say:
<pre>
OutputStream os = req.getResponse().getOutputStream();
// Write web service's response to OutputStream
</pre>
This is glossing over a decade of content generation technology
and complexity. JSP, JSF, Struts, cocoon, sitemesh, grails,
jruby etc. etc. will not be able to be used to generate such content.

Unless existing frameworks and applications can be invoked,
all the content generation will have to be done by new
frameworks and applications. More over, many of the techniques
and approaches familiar to current content generation will not
be available as they are designed around filtering, forwarding
and including.

The new frameworks will have to operate in a environment
without many useful features provided by the servlet
specification.

• The container Thread pool. Additional thread pools will be needed (when the intent is to reduce thread usage!). Also threading is not trivial and the CometProcess example given is broken. Using a thread pool for comet handling will either use more threads or will queue comet processors behind others waiting for events.

• No JNDI resource or other JEE resource may be used by the threads generating the content as the thread will not be anointed with JNDI initial context.

• No authorized resources will be able to be used as the thread will not be anointed with the authentication.

• Error handling will need to be reimplemented as the existing container error page mechanism will not be able to be used. Nor will any filters used be able to catch exceptions.

There is no need to reinvent content generation
frameworks and applications. We should be able to
invoked existing content generation from a suspended
request.

Resume allows existing applications and frameworks to
be invoked using the existing container threadpool,
error handling mechanism, JNDI context and authentication
model. It is no different to the existing Error page
mechanism that redispatches a request.

I have demonstrated resumed requests working with JSF,
JSP, Struts, spring MVC, Spring acegi, GWT, DWR, GzipFilter,
IceFaces, IBM Tivoli Auth Filter and more. Some of these are
already in production.

Resumed requests mostly work without any modification to
either frameworks or application. When modifications are
needed they tend to be simple. The modification needed
for filters will have to be done anyway as the they are
needed for suspend in the alternative proposal.

Resume is optional. If developers find the suspend-close
model sufficient, then they can still use that style.

Resume supports much needed use cases for which I have shown
endorsements from amazon, ebay, linkedIn and yahoo/zimbra.
Some of these companies are already using proprietary resume
extensions to access servlet content generation. If
servlet 3.0 does not support these use-case, then alternates
to the standard will continue to be used and the JCP
standard will be devalued.

The proposal of suspend without resume is only half a
solution.

It is bizzarre to have suspend only, where a thread or
a call back can do anything to a ServletResponse other than
pass it to an application running in an well known framework
running in a servlet container!