Async Use-Case Feedback

The following feedback was received in response to an enquiry about
use cases for asynchronous servlets:

Sanjin Lee Ebay

eBay has several critical use cases for the asynchronous servlet.

First, the AJAX push is one of the most important examples, and the list
of eBay applications that employ push is growing rapidly. In any
situation where a client needs to maintain long-lived requests to get
status updates (long polls simulating server-side push), the
conventional blocking container simply doesn't scale. The behavior
where one can suspend requests and resume later on is a must in handling
such applications.

Second, one of the most critical use cases is service
coordination/orchestration (e.g. SOA and/or RESTful services) where one
server forwards or routes requests to another server or functionality to
complete request handling. Such a service orchestration is always I/O
heavy, and having an asynchronous servlet is extremely critical in
scalability. Other similar examples include server-side content
aggregation (e.g. portal).

In all these use cases, we need access to all servlet features including
servlet requests/responses, dispatchers, authentication, JSPs, etc. in
order to prepare the responses.
...

Today, all the vendors have their own solutions that
trigger vendor lock-in. We strongly believe that a standard
asynchronous servlet specification will benefit all that are involved.

JJ Zhuang Software Architect Yahoo Zimbra

Now regarding actual realization of async servlets, I can tell you that from my own experience the suspend/resume semantics is by far the simplest. That's because the application developer doesn't need to know how to write actual asynchronous code when adopting suspend/resume. Everything is still as linear as regular servlets. Again using Zimbra as an example, we originally developed our server code to run with tomcat. After switching to Jetty, it look only a few lines of change to make everything asynchronous. All we needed to do was to check a boolean in the request object to know whether this is an initial request or a resumed request. Every line of application code is still synchronous, but with the container's help all of a sudden there's no more blocked threads in the process.

In short, I fully support your view of the importance of async servlets and hope it can be adopted by JSR to become part of 3.0 standard.

Arnold Goldberg VP Software Engineering Linked In

One use case that is most obvious is simple.

Advertising systems have taken over the web and everyone is using them
to monetize. Companies are trying to integrate them either from the
client side which then leaves you up to the QoS of the ad provider
(usually quite bad) or a server side integration which is what most
larger enterprises are doing as there is more control on the user
experience.

With the standard servlet spec there is no way to scale this
architecture . We are forced to leave the container almost immediately
to deal with the highly latent requests that require almost no
processing on the appserver side and should scale to 1000+ connections.
There has to be a spec level detachment between threads of execution and
connections, or the spec will become increasing meaningless for these
common web2.0 integrations.

Tatu Saloranta Senior. software engineer at Amazon.com

Number one requirement is to be able to use non-blocking (non-one-thread-per-request) handling; and next wish is obviously to do that using well-defined standard interfaces. And as to service requests, well, Servlet API is The Standard on Java platform. There is very little else, so coming up with Yet Another New (proposed) standard does not sound very tempting.

Tim Kimmet Paypal

I see a lot of talk out there regarding the blocking of read/write on the connector side... Unfortunately, a lot of people are missing the point- typically companies deploy hardware to solve that problem (we use a LB, which employs TCP buffering to handle this and other companies typically deploy a web server farm to handle this and more)... What I tried to point out is "what happens during the biz call..." and how to resolve calls blocking from I/O during the biz processing of the request. This is where we discussed the requirement of the Continuation and the HTTP Client, which are really key components to async...