Chapter 6. Processing Large Data

Chapter 6. Processing Large Data
Prev		Next

Table of Contents

6.1. Receiving large SOAP requests

6.1.1. Provider<Message>

6.2. Binary Attachments (MTOM)

6.2.1. MTOM
6.2.2. Enabling MTOM on server
6.2.3. Enabling MTOM on client
6.2.4. MTOM threshold
6.2.5. .NET interoperability

6.3. Large Attachments

6.3.1. Client Side
6.3.2. Server Side
6.3.3. Configuration
6.3.4. Large Attachments Summary

6.1. Receiving large SOAP requests

Processing of large incoming SOAP requests can be made more efficient with some additional effort.

6.1.1. Provider<Message>

JAX-WS RI extension Provider<Message> can be used to read an incoming SOAP message by using XMLStreamReader (among other things.) This allows you to read the SOAP message on-demand lazily, without needing to buffer the whole message in memory.

See the relevant JAXB users guide section for how to combine JAXB with such streaming processing.

6.2. Binary Attachments (MTOM)

6.2.1. MTOM

MTOM (and XOP) allows you to send and receive binary attachments (such as files and images) efficiently and in an interoperable manner.

6.2.1.1. What is MTOM?

Perhaps the best way to understand the pros and cons of MTOM is to see an actual on-the-wire message. See an example below:

Example 6.1. Sample MTOM message

Content-Type: Multipart/Related; start-info="text/xml"; type="application/xop+xml"; boundary="----=_Part_0_1744155.1118953559416"
Content-Length: 3453
SOAPAction: ""

------=_Part_1_4558657.1118953559446
Content-Type: application/xop+xml; type="text/xml"; charset=utf-8

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
  <S:Body>
    <Detail xmlns="http://example.org/mtom/data">
      <image>
        <xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:5aeaa450-17f0-4484-b845-a8480c363444@example.org" />
      </image>
    </Detail>
  </S:Body>
</S:Envelope>


------=_Part_1_4558657.1118953559446
Content-Type: image/jpeg
Content-ID: <5aeaa450-17f0-4484-b845-a8480c363444@example.org>


... binary data ...

The noteworthy points are:

The binary attachment is packaged in a MIME multi-part message (the same mechanism used in e-mail to handle attachments.)
An <xop:Include> element is used to mark where the binary data is.
The actual binary data is kept in a different MIME part.

MTOM is efficient, in the sense that it doesn't have the 33% size increase penalty that xs:base64Binary has. It is interoperable, in the sense that it is a W3C standard. However, MIME multipart incurs a small cost proportional to the number of attachments, so it is not suitable for a large number of tiny attachments.

The schema that describes the above message is below. The MTOM spec is designed to work below the XML infoset level, so the schema describes the image as being of type xs:base64Binary, even though it can be attached as seen above. When using MTOM, any base64Binary can be attached or inlined.

Example 6.2. Schema

<element name="Detail" type="types:DetailType"/>
<complexType name="DetailType">
  <sequence>
    <element name="image" type="base64Binary" />
  </sequence>
</complexType>

6.2.1.2. xmime:expectedContentType to Java type mapping

Schema elements of type xs:base64Binary or xs:hexBinary can be annotated by using the xmime:expectedContentType attribute to indicate the type of binary that is expected. wsimport recognizes this annotation and will map the binary data to its proper Java representation instead. The table below is taken from JAXB spec Table 9-1, which shows the mapping rules:

Table 6.1. JAXB Mapping Rules

MIME Type	Java Type
image/*	java.awt.Image
text/plain	java.lang.String
application/xml, text/xml	javax.xml.transform.Source
(others)	javax.activation.DataHandler

6.2.1.3. xmime:contentType attribute

The schema can further use the xmime:contentType attribute to designate the actual content type of the binary data used in the message. (In contrast, xmime:expectedContentTypes specifies what are allowed. This combination allows you to say "image/* is expected but this message contains image/jpeg".)

This attribute can be used with elements whose content is either xs:base64Binary or xs:hexBinary. Consider the following example:

Example 6.3. Using xmime:contentType

<element name="TestMtomXmimeContentType" type="types:PictureType"/>

<complexType name="PictureType">
  <simpleContent>
    <restriction base="xmime:base64Binary">
      <attribute ref="xmime:contentType" use="required" />
    </restriction>
  </simpleContent>
</complexType>

Here xmime:base64Binary is defined by Describing Media Content of Binary Data in XML. The following code shows how your program can set the MIME type to the generated beans:

Example 6.4. Setting content type

PictureType req = new PictureType();
req.setValue(name.getBytes());
req.setContentType("application/xml");

On the wire this is how it looks:

Example 6.5. SOAP Message that uses xmime:contentType

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:ns1="http://example.org/mtom/data"
  xmlns:ns2="http://www.w3.org/2005/05/xmlmime">
  <S:Body>
    <ns1:TestMtomXmimeContentTypeResponse ns2:contentType="application/xml">
      <xop:Include
          xmlns:xop="http://www.w3.org/2004/08/xop/include"
          href="cid:193ed174-d313-4325-8eed-16cc25595e4e@example.org"/>
    </ns1:TestMtomXmimeContentTypeResponse>
  </S:Body>
</S:Envelope>

6.2.2. Enabling MTOM on server

There are several ways to enable MTOM.

By using the @MTOM annotation on SEI. This is convenience and preferable for developers.
Example 6.6. Annotating SEI with @MTOM
```
@javax.xml.ws.soap.MTOM
@WebService
public class HelloImpl implements Hello {
    ....
}
```
By using the enable-mtom attribute in the sun-jaxws.xml configuration file. This allows the MTOM setting to be changed at deployment time.
Example 6.7. Enabling MTOM in sun-jaxws.xml
```
<endpoints xmlns='http://java.sun.com/xml/ns/jax-ws/ri/runtime' version='2.0'>
  <endpoint name="Mtom" implementation="mtom.server.HelloImpl"
    url-pattern="/hello"
    enable-mtom="true"/>
</endpoints>
```
If you are using JAX-WS RI via Spring, you can also enable MTOM from the bean definition file. See the JAX-WS spring extension for more.

6.2.3. Enabling MTOM on client

There are several ways to enable MTOM.

By doing nothing. If the server WSDL advertises that it supports MTOM, the MTOM support in the client will be automatically enabled. This is the preferable way.
By passing MTOMFeature as WebServiceFeature parameter while creating a proxy or a Dispatch. See the example below:
Example 6.8. Passing MTOMFeature
```
Hello port = new HelloService().getHelloPort(new MTOMFeature());
Dispatch d = new HelloService().createDispatch(....,new MTOMFeature());
```

6.2.4. MTOM threshold

As discussed above, in rare situations where you have a lot of tiny attachments, the overhead of MTOM may outweigh the benefit of binary transfer. To cope with this situation, the JAX-WS RI supports the notion of "threshold" --- if an attachment is smaller than the size specified in threshold, it will simply inline the binary data as base64 binary instead of making it an attachment. Because of the way MTOM spec is designed, such inline vs attachment decision is handled by the toolkits of both ends and will not harm the application running on both sides.

There are two ways to control the threshold:

By using the com.sun.xml.ws.developer.JAXWSProperties.MTOM_THRESHOLD_VALUE property in the RequestContext on the client side and in the MessageContext on the server side.
By adding parameter to the @MTOM annotation, such as @MTOM(threshold=3000)
By adding parameter to the MTOMFeature object, such as new MTOM(3000)

6.2.5. .NET interoperability

6.2.5.1. Using Metro distribution

MTOM support is fully interoperable with .NET clients and servers. If you are working with Metro then your MTOM solution as a endpoint or as client will work out of the box.

6.2.5.2. Using JAX-WS RI distribution

If you are using JAX-WS RI distribution, MTOM interop with .NET client or server will not work out of the box. The reason behind this is that JAX-WS RI does not have built in support for WS-Policy and .NET 3.0/.NET 3.5 looks for an MTOM policy assertion in the WSDL before turning on MTOM encoding. So, you will need to turn it on explicitly on your .NET 3.0/3.5 or JAX-WS RI client.

The MTOM policy assertion that .NET 3.0/.NET 3.5 understands is: <wsoma:OptimizedMimeSerialization/>

6.2.5.2.1. JAX-WS RI endpoint and .NET client

Turn on MTOM explicitly on your .NET client using the WCF editor available with Visual Studio 2005.

6.2.5.2.2. JAX-WS RI client and .NET endpoint

Turn on MTOM on JAX-WS RI client as defined above.

Here is a sample Metro MTOM endpoint and a .NET 3.0 client.

6.3. Large Attachments

JAX-WS RI provides support for sending and receiving large attachments in a streaming fashion. Often times, large attachments need to be stored on the file system since they cannot be kept in memory(limited by heap size). But in certain cases, streaming of attachments is possible without ever storing the content on the file system. RI will try to stream the attachments whenever it is possible. Otherwise, it would store the large attachments on the file system.

The programming model is based on MTOM and DataHandler. You want to send large data as a SOAP attachment, see this section for more details. Also you want to bind large data to DataHandler instead of byte[]. RI provides a StreamingDataHandler, a DataHandler implementation that can be used to access the data efficiently in a streaming fashion.

6.3.1. Client Side

RI uses Java SE's HttpURLConnection for web service invocations. HttpURLConnection buffers the request data by default. So the client applications need to enable streaming explicitly, see http client streaming. The following sample show how to send and receive large data.

Example 6.9. Sample client for large attachments

import com.sun.xml.ws.developer.JAXWSProperties;
import com.sun.xml.ws.developer.StreamingDataHandler;

MTOMFeature feature = new MTOMFeature();
MyService service = new MyService();
MyProxy proxy = service.getProxyPort(feature);
Map<String, Object> ctxt = ((BindingProvider)proxy).getRequestContext();
// Enable HTTP chunking mode, otherwise HttpURLConnection buffers
ctxt.put(JAXWSProperties.HTTP_CLIENT_STREAMING_CHUNK_SIZE, 8192); 
DataHandler dh = proxy.fileUpload(...);
StreamingDataHandler sdh = (StreamingDataHandler)dh;
InputStream in = sdh.readOnce();
...
in.close();
sdh.close();

6.3.2. Server Side

Use @MTOM feature for a service and DataHandler parameter for large data. If the WSDL contains xmime:expectedContentTypes="application/octet-stream", it would be mapped to DataHandler in the generated SEI. If the service is starting from java, @XmlMimeType("application/octet-stream") can be used to generate an appropriate schema type in the generated WSDL.

The following sample shows how to upload files. It uses StreamingDataHandler.moveTo(File) convenient method to store the contents of the attachment to a file.

Example 6.10. Sample service for large attachments

import com.sun.xml.ws.developer.StreamingDataHandler;
...

@MTOM
@WebService
public class UploadImpl {

    // Use @XmlMimeType to map to DataHandler on the client side
    public void fileUpload(String name, 
                           @XmlMimeType("application/octet-stream") 
                           DataHandler data) {
        try {
            StreamingDataHandler dh = (StreamingDataHandler) data;
            File file = File.createTempFile(name, "");
            dh.moveTo(file);
            dh.close();
        } catch (Exception e) {
            throw new WebServiceException(e);
        }
    }
}

6.3.3. Configuration

You can configure streaming attachments behaviour using @StreamingAttachment on the server side, and using StreamingAttachmentFeature on the client side. Using this feature, you can configure only certain sized attachments are written to a file.

Example 6.11. Sample Service Configuration

import com.sun.xml.ws.developer.StreamingAttachment;

// Configure such that whole MIME message is parsed eagerly,
// Attachments under 4MB are kept in memory
@MTOM
@StreamingAttachment(parseEagerly=true, memoryThreshold=4000000L)
@WebService
public class UploadImpl {
}

Example 6.12. Sample client configuration

import com.sun.xml.ws.developer.StreamingAttachmentFeature;

MTOMFeature mtom = new MTOMFeature();
// Configure such that whole MIME message is parsed eagerly,
// Attachments under 4MB are kept in memory
StreamingAttachmentFeature stf =
        new StreamingAttachmentFeature(null, true, 4000000L);
MyService service = new MyService();
MyProxy proxy = service.getProxyPort(feature, stf);

6.3.4. Large Attachments Summary

Use MTOM and DataHandler in the programming model.
Cast the DataHandler to StreamingDataHandler and use its methods.
Make sure you call StreamingDataHandler.close() and also close the StreamingDataHandler.readOnce() stream.
Enable HTTP chunking on the client-side.

Prev		Next
Chapter 5. HTTP	Home	Chapter 7. Bootstrapping and Configuration