Java Servlet Programming

Chapter 6. Sending Multimedia Content

Until now, every servlet we've written has returned a standard HTML page. The web consists of more than HTML, though, so in this chapter we'll look at some of the more interesting things a servlet can return. We begin with a look at why you'd want to return different MIME types and how to do it. The most common use of a different MIME type is for returning an image graphic generated by a servlet (or even by an applet embedded inside the servlet!). The chapter also explores when and how to send a compressed response and examines using multipart responses to implement server push.

6.1. Images

People are visually oriented--they like to see, not just read, their information. Consequently, it's nearly impossible to find a web site that doesn't use images in some way, and those you do find tend to look unprofessional. To cite the well-worn cliche (translated into programmer-speak), "An image is worth a thousand words."

Luckily, it's relatively simple for a servlet to send an image as its response. In fact, we've already seen a servlet that does just this: the ViewFile servlet from Chapter 4, "Retrieving Information". As you may recall, this servlet can return any file under the server's document root. When the file happens to be an image file, it detects that fact with the getMimeType() method and sets its response's content type with setContentType() before sending the raw bytes to the client.

This technique requires that we already have the needed image files saved on disk, which isn't always the case. Often, a servlet must generate or manipulate an image before sending it to the client. Imagine, for example, a web page that contains an image of an analog clock that displays the current time. Sure, someone could save 720 images (60 minutes times 12 hours) to disk and use a servlet to dispatch the appropriate one. But that someone isn't me, and it shouldn't be you. Instead, the wise servlet programmer writes a servlet that dynamically generates the image of the clock face and its hands--or as a variant, a servlet that loads an image of the clock face and adds just the hands. And, of course, the frugal programmer also has the servlet cache the image (for about a minute) to save server cycles.

There are many other reasons you might want a servlet to return an image. By generating images, a servlet can display things such as an up-to-the-minute stock chart, the current score for a baseball game (complete with icons representing the runners on base), or a graphical representation of the Cokes left in the Coke machine. By manipulating preexisting images, a servlet can do even more. It can draw on top of them, change their color, size, or appearance, or combine several images into one.

6.1.1. Image Generation

Suppose you have an image as raw pixel data that you want to send to someone. How do you do it? Let's assume it's a true-color, 24-bit image (3 bytes per pixel) and that it's 100 pixels tall and 100 pixels wide. You could take the obvious approach and send it one pixel at a time, in a stream of 30,000 bytes. But is that enough? How does the receiver know what to do with the 30,000 bytes he received? The answer is that he doesn't. You also need to say that you are sending raw, true-color pixel values, that you're beginning in the upper left corner, that you're sending row by row, and that each row is 100 pixels wide. Yikes! And what if you decide to send fewer bytes by using compression? You have to say what kind of compression you are using, so the receiver can decompress the image. Suddenly this has become a complicated problem.

Fortunately this is a problem that has been solved, and solved several different ways. Each image format (GIF, JPEG, TIFF, etc.) represents one solution. Each image format defines a standard way to encode an image so that it can later be decoded for viewing or manipulation. Each encoding technique has certain advantages and limitations. For example, the compression used for GIF encoding excels at handling computer-generated images, but the GIF format is limited to just 256 colors. The compression used for JPEG encoding, on the other hand, works best on photo-realistic images that contain millions of colors, but it works so well because it uses "lossy" compression that can blur the photo's details.

Understanding image encoding helps you understand how servlets handle images. A servlet like ViewFile can return a preexisting image by sending its encoded representation unmodified to the client--the browser decodes the image for viewing. But a servlet that generates or modifies an image must construct an internal representation of that image, manipulate it, and then encode it, before sending it to the client.

6.1.1.1. A "Hello World" image

Example 6-1 gives a simple example of a servlet that generates and returns a GIF image. The graphic says "Hello World!", as shown in Figure 6-1.

Figure 6-1. Hello World graphics

Although this servlet uses the java.awt package, it never actually displays a window on the server's display. Nor does it display a window on the client's display. It performs all its work in an off-screen graphics context and lets the browser display the image. The strategy is as follows: create an off-screen image, get its graphics context, draw to the graphics context, and then encode the resulting image for transmission to the client.

Obtaining an off-screen image involves jumping through several hoops. In Java, an image is represented by the java.awt.Image class. Unfortunately, an Image object cannot be instantiated directly through a constructor. It must be obtained through a factory method like the createImage() method of Component or the getImage() method of Toolkit. Because we're creating a new image, we use createImage(). Note that before a component can create an image, its native peer must already exist. Thus, to create our Image we must create a Frame, create the frame's peer with a call to addNotify() , and then use the frame to create our Image.[1] Once we have an image, we draw onto it using its graphics context, which can be retrieved with a call to the getGraphics() method of Image. In this example, we just draw a simple string.

[1] For web servers running on Unix systems, the frame's native peer has to be created inside an X server. Thus, for optimal performance, make sure the DISPLAY environment variable (which specifies the X server to use) is unset or set to a local X server. Also make sure the web server has been granted access to the X server, which may require the use of xhost or xauth.

After drawing into the graphics context, we call setContentType() to set the MIME type to "image/gif" since we're going to use the GIF encoding. For the examples in this chapter, we use a GIF encoder written by Jef Poskanzer. It's well written and freely available with source from http://www.acme.com.[2] To encode the image, we create a GifEncoder object, passing it the image object and the ServletOutputStream for the servlet. When we call encode() on the GifEncoder object, the image is encoded and sent to the client.

[2] Note that the LZW compression algorithm used for GIF encoding is protected by Unisys and IBM patents which, according to the Free Software Foundation, make it impossible to have free software that generates the GIF format. For more information, see http://www.fsf.org/philosophy/gif.html. Of course, a servlet can encode its Image into any image format. For web content, JPEG exists as the most likely alternative to GIF. There are JPEG encoders in JDK 1.2 and commercial products such as the JIMI product (Java Image Management Interface), available from Activated Intelligence at http://www.activated.com.

After sending the image, the servlet does what all well-behaved servlets should do: it releases its graphical resources. These would be reclaimed automatically during garbage collection, but releasing them immediately helps on systems with limited resources. The code to release the resources is placed in a finally block to guarantee its execution, even when the servlet throws an exception.

6.1.1.2. A dynamically generated chart

Now let's look at a servlet that generates a more interesting image. Example 6-2 creates a bar chart that compares apples to oranges, with regard to their annual consumption. Figure 6-2 shows the results. There's little need for this chart to be dynamically generated, but it lets us get the point across without too much code. Picture in your mind's eye, if you will, that the servlet is charting up-to-the-minute stock values or the server's recent load.

Figure 6-2. A chart comparing apples and oranges

The basics are the same: create an off-screen image and get its graphics context, draw to the graphics context, and then encode the image for transmission to the client. The difference is that this servlet constructs a BarChart object to do the drawing. There are more than a dozen charting packages available in Java. You can find several showcased at http://www.gamelan.com. The BarChart class from this example came from Visual Engineering's JavaChart package, available at http://www.ve.com/javachart. It's a commercial product, but for readers of this book they have granted free permission to use the portion of the API presented above. The JavaChart package also includes a set of free chart-generating applets that we will use later in this chapter.

6.1.2. Image Composition

So far, we've drawn our graphics onto empty images. In this section, we discuss how to take preexisting images and either draw on top of them or combine them to make conglomerate images. We also examine error handling in servlets that return images.

6.1.2.1. Drawing over an image

Sometimes it's useful for a servlet to draw on top of an existing image. A good example is a building locator servlet that knows where every employee sits. When queried for a specific employee, it can draw a big red dot over that employee's office.

One deceptively obvious technique for drawing over a preexisting image is to retrieve the Image with Toolkit.getDefaultToolkit().getImage(imagename), get its graphics context with a call to the getGraphics() method of Image, and then use the returned graphics context to draw on top of the image. Unfortunately, it isn't quite that easy. The reason is that you cannot use getGraphics() unless the image was created with the createImage() method of Component. With the AWT, you always need to have a native peer in the background doing the actual graphics rendering.

Here's what you have to do instead: retrieve the preexisting image via the Toolkit.getDefaultToolkit().getImage(imagename) method and then tell it to draw itself into another graphics context created with the createImage() method of Component, as shown in the previous two examples. Now you can use that graphics context to draw on top of the original image.

Example 6-3 clarifies this technique with an example. It's a servlet that writes "CONFIDENTIAL" over every image it returns. The image name is passed to the servlet as extra path information. Some example output is shown in Figure 6-3.

Figure 6-3. Drawing over an image to mark it confidential

You can see that this servlet performs each step exactly as described above, along with some additional housekeeping. The servlet creates its unshown Frame in its init() method. Creating the Frame once and reusing it is an optimization previously left out for the sake of clarity. For each request, the servlet begins by retrieving the name of the preexisting image from the extra path information. Then it retrieves a reference to the image with the getImage() method of Toolkit and physically loads it into memory with the help of a MediaTracker. Normally it's fine for an image to load asynchronously with its partial results painted as it loads, but in this case we paint the image just once and need to guarantee it's fully loaded beforehand. Then the servlet gets the width and height of the loaded image and creates an off-screen image to match. Finally, the big moment: the loaded image is drawn on top of the newly constructed, empty image. After that it's old hat. The servlet writes its big "CONFIDENTIAL" and encodes the image for transmission.

Notice how this servlet handles error conditions by throwing exceptions and logging any errors that may interest the server administrator. When returning images, it's difficult to do much more. After all, a textual description doesn't help when a servlet is referenced in an <IMG> tag. This approach allows the server to do whatever it deems appropriate.

6.1.2.2. Combining images

A servlet can also combine images into one conglomerate image. Using this ability, a building locator servlet could display an employee's smiling face over her office, instead of a red dot. The technique used for combining images is similar to the one we used to draw over the top of an image: the appropriate images are loaded, they're drawn onto a properly created Image object, and that image is encoded for transmission.

Example 6-4 shows how to do this for a servlet that displays a hit count as a sequence of individual number images combined into one large image. Its output can be seen in Figure 6-4. The number images it uses are available at http://www.geocities.com/SiliconValley/6742/, along with several other styles.

Figure 6-4. Combining images to form a graphical counter

This servlet receives the number to display by reading its raw query string. For each number in the count, it retrieves and loads the corresponding number image from the directory given by DIR. (DIR is always under the server's document root. It's given as a virtual path and translated dynamically to a real path.) Then it calculates the combined width and the maximum height of all these images and constructs an off-screen image to match. The servlet draws each number image into this off-screen image in turn from left to right. Finally, it encodes the image for transmission.

To be of practical use, this servlet must be called by another servlet that knows the hit count to be displayed. For example, it could be called by a server-side include servlet embedded in a page, using syntax like the following:

<IMG src="/servlet/graphicalcounter?121672">

This servlet handles error conditions in the same way as the previous servlet, by throwing a ServletException and leaving it to the server to behave appropriately.

6.1.3. Image Effects

We've seen how servlets can create and combine images. In this section, we look at how servlets can also perform special effects on images. For example, a servlet can reduce the transmission time for an image by scaling down its size before transmission. Or it can add some special shading to an image to make it resemble a pressable button. As an example, let's look at how a servlet can convert a color image to grayscale.

6.1.3.1. Converting an image to grayscale

Example 6-5 shows a servlet that converts an image to grayscale before returning it. The servlet performs this effect without ever actually creating an off-screen graphics context. Instead, it creates the image using a special ImageFilter. (We'd show you before and after images, but they wouldn't look very convincing in a black-and-white book.)

Much of the code for this servlet matches that of the Confidentializer example. The major difference is shown here:

// Create an image to match, run through a filter
Image filtered = frame.createImage(
  new FilteredImageSource(image.getSource(),
                          new GrayscaleImageFilter()));

This servlet doesn't use the createImage(int,int) method of Component we've used up until now. It takes advantage of the createImage(ImageProducer) method of Component instead. The servlet creates an image producer with a FilteredImageSource that passes the image through an GrayscaleImageFilter. This filter converts each color pixel to its grayscale counterpart. Thus, the image is converted to grayscale as it is being created. The code for the GrayscaleImageFilter is shown in Example 6-6.

For each value in the colormap, this filter receives a pixel value and returns a new filtered pixel value. By setting the canFilterIndexColorModel variable to true, we signify that this filter can operate on the colormap and not on individual pixel values. The pixel value is given as a 32-bit int, where the first octet represents the alpha (transparency) value, the second octet the intensity of red, the third octet the intensity of green, and the fourth octet the intensity of blue. To convert a pixel value to grayscale, the red, green, and blue intensities must be set to identical values. We could average the red, green, and blue values and use that average value for each color intensity. That would convert the image to grayscale. Taking into account how people actually perceive color (and other factors), however, demands a weighted average. The 0.299, 0.587, 0.114 weighting used here matches that used by the National Television Systems Committee for black-and-white television. For more information, see Charles A. Poynton's book A Technical Introduction to Digital Video (Wiley)and the web site http://www.color.org.

6.1.3.2. Caching a converted image

The process of creating and encoding an image can be expensive, taking both time and server CPU cycles. Caching encoded images can often improve performance dramatically. Instead of doing all the work for every request, the results can be saved and resent for subsequent requests. The clock face idea that we mentioned earlier is a perfect example. The clock image needs to be created at most once per minute. Any other requests during that minute can be sent the same image. A chart for vote tabulation is another example. It can be created once and changed only as new votes come in.

For our example, let's give the DeColorize servlet the ability to cache the grayscale images it returns. The servlet life cycle makes this extremely simple. Our new DeColorize servlet saves each converted image as a byte array stored in a Hashtable keyed by the image name. First, our servlet needs to create a Hashtable instance variable. This must be declared outside doGet():

Hashtable gifs = new Hashtable();

To fill this hashtable, we need to capture the encoded graphics. So, instead of giving the GifEncoder the ServletOutputStream, we give it a ByteArrayOutputStream. Then, when we encode the image with encode(), the encoded image is stored in the ByteArrayOutputStream. Finally, we store the captured bytes in the hashtable and then write them to the ServletOutputStream to send the image to the client. Here's the new code to encode, store, and return the filtered image:

// Encode, store, and return the filtered image
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GifEncoder encoder = new GifEncoder(filtered, baos);
encoder.encode();
gifs.put(source, baos);
baos.writeTo(out);

This fills the hashtable with encoded images keyed by image name. Now, earlier in the servlet, we can go directly to the cache when asked to return a previously encoded image. This code should go immediately after the code executed if source==null:

// Short circuit if it's been done before
if (gifs.containsKey(source)) {
  ByteArrayOutputStream baos = (ByteArrayOutputStream) gifs.get(source);
  baos.writeTo(out);
  return;
}

With these modifications, any image found in the cache is returned quickly, directly from memory.

Of course, caching multiple images tends to consume large amounts of memory. To cache a single image is rarely a problem, but a servlet such as this should use some method for cleaning house. For example, it could cache only the 10 most recently requested images.

6.1.4. Image Effects in Filter Chains

We haven't talked about filter chains yet in this chapter, but they are actually quite useful for performing image effects. If you recall, a servlet in a filter chain receives content on its input stream and sends a filtered version of that content out its output stream. In previous examples, we have always filtered textual HTML. Now we can see how to filter images in a servlet chain.

Performing special effects on an image works the same whether it happens in a filter chain or in a standard servlet. The only difference is that instead of loading the image from a file, a chained servlet receives its image as an encoded stream of bytes. Example 6-7 shows how a servlet receives an encoded stream of bytes and creates an Image from them. In this case, the servlet shrinks the image to one-quarter its original size.

The createImage(byte[]) method of Toolkit creates an Image from an array of bytes. The method determines the image format automatically, as long as the image is in one of the formats understood and decodable by the AWT (typically GIF, JPEG, and XBM, although it's possible to add a custom content handler).

The servlet uses the createImage() method to create an Image out of the incoming bytes. Because the createImage() method doesn't accept an input stream, the servlet first captures the bytes with a ByteArrayOutputStream. After creating the Image, the servlet loads it in order to get its true width and height. Then the servlet gets a scaled instance that is half as wide and half as tall, using the getScaledInstance() method of Image. Last, it encodes the image and sends it out its output stream.

Why use a filter chain to perform an image effect instead of a standard servlet? The main reason is for increased flexibility. For example, a server can be told that all the large classified images in one subdirectory should be run through a "shrink" filter and a "confidential tag" filter. Closer to reality, the server can be told that any image on the web site should be served in its "shrunken" form if the request URI begins with "/lite". Another possibility is to tell the server that all images of type image/xbm need to be run through a basic filter that converts the XBM image into a GIF.

Are you wondering why we aren't taking advantage of object serialization to pass our image from servlet to servlet? The reason is simple: images are not Serializ-able. If a servlet can guarantee that the next link in the chain is another servlet and not the client, though, then it can pass the Image more efficiently using techniques described in Chapter 11, "Interservlet Communication".

6.1.5. An Image of an Embedded Applet

Now let's take a look at one of the more creative ways a servlet can generate an image: by taking a picture of an embedded applet. Applets are small Java programs that can be sent to a client for execution inside a web page--they've been used to create everything from animations to interactive programs to static charts. Here we're going to twist their use a bit. Instead of having the server send a program to the client for execution, we have it send just a picture of the program executing on the server. Now we'll admit that replacing an executing applet with an image is hardly a fair trade, but it does has its advantages. For a static, noninteractive applet, it's often more efficient to send its image than to send the code and data needed to have the client create the image itself. Plus, the image displays even for clients whose browsers don't support Java or who may have Java support disabled.

6.1.5.1. An image of a simple applet

Example 6-8 shows an applet that may look familiar to you. It's the SecondApplet example taken from David Flanagan's Java Examples in a Nutshell book (O'Reilly). Figure 6-5 shows its "fancy graphics."

Figure 6-5. The simple applet's fancy graphics

This applet can be embedded the traditional way inside an HTML file with the <APPLET> tag:

<APPLET CODE="SecondApplet.class" WIDTH=500 HEIGHT=200>
</APPLET>

An <APPLET> tag can include a CODEBASE parameter that tells the client where to fetch the given class. Because the previous <APPLET> tag does not provide a CODEBASE parameter, the SecondApplet.class file is assumed to be in the same directory as the HTML file.

This applet can also be embedded inside HTML content returned by a servlet:

import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;

public class SecondAppletHtml extends HttpServlet { 

  public void doGet(HttpServletRequest req, HttpServletResponse res)
                               throws ServletException, IOException {
    res.setContentType("text/html");
    PrintWriter out = res.getWriter();

    // ...
    out.println("<APPLET CODE=SecondApplet.class CODEBASE=/ " +
                "WIDTH=500 HEIGHT=200>");
    out.println("</APPLET>");
    // ...
  }
}

Notice that here the CODEBASE parameter must be supplied. If it's not given, the code base is erroneously assumed to be /servlet or whatever other virtual path was used to launch the servlet.

Now let's look at a servlet that embeds SecondApplet inside itself and sends a picture of the applet to the client. The code is shown in Example 6-9 and its output in Figure 6-6. In order to embed an applet, a servlet needs a special Frame subclass that implements AppletContext and AppletStub. For these examples, we can use a modified version of Jef Poskanzer's Acme.MainFrame class. In addition to some minor bug fixes, the class has been modified to not call its own show() method (to keep it from actually displaying during execution) and to call the applet's init() and start() methods synchronously instead of in a separate thread (to guarantee the applet is ready when we call its paint() method). A copy of Acme.MainFrameModified is available with the book examples as described in the the Preface.

Figure 6-6. Another view of the simple applet's fancy graphics

This servlet begins by dynamically loading the SecondApplet class and creating a single instance of it. For SecondApplet to be found, it must be somewhere in the server's standard CLASSPATH--which for the Java Web Server by default excludes the server_root/servlets directory. Then the servlet prepares the applet's arguments. These are passed to the MainFrameModified constructor as an array of "name=value" strings. SecondApplet takes no parameters, so this step would seem to be unnecessary. However, MainFrameModified piggy-backs into the argument list its own "barebones" argument, which we set to true to indicate it should display the applet without any special decoration. Finally, the servlet creates an appropriately sized off-screen graphics context, has the applet paint itself using that context, and encodes the image for transmission to the client.

6.1.5.2. A generic applet viewer

We can build on this example to develop a generic servlet capable of embedding and taking a picture of any applet. It can accept as request parameters the applet name, its width and height, and its parameters. Example 6-10 contains the code.

There are two major differences between this servlet and SecondAppletViewer: how it handles parameters and how it paints the applet's components. All the details, from the applet's name to its parameters, are passed to this servlet via request parameters. It receives the name of the applet as the "applet" parameter and its width and height as the "width" and "height" parameters; it passes all the other parameters on to the applet itself.

The painting is more radically different. This servlet uses a custom-built paintContainerChildren() utility method to paint all the components of the applet. For the servlet to call applet.paintComponents(g) is not sufficient because paintComponents(g) does not paint to the passed-in Graphics object! Instead, it uses the Graphics parameter only to get a clipping region. This servlet also uses paintAll() instead of paint(), so that it correctly paints lightweight components. Note that for this technique to work well, the embedded applet has to fully paint itself during its first paint() invocation. It can't display a splash screen or perform a lazy load of its images.

The AppletViewer servlet can replace SecondAppletViewer. Just invoke it with the URL http://server:port/servlet/AppletViewer?applet=SecondApplet. It can also replace our SimpleChart example. Remember when we said JavaChart includes a set of free chart-generating applets? We can use AppletViewer to embed any of these free applets and send the resulting chart as an image to the client. To duplicate the SimpleChart example requires this lengthy URL (split into separate lines for readability, probably so long that many servers won't be able to handle it):

http://server:port/servlet/AppletViewer?
applet=javachart.applet.columnApp&
titleFont=TimesRoman%2c24%2c0&
titleString=Comparing+Apples+And+Oranges&
xAxisTitle=Year&
yAxisTitle=Tons+Consumed&
xAxisLabels=1993%2c1994%2c1995%2c1996%2c1997&
dataset0yValues=950%2c1005%2c1210%2c1165%2c1255&
dataset1yValues=1435%2c1650%2c1555%2c1440%2c1595&
dataset0Color=red&
dataset0Name=Apples&
dataset1Color=orange&
dataset1Name=Oranges&
legendOn=yes&
legendHorizontal=true&
legendllX=0.4&
legendllY=0.75&
iconHeight=0.04&
iconWidth=0.04&
iconGap=0.02&
xAxisOptions=gridOff&
yAxisOptions=gridOff

The graph generated by this URL looks identical to Figure 6-2 shown earlier (with the one difference that the applet version contains a blue dot in the lower right corner that can be removed with the purchase of a JavaChart license).

6.1.5.3. Advantages and disadvantages

We think you'll agree that embedding an applet in a servlet has a certain coolness factor. But is it ever practical? Let's look over its advantages and disadvantages. First, the advantages:

It can save money.

Hey, the JavaChart applets are free, and Visual Engineering assured us that this use doesn't violate their license!

It can save download time.

Why send all the code and data needed to make an image when you can send the image itself, especially when the image can be pregenerated?

It works for every client.

It works even when the client browser doesn't support Java or has Java disabled.

However, on the downside:

It requires extra resources on the server.

Specifically it consumes CPU power and memory.

It works well for only a few applets.

Specifically it works best on static, noninteractive applets that fully paint themselves with their first paint() invocation.

5.7. When Things Go Wrong		6.2. Compressed Content

Copyright © 2001 O'Reilly & Associates. All rights reserved.