Graphics Display Under RDP

If you view a user interface or a dynamic Windows application on a remote client, it is reminiscent of a digital movie. The action might be a bit abstract, and the cutover as application windows change or start is very abrupt. Displaying a graphical user interface on a remote client mainly requires the transmission of huge amounts of data. If modified screen sections were always transferred as a complete set from server to client, the data would easily total from several hundred kilobytes to more than a megabyte. For this reason, starting or ending a full-screen application is particularly expensive in terms of the required network bandwidth.

Therefore, optimizing the RDP protocol and its handling of graphical elements has been given the utmost attention and effort. The most efficient compression algorithms and administration mechanisms have been used, supporting almost all graphics operations of both a Windows desktop and Windows-based applications. The intended result is to keep computational effort and required network bandwidth at the lowest possible level.

The RDP graphics driver receives the graphics commands from the applications via the graphics device interface (GDI or GDI+, discussed in Chapter 1). The application instructs the GDI(+) where and what to draw. The GDI(+) forwards the instructions to the RDP driver. Nevertheless, the GDI(+) does not recognize that the graphics elements are not output on a local screen. The RDP driver transmits that data to a remote client. This is exactly when optimization must occur. Most graphics operations focus on producing raster images (bitmaps), drawing graphics primitives, and displaying text characters.

A bitmap is a rectangle consisting of pixels of different colors, thus creating a specific pattern. For instance, an application symbol (icon) is such a bitmap. The display of a static bitmap can be accelerated using compression when transmitting. Compression is possible at low color depth through a simple temporary encryption scheme that transmits only one color value if several pixels of the same color appear in a row. The rates of compression can also be influenced by the applications used and by the design of the graphical user interface. For example, using many colors and horizontal color transitions is very critical. This is why the decoration of panels in a remote client session is much simpler than on the console. The total number of colors used, as defined under bitmap properties, is also very important in the selection of the compression algorithm and the resulting compression rate.

Color palettes can further optimize bitmap use. A table containing color values is created and transmitted to the client. If individual pixels in a bitmap need coloration, the position coordinates in the table are transmitted, not the color value. The amount of data is always smaller when the color depth is high but the number of simultaneously used colors is relatively low. Thus, an individual color value requires up to three bytes, whereas the color position in a table with a maximum of 256 entries needs only one byte.

Even more problematic are animated images, that is, animated bitmaps. They result in significantly higher transfer rates on the network, even if they are very small (for example, animated or blinking mouse cursor). In this instance, we need another mechanism to limit the data volume: caching.

How can the output of graphics be realized, for example, lines or rectangles that make up many elements of the user interface? The easiest solution is to draw a line by transmitting each and every pixel. A much more efficient method is a command that defines the start and endpoint, thickness, and color of a line. The rest can be calculated by the output device. Naturally, this feature also uses much more complex commands, such as redrawing a window after it has been overlapped. Here, caching also plays a key role in temporarily storing data of all panel elements and contents.

Individual letters in character strings are managed by a special kind of bitmap, the glyph. In principle, the output of characters using glyphs is much easier than the transmittal of letter bitmaps. A command requests only the output of a glyph at a set screen position. However, the number of different fonts and font sizes of the current font sets is quite problematic, as is the initial transport of the glyphs to the client.

Caching

We already mentioned another mechanism to reduce the amount of data transmitted. It is called caching, or temporary storage. In this process, the client reserves memory that houses frequently used image fragments that can be displayed again without any network transmission taking place. Some character operations do not work at all without caching, due to technical reasons.

Caching functions are not realized solely through the client’s main memory. If local hard drives exist, they can permit so-called persistent caching, in which the cached data is still available even after a client reboot. In principle, special network components can store RDP data and thus provide a global caching space on a LAN segment.

RDP supports the following caches:

• Bitmap cache Cache for different kinds of bitmaps. Size and number of these caches is determined upon connection.

• Font cache Cache for glyphs (character bitmaps). The cache size must be sufficient for storing all characters of a defined character set.

• Desktop cache Cache for a desktop screenshot. A character command stores or outputs this special bitmap.

• Cursor cache Cache for mouse cursors that need special handling. Special graphics algorithms ensure that the correct mouse cursor is displayed on the desktop without generating a lot of network traffic and overloading the local processor resources. Drag-and-drop operations with the mouse therefore do not require the transmission of new graphical elements, but only the transmission of the new coordinates.

• Text cache Cache for frequently used character strings and the corresponding formatting information. Glyphs from the font cache are used to generate a complete character string.

Virtual Channels and Mirroring

An RDP client or application can use a virtual channel to transmit specific information. Virtual channels thus help add functions that are not yet specified in the RDP protocol. They represent a platform that future developments can be based on without having to modify the communication methods between a terminal server and its clients. With Windows Server 2003 and RDP 5.2, virtual channels are used for joint client and server clipboards and for redirecting print jobs to local client printers.

Basically, the RDP protocol allows distribution of data streams from one source to many destinations without having to send the data separately. An application can therefore be mirrored on another client. Even the point of input can be transferred from one user to another.

Required Network Bandwidth

It is very important to know how much bandwidth the RDP protocol uses on the network. Unfortunately, there is no universal answer to this question because it is influenced by so many external conditions, such as:

• Color depth and screen resolution set and supported on the client

• Encryption and compression

• Desktop backgrounds allowed or prohibited, window contents displayed while dragging, animated menus and windows, designs, or bitmap caching

• Client and server platform capacity

• Type and combination of applications used

• User behavior

RDP is usable at a bandwidth of 20 kilobits per second (Kbps), and most efficient at 50 Kbps. If there is sufficient bandwidth, the RDP protocol can use up to 500 Kbps. For instance, it uses more bandwidth if a program produces large graphics quickly in series, and if it is limited only by the output channel’s capacity. This situation occurs when computer games, videos, animations, or certain benchmark programs are launched via an RDP client. Therefore, these programs are normally not suitable for running on terminal servers.

Only comprehensive tests, as described in Chapter 11, can give us more detailed information on the bandwidth required in a target environment. Just be prepared for some surprises! The RDP protocol does not always behave as expected. For instance, it is not necessarily true that increasing the supported color depth from 8 bits to 16 bits inevitably results in poorer performance. An 8-bit color space was supported in the past, and for compatibility reasons, the algorithms used then are still available. Modern algorithms are generally faster and optimized to a larger color space. So using more colors could actually improve performance.

Client and server platforms play a key role, too. Network bandwidth can be reduced through compression. However, if the client is too slow to process the received data stream and simultaneously encrypt it, the sessions will not run smoothly. Therefore, optimizing the transmission rate by prohibiting desktop backgrounds or animated menus is of little use.

Figure 3-5 shows a great example of a measurement with limited validity. Six RDP sessions, each configured differently, were executed in a sequence on one client. The bandwidth required by a 16-bit program for graphical load tests was measured for each session. During the measurement, both computer platforms had exclusive use of a 100-megabit network. Each session had its own user account in order to prevent mutual influence. The 16-bit test program was run within each user session in the 16 bit emulation environment. (See the discussion of WoW and VDM in Chapter 1.)

The results are very similar, and peak loads are all between 25 kilobytes per second (Kbps) and 35 Kbps. This is even more surprising when we review the individual RDP session parameters, as shown in the following table:

We see that the different RDP session configurations resulted mainly in different run times of the test program. The run times differed by up to 50 percent, depending on the parameters selected. However, the values for the network peak loads did not differ very much. It is rather noteworthy, though, that the highest peak load occurred during session 9 configured with only 256 colors (8 bit). In this case, the display speed on the client had the highest value, which correlated with the corresponding demands of the network.

In contrast, Figure 3-6 shows a normal RDP session whose parameters match the ones used in session ID 40 from Table 3.1. Again, both computer platforms had exclusive use of a 100-megabit network without bandwidth limitations. The user of the measured session first started the Paint program and opened a large graphic. The user then started WordPad, opened an existing document, and started adding to it. The application windows were continuously dragged on the desktop. Short-term network load peaks of more than 10 Kbps were observed only when a new application was opened and the window needed to be transmitted for the first time. The processor load was also moderate during these actions, which could be attributed to the fact that the user was the only one on the terminal server.

Table 3.1: Configuration of the Individual RDP Sessions for the Bandwidth Test

Session ID	Resolution	Color Depth	Bitmap Caching	Network Optimization
7	1024 x 768	16 bit	Yes	LAN (10 Mbps)
8	640 x 480	16 bit	Yes	LAN (10 Mbps)
9	1024 x 768	8 bit	Yes	LAN (10 Mbps)
10	640 x 480	8 bit	Yes	LAN (10 Mbps)
11	1024 x 768	16 bit	No	Modem (28.8 Kbps)
12	640 x 480	16 bit	No	Modem (28.8 Kbps)

The average bandwidth measured was 2.6 Kbps, which corresponds to approximately 20 Kbps. If the session were run over a 56 Kbps modem, the bandwidth would definitely suffice. The only difference observed was a short delay in the initial construction of the application windows. Optimizing the transmission rate in the client settings would further reduce the bandwidth required. This scenario would result in relatively smooth operation even on very slow network connections.

Latency

User satisfaction in a terminal-server environment is even more influenced by network delays (latency) than by nominally available bandwidth. For instance, a wide area network (WAN) is usually unsuitable for a terminal server, despite its extremely high bandwidth. Signal transmission from user input to client response can take up to one second, because the signal has a long way to travel: from mouse or keyboard to the server, eliciting a graphical response from the corresponding user session, which is then transmitted back to the client.

In addition, not every keystroke made on the RDP client is immediately sent to the server. An input buffer first collects user input to send it as a package. This can also lead to obvious delays in the response behavior of application programs. However, the problem can be resolved by modifying the parameters for buffering RDP data streams. This functionality is described in detail in Chapter 6 and Chapter 15.

If you have ever tried to run a highly interactive application program smoothly, you know it can be rather problematic. Users often also insist on a certain grade of service in terms of maximum response limits and mask change times. Therefore, latency in production terminal server environments should be less than 100 milliseconds (ms). At 150 to 200 ms, delays are very obvious, and at 300 to 500 ms, users normally no longer tolerate system behavior.

A simple method to measure latency between two network nodes (for example, RDP client and terminal server) is the ping command. This command invokes a service program for measuring connections to a remote computer. Through echo request and echo reply packages, ping determines whether a certain IP node (that is, a certain computer system) within a network is operational.

A typical ping command contains a number of parameters that further specify the command’s behavior. Checking latency between RDP clients and terminal server would look like this:

ping –t –n 20 –l 200 <Servername>

Table 3.2: Selected Parameters of the Ping Command

Parameter	Description
-t	Continuously sends ping signals to the computer specified.
-N number	Sends the number of echo packages defined. Default = 4.
-l length	Sends echo packages with the amount of data specified by length. Default = 32 bytes, maximum value = 65,527 bytes. The typical length of an RDP package is between 50 bytes and 1000 bytes.

Network Monitor Functions

The network monitor allows you to define capture filters so that only certain network data is stored for analysis. The filter options can be based on source and destination addresses of the network adapter, source and destination addresses of the protocol, and on pattern matching in the network data stream. As soon as a capture is complete, further filters can restrict the display or limit it to certain protocols.

To simplify the analysis, the collected binary network data streams are interpreted based on their protocol, edited, and displayed. Protocol headings are clearly separated from usable data. They are easy to read and, with a little practice, to understand.

If problems originate in the network during a terminal server session, the network monitor captures all the corresponding information. If you save the collected data in a file, you can analyze it later with the help of the network monitor. Sending this file to a remote specialist allows remote troubleshooting, too.

Caution

The network monitor is a powerful tool for collecting and displaying data streams in corporate networks. It is also an excellent means of targeting, capturing, and displaying unencrypted data. This also applies to unencrypted passwords. Therefore, an administrator should understand how a network monitor works in his or her corporate environment. The Microsoft Systems Management Server network monitor can trace all other network monitors operating on the network.

Network monitor agents greatly simplify terminal server management. All you need is an available network monitor and a basic understanding of the communication mechanisms to the different clients. With network monitor agents, you can also monitor data transmission between terminal servers and domain controllers, and between file and print servers. As you see, this is a valuable tool, especially for solving broadcast problems in complex, structured networks using routers and switches.

Analysis of the RDP Data Stream

The following system structure allows easy measurement of an RDP data stream:

1. A terminal server called WTSUS1 is waiting for a connection request from an RDP client.

2. The user of a client platform called STETBACH and an installed RDP client connects to the terminal server.

The network monitor is launched on the administrator console of the terminal server. Data collection begins just before connecting to the client. Now, the RDP session can be easily monitored in the network monitor. After the user of the RDP client has logged on and performed several standard actions on the desktop of the terminal server session, data collection can be stopped. You can now analyze the data by displaying it in a special window by pressing the F12 key. Alternatively, you can save the data in a file and reload and analyze it at a later time.

You will see a summary of the individual packages at the top of the analysis window of the network monitor. The protocol information recognized appears in the middle, and the raw data is displayed in hexadecimal form at the bottom. The summary includes an item number, the time the collection process started, the hardware’s source and target address, the protocol, a description, and the network’s source and target address. Because the network monitor does not have a module for recognition of RDP, all packages are labeled with the TCP protocol.

The first three frames of the RDP sequence contain the TCP connection (this corresponds to item numbers 1 to 3 in Figure 3-8). If you analyze the first two rows in detail, you can recognize the TCP connection by the set synchronization bits (S in the TCP protocol information flags in the middle part of the window panel). Frames two and three contain the acknowledgement A for the previous frame. With frames four and five (item numbers 4 and 5), you establish an ISO-over-TCP/IP connection according to Internet standard RFC 1006. This is necessary because the T.120 protocol family, on which RDP is based, requires an X.224-compatible transport protocol.

Frame six (item 6 in Figure 3-8) initializes the T.120 connection and transmits the first parameters for the RDP protocol. You can clearly see the client’s name here (from byte D2, the string reading STETBACH). It is a bit more difficult to find the client’s resolution of 800 x 600 pixels. You can find it from position C2 in hexadecimal form as 0320 x 0258 pixels with exchanged bytes. In the following frames, encryption is negotiated, the client user logon name is sent, and the logon screen is transmitted.

If you are willing to spend some time on this, you can decipher the basic mechanisms of the RDP protocol as described earlier. This can be very helpful for diagnosing errors and solving problems in network communication.