Evolution of XML Protocols

The enabling technology behind Web services is built around XML protocols XML protocols govern how communication happens and how data is represented in XML format on the wire. XML protocols can be broadly classified into two generations. First-generation protocols are based purely on XML 1.0. Second-generation protocols take advantage of both XML Namespaces and XML Schema. SOAP is a second-generation XML protocol.

First-Generation XML Protocols

There were many interesting first-generation XML protocol efforts. They informed the community of important protocol requirements and particular approaches to satisfying these requirements. Unfortunately, very few of the first-generation XML protocols achieved multi-vendor support and broad adoption. Two are worth mentioning: Web Distributed Data Exchange (WDDX) and XML-RPC.

WDDX

WDDX provides a language-and platform-neutral mechanism for data exchange between applications. WDDX is perfect for data syndication and remote B2B integration APIs because it is all about representing data as XML. For example, Moreover Technologies, the Web feed company, exposes all its content through a WDDX-based remote API. Access http://moreover.com/cgi-local/page?index+wddx with an XML-aware browser such as Internet Explorer and you will get a WDDX packet with current headline news. A simplified version of the packet is shown in the following example. You can see from it that the data format is a recordset (tabular data) with three fields containing the URL to the full article, its headline text, and the publishing source:

<wddxPacket version="1.0">
   <header/>
   <data>
      <recordset rowCount="2" fieldNames="url,headline_text,source">
         <field name="url">
            <string>http://c.moreover.com/click/here.pl?x22535276</string>
            <string>http://c.moreover.com/click/here.pl?x22532205</string>
         </field>
         <field name="headline_text">
            <string>Firefighters hold line in Wyoming</string>
            <string>US upbeat as China tensions ease</string>
         </field>
         <field name="source">
            <string>CNN</string>
            <string>BBC</string>
         </field>
      </recordset>
   </data>
</wddxPacket>

Allaire Corporation (now Macromedia, Inc.) created WDDX in 1998. WDDX is currently supported in many environments and is flexible enough to handle most useful datatypes (strings, numbers, booleans, date/time, binary, arrays, structures, and recordsets), but it cannot represent arbitrary data in XML. It is an epitome of the 80/20 rule: flexible enough to be useful yet simple enough to be broadly supported. Because WDDX is not bound to any particular transport, applications can exchange WDDX packets via HTTP, over e-mail, or by any other means. Many applications persist data as XML in a relational database using WDDX.

XML-RPC

XML-RPC is an RPC protocol introduced in the market in 1998 by Userland. XML-RPC supports a set of datatypes similar to that supported by WDDX and uses HTTP as the underlying transport protocol. Because of its simplicity, XML-RPC enjoyed good multi-vendor support. Here's an example XML-RPC method call and response:

<methodCall>
   <methodName>NumberToText</methodName>
   <params>
      <param>
         <value><i4>28</i4></value>
      </param>
   </params>
</methodCall>
...
<methodResponse>
   <params>
      <param>
         <value><string>twenty-eight</string></value>
      </param>
   </params>
</methodResponse>

First-Generation Problems

Although first-generation XML protocols have been and still are very useful, their simplicity and reliance on XML 1.0 alone causes some problems.

First-generation protocols are not very extensible. The protocol architects had to reach agreement before any changes were implemented, and the protocol version had to be revved up in order to let tools distinguish new protocol versions from old ones and handle the XML appropriately. For example, when XML-RPC and WDDX added support for binary data, both protocols had to update their specifications, and the protocol implementations on all different languages and platforms supporting the protocols had to be updated. The overhead of constantly revising specifications and deploying updated tools for handling the latest versions of the protocols imposed limits on the speed and scope of adoption of first-generation protocols. Second-generation protocols address the issue of extensibility with XML namespaces.

The second problem with first-generation protocols had to do with datatyping. First-generation XML protocols stuck to a single Document Type Definition (DTD) to describe the representation of serialized data in XML. In general, they used just a few XML elements. This approach made building tools supporting these protocols relatively easy. The trouble with such an approach is that the XML describing the data in protocol messages expressed datatype information and not semantic information. In other words, to gain the ability to represent data in XML, first-generation XML protocols went without the ability to preserve information about the meaning of the data. Second-generation XML protocols use XML schema as a mechanism to combine descriptive syntax with datatype information.

To sum things up, the need to provide broad extensibility without centralized standardization and the need to combine datatype information with semantic information were the driving forces behind the effort to improve upon first-generation efforts and to create SOAP, the de facto standard XML protocol for modern Web services and B2B applications.