A.6. HTTP Programming Libraries

When all else fails, you may have to resort to programming to perform a request or a series of requests that would be impossible otherwise. If you are familiar with shell scripting, then the combination of expect (a tool that can control interactive programs programmatically), netcat, curl, and stunnel may work well for you. (If you do not already have expect installed, download it from http://expect.nist.gov.)

For those of you who are more programming-oriented, turning to one of the available HTTP programming libraries will allow you to do what you need fast:

libwww-perl (http://lwp.linpro.no/lwp/)

A collection of Perl modules that provide the functionality needed to programmatically generate HTTP traffic.

libcurl (http://curl.haxx.se/libcurl/)

The core library used to implement curl. Bindings for 23 languages are available.

libwhisker (http://www.wiretrip.net/rfp/lw.asp)

A Perl library that automates many HTTP-related tasks. It even supports some IDS evasion techniques transparently. A SecurityFocus article on libwhisker, "Using Libwhisker" by Neil Desai (http://www.securityfocus.com/infocus/1798), provides useful information on the subject.

Jakarta Commons HttpClient (http://jakarta.apache.org/commons/httpclient/)

If you are a Java fan, you will want to go pure Java, and you can with HttpClient. Feature-wise, the library is very complete. Unfortunately, every release comes with an incompatible programming interface.