HTTP Tunneling Revealed: Part 1/3 - What is HTTP Tunneling? (Page 2 of 6 )
The first time I read "HTTP tunneling" was as a term from Karl Moss's "Java Servlets" (ISBN: 0071351884); but this technology has remained in the surrounding, always with a different name. It's distributed EDI, EDI over HTTP-RPC, grabbing information from other servers, external SSI, super glue for the web, sharing information from other websites, tearing the pages, importing and exporting data over HTTP, get and post automation, ripping off web pages, importing part of pages, or it can be even be read in simple questions such as "how can I display CNN.com's headlines on my webpage?".
It sounds strange, but actually, there was no standard mechanism for such information sharing and the techniques involved were raw and unreliable, becoming analogous with stealing information from others. From tracing the tables, beginning and ending via string search operations, or more sophisticated ways – such as regular expressions – the necessity of having a common sharing standard was strongly felt by developers. This was what finally led to the development of what we now call SOAP (Simple Object Access Protocol) and web services.
Distributed EDI (Electronic data interchange) automation has eminent usages equally on the web and desktop; from meta search engines displaying keyword search results in multiple UI's to spell checks, newspaper or article references, image galleries, speeding up dynamic pages (caching), sending SMS, monitoring the web for new information on subjects, money exchange rates & stock rate updates, time servers, live auction stats, news tickers, monitoring researches and trends over time, user interfaces in desktop application for web based data and more!
With the implementation of HTTP 1.0 RFC, GET and POST emerged as two frequently used methods for requests and the programming emphasis came towards exploiting the power of URI's:
As it starts, Microsoft's support for tunneling was through the Inet library controls, aka the Microsoft Internet Transfer Control. Using this control was quite easy, like other COM components; Instantiate, set the properties and you're away. Beside the basic GET/POST/Head support, other features of the Internet controls via INET include HTTP via proxy support (no SSL off course). Accesstype attributes are used to set the proxy, for example:
Inet.AccessType = icDirect
The constants values for Accesstype are:
IcUseDefault: The default settings found in the registry to access the Internet
icDirect or 1: Direct connection to the Internet
icNamedProxy or 2: An enumerated icNamedProxy to use the proxy server specified in the proxy property
The protocol attribute is intuitively simple; it can be set toa numeric value or its enumeration:
IcDefault = 1
icFTP = 2 'FTP. File Transfer Protocol
icGopher = 3 'Gopher Const
icHTTP = 4 'HTTP. Hypertext Transfer Protocol
icHTTPS = 5 'Secure HTTP
RemotePort is the port to connect with, and is 80 by default. The RequestTimeOut property can be set for possible delays in downloading time. To assign the URL, it we set the URL attribute.
OpenURL returns the data as a string, which can be assigned and held as stream of bytes. For instance:
resultant = inet.OpenURL()
For detailed information on error messages and constants of Internet controls, I found Don Hayward of Insight Graphics article very useful. See it here.
This control, at first glance looks quite quick and easy to use, but later on, my personal experience was quite bitter. It crashes often, especially during FTP operations and eats lot of CPU cycles during simple operations.
Microsoft published this error as an OpenURL method bug in the Internet control, later, as the WinInet limited connections per server, and then finally WinInet not supported for use in services.
While developing an application to load a text file from a UNIX based FTP site, I found using the OpenURL method on the Internet transfer control for FTP URL’s causes my web server crash and it skyrockets 100% CPU utilization.