Cross-Domain Proxies and the Browser-Server Dialogue - Decisions about Cross-Domain Proxies
(Page 2 of 4 )
What external content will be accessed?
In theory, anything you can do with a browser can theoretically be accomplished by an automated HTTP client. However, there are likely to be legal constraints, and technical challenges too if there is serious usage of JavaScript or browser plugins. Furthermore, your script will be vulnerable to changes in the site layout, and many content providers have deliberately performed subtle changes to prevent automated access. Thus, relying on the ever-increasing collection of public APIs and structured data would make a better choice.
You can find a collection of public APIs at http://wsfinder.com. Some popular APIs include Amazon, Delicious, EvDB, Flickr, Google Maps, Google Search, Technorati, and Yahoo Maps.
How will you connect to the external server?
If you're scraping content directly from a public web site, you'll need to use the underlying web protocol, HTTP. And even if you're talking to an API, you'll probably be communicating with HTTP anyway. In some cases, API publishers will provide code to access the content, rather than just publishing the specification. Also, with more complex web services protocols like SOAP, you don't have to write a lot of low-level code yourself. In many cases, though, the easiest thing to do is talk HTTP and manually format and parse messages yourself.
Many scripting languages feature built-in libraries to communicate in HTTP and are often well-equipped for Cross-Domain Proxy implementations due to strong support for regular expression manipulation. A PHP example is featured in the code example that follows shortly.
In the Java world, for instance, the standard API provides some relatively low-level support for HTTP clients, but you can use some of the web site testing libraries to quickly extract content from external sites. HttpUnit is a good example, and it also has some support for JavaScript manipulation. To grab some content:
import com.meterware.httpunit.WebConversation;
import com.meterware.httpunit.GetMethodWebRequest; import com.meterware.httpunit.WebRequest;
public class PatternSorter {
public String getContent(String url) {
try {
WebConversation wc = new WebConversation();
WebRequest request = new GetMethodWebRequest(url);
String response = wc.getResponse(request).getText();
} catch(Exception ex) {
throw new RuntimeException("Could not get content from " + url, ex);
}
}
public static void main(String[] args) {
System.out.println ("http://ajaxpatterns.org");
}
}
How will you deal with errors and delays in accessing the service?
At times, there will be errors accessing an external web service. Think twice before attributing blame because the problem might be at your end, or somewhere along the network. This has implications for any error messages you might show users. Ideally, you should be able to detect that an error has occurred, and then log the details for immediate attention. In responding to the user, you have several options:
- Ensure you have a good server-side framework in place to detect errors, so they're not directly--ignorantly--passed on to users. In addition, be sure to detect timeouts and handle them proactively; e.g., respond with an appropriate error message.
- In some cases, you will simply have to admit failure and show an error message to users. If possible, suggest an alternative or tell them when they should try again.
- Provide reduced functionality.
- Switch to a prearranged substitute web service.
- Rely on cached results, if they're still relevant.
Under what licensing terms will you access the remote service?
The legals of invoking Web Services (Chapter 6) are tricky, vaguely-defined, and always evolving. Some services are open for public use, others require you hold an API key, and others are critical enough to warrant authentication via digital signatures. In each case, there are usage terms involved, even if they're not explicit, and you will need to investigate issues such as the authentication mechanism, the number of queries per day, how the data will be used, server uptime, and support level.
Next: Real-World Examples of Cross-Domain Proxies >>
More JavaScript Articles
More By O'Reilly Media
|
 This article is excerpted from chapter 10 of the book Ajax Design Patterns, written by Michael Mahemoff (O'Reilly, 2006; ISBN: 0596101805). Check it out today at your favorite bookstore. Buy this book now.
|
|
