The Jason Salas Experience

Software developers and system architects, whether they know it or not, are approaching a rather intriguing crossroads of possibilities in terms of achieving efficient data exchange and creating compelling user real-time experiences. Two established platforms - the event-driven callbacks method of the aptly-named Web Hooks and the bandwidth-friendly federated communications of the eXtensible Messaging and Presence Protocol (XMPP) - are rapidly emerging as major players for managing proper communications across the Internet.

It's important to know the strengths and benefits of each technology and recognize which technique is best suited to achieve your objectives. So let's consider each platform's effectiveness.

Scalability
Anytime a Web Hooks and XMPP discussion arises, scalability is typically the first criterion challenged. Politics notwithstanding, the general consensus seems to favor XMPP for rapid-fire, high-volume, push-based traffic. Web Hooks on the other hand stake a strong claim in scenarios seeing smaller, less frequent calls to public APIs. The time-tested web growth strategy to create server farms, server gardens, utilize edge computing or to go the cloud route all exist and are applicable for callbacks.

Yet opinions still vary on exactly how scalable XMPP-based stacks might truly be, noting performance and latency concerns in cases where servers have to manage massively large rosters in IM applications - Twitter's May 2008 removal of its XMPP firehose data stream being the canonical example. MetaJack suggests such cases as being candidate for componentized or S2S bot design, as opposed to the bot-as-client method so often used.

Load Balancing
Not many, if any, case studies or formal patterns are in existence to guide us in designing well-tuned networks in cases of XMPP applications. Again, the decision tree on whether to scale up or scale out in Web Hooks scenarios, based on the number of requests against a server, is well documented. The most important thing to remember when comparing XMPP and HTTP stacks is that the traffic levels are different - the former being low-bandwidth push of XML fragments to users based on availability, the latter being the traditional request/response method.

Cost
With Web Hooks relying on a URL registering a server script to an event on a remote application, a web server with appropriate platform support/permissions is all that's needed to get up and running. Most people can put their own development web servers online, and free services like AppJet and GitHub exist for no-cost script authoring/storage, but are subject to downtime and other inconveniences common to third-party hosting.  Conversely, free open source Jabber servers and clients are legion but are arguably more difficult to setup than Apache or IIS. And hosted services for instant messaging like Jabber.org and Google Talk have historically throttled clients incurring heavy loads (i.e., Gnip's main gripe). The Pownce team postulated that clients running their own web servers will greatly outnumber those maintaining XMPP servers.

Speed/Efficiency
XMPP's redeeming quality is its efficient use of network resources, sending small messages only to those clients who are capable of receiving them at that moment, as well as supporting out-of-band communications for larger payloads like voice and video. But the feature is largely limited to messaging. Callbacks can perform whatever functionality you program them to do, with the tradeoff being variable consumption levels of bandwidth, memory and processing cycles. So there's the rub.

Reliability
Yahoo! has stated that a major architectural dealbreaker in choosing pure XMPP PubSub over Web Hooks for its Fire Eagle real-time geolocation service was a concern about a lack of presence data, not wanting to unnecessarily send messages to hosts that may be offline, as well as the taxation of enforcing SSL for each and every endpoint. (Comparatively, Google Latitude provides such geolocation via triangulation of Wi-Fi access points and radio towers.)

Security
XMPP features a secure environment through enforced server dialbacks and inherent support for SSL and SASL. However, XMPP Standards Foundation executive director Peter St. Andre begrudgingly admitted that after a decade of using the protocol, the community hasn't been able to implement an official encryption format. From the Web Hooks perspective, SSL, authorization and other traditional security measures can be leveraged over existing HTTP techniques, as any of the major web frameworks supports their own cryptography libraries.

Network Access
Generally speaking, web API calls can directly move through Port 80 without additional configuration. The XMPP community has made some very impressive strides with NAT traversal, facilitating messages seamlessly passing through the corporate firewall.

Standards Compliance
Google developers Brad Fitzpatrick and Bret Slatkin expressed their frustration at how XMPP development has fragmented due to the fact that XEP-0060 (the PubSub extension) is more convoluted than the Core spec, coupled with architects not following the protocol extension as written. (This clutter gave rise to the development of the Personal Eventing Protocol, a greatly reduced subset of PubSub.) Conversely, the web services model, operating over standards such as SOAP, HTTP, WSDL and other structures, is widely understood and adhered to in client apps.  Web Hooks, being a software pattern not bound to a formal spec, also has a little bit more leeway than the rigidity of a full protocol.

XMPP over HTTP
During that same presentation, the Googlers also noted how a simple framework can be built on XMPP using JavaScript techniques via their pubsubhubbub sample app. This evoked an immediate reaction from supporters of BOSH and Web Hooks, citing how web services can leverage keep-alive HTTP sessions and asynchronous calls. Let's face it - AJAX Comet hasn't really taken off the way many had hoped. (Comet vs. BOSH merits its own discussion altogether.) A major area of development for achieving near-real-time notifications of web data is Atom feeds over XMPP PubSub, which may prove to bridge the two schools of thought.

Public Perception
While each technology hasn't quite hit the development mainstream just yet, cursory views on it vary. The big knock against XMPP today is twofold: people see XMPP merely as IM and not a flexible messaging framework, and that it's not HTTP. Pundits quickly point out HTTP's many shortcomings in cross-network communications, while others remind us not to forget its simplicity. The appropriately named "Web Hooks" has more of a natural grokability to it.

Development Model
Any modern-day developer knows server-side coding in some language, so adopting Web Hooks makes for a very smooth transition. XMPP's use of long-lived TCP connections to stream XML fragments between servers might mean web devs have to delve into the unfamiliar world of socket programming; and perhaps going even deeper, having to get into GUIs, servers, and possibly bots. This makes for a sizable learning curve not everyone is going to want to undertake.

Platform Accessibility
One area where Web Hooks have a distinct advantage over XMPP is their accessibility across developmental platforms. A handful of XMPP clients, libraries and SDKs exist for .NET and Mono, but they're still vastly outnumbered by those for Ruby, Python, Erlang, PHP, Java, Perl and the like. Ironically, the Jabber project, the IM platform driving XMPP, has been around nearly as long as server-side web development, but has really only seen a large interest uptake in recent history.

Conclusion
While advocates of both Web Hooks and XMPP passionately defend their preferred system of data exchange, friendly and at times pragmatically aggressive challenges are being lobbed across forums, blogs and IRC channels about HTTP's survivability as we move toward delivering real-time experiences. Some have even proposed abandoning the moniker "web services" altogether, suggesting we using "online services" by virtue of political correctness.

There's little argument against the fact that XMPP scales better. But the low barrier to entry of the Web Hooks model of registering callbacks is orders of magnitude more achievable than setting up a back-end architecture that to many IT shops will be foreign territory. The learning curve, supported platforms, infrastructural requirements, security issues and ease of deployment make this a very compelling discussion.

// posted by Jason Salas @ 5:22 PM