Four short links: 7 September 2009

XMPP, Future of Web Frameworks, Infrastructure Stories, Better Email Client

1. App Engine Now Supports XMPP (Jabber) -- messaging servers, whether XMPP or PubSubHubBub, are becoming an increasingly important way to loosely join the small pieces. Google's incorporation of XMPP into GAE reflects this (and the fact that Wave is built on XMPP). (via StPeter on Twitter)
2. Snakes on the Web (Jacob Kaplan-Moss) -- The best way to predict the future of web development, I think, is to keep asking ourselves the question that led to all the past advances: what sucks, and how can we fix it? So: what sucks about web development? An excellent and thought-provoking talk about the possible directions for improvement in web framework design.
3. Ravelry (Tim Bray) -- We’ve got 430,000 registered users, in a month we’ll see 200,000 of those, about 135,000 in a week and about 70,000 in a day. We peak at 3.6 million pageviews per day. That’s registered users only (doesn’t include the very few pages that are Google accessible) and does not include the usual API calls, RSS feeds, AJAX. [...] We have 7 servers running Gentoo Linux and virtualized into a total of 13 virtual servers with Xen. [...]". Interesting technical and business discussion with an unexpected busy site.
4. So's Your Facet: Faceted Global Search for Mozilla Thunderbird -- email clients are LONG overdue for improvement. Encouraging to see an active and open research project to improve it from the folks at Mozilla Messaging.

Four short links: 9 June 2009

Biological Radio, Laggy Smart Grids, API Moneys, and Pubsub Server

1. Drawing Inspiration From Nature To Build A Better Radio -- based on the design of the cochlear, this MIT-built RF chip is faster than others out there, and consumes 1/100th the power. Biomimicry and UWB radio are on our radar.
2. Why the Smart Grid Won’t Have the Innovations of the Internet Any Time Soon -- While it’s significant that utilities are starting to build out smart grid infrastructure, utilities are largely opting for networks that provide connections that are far from real time, and this could stifle the desired innovation. [...] smart meter data that is pushed to Google’s PowerMeter energy tool has to make its way back to the utility before it can be sent to Google. That means that even for Google’s energy tool, there can be both a significant delay before information reaches consumers, and significant gaps in energy data details. These delays and gaps can undercut the premise of how smart meter technologies will empower consumers to make decisions about their energy use based on real-time costs. Smart grids (houses and devices able to take use of instantaneous pricing changes) have the potential to help us with our energy obesity problem, but the architecture must be right.
3. API Value Creation, Not Monetization -- On the side of the unexpected but interesting outcomes, Kevin said they have seen a flurry of internally developed business applications. In the past many valuable, internal-facing projects were turned down because the programs had to meet strict top line to bottom line ratios. With the availability to data and services, many teams within the company now have access to things they didn’t in the past, and project costs have been minimized. Throughout the company, consumers of the API have been able to launch successful projects that have created additional revenue and have reduced the overall development costs for new projects. Some solid numbers and names to help convince businesses to offer APIs, though the battle is still much harder than it should be.
4. Watercoolr -- a pubsub server for your apps. A channel is a list of URLs to be notified whenever a message is posted to that channel. Clever little piece of infrastructure for web apps, embodying the Unix philosophy of small tools that each do one thing very well. (via straup on Delicious)

AT&T Fiber cuts remind us: Location is a Basket too!

The fiber cuts affecting much of the San Francisco Bay Area this week are similar to the outages in the Middle East last year (radar post), although far more limited in scope and impact.   What I said last year still holds true and is repeated below: 

From an operations perspective these kinds of outages are nothing new, and underscore why having "many eggs in few baskets" is such a problem. I believe we will see similar incidents when we have the first multi-datacenter failures where multiple providers lose significant parts of their infrastructure in a single geographic area.

Remember: Don't put all your eggs in one basket... and Location is a basket too!

To really understand the issue, I recommend Neal Stephenson's incredible (and lengthy) Wired article from 1996 entitled "Mother Earth Mother Board":

[...] It sometimes seems as though every force of nature, every flaw in the human character, and every biological organism on the planet is engaged in a competition to see which can sever the most cables. The Museum of Submarine Telegraphy in Porthcurno, England, has a display of wrecked cables bracketed to a slab of wood. Each is labeled with its cause of failure, some of which sound dramatic, some cryptic, some both: trawler maul, spewed core, intermittent disconnection, strained core, teredo worms, crab's nest, perished core, fish bite, even "spliced by Italians." The teredo worm is like a science fiction creature, a bivalve with a rasp-edged shell that it uses like a buzz saw to cut through wood - or through submarine cables. Cable companies learned the hard way, early on, that it likes to eat gutta-percha, and subsequent cables received a helical wrapping of copper tape to stop it.

[...] There is also the obvious threat of sabotage by a hostile government, but, surprisingly, this almost never happens. When cypherpunk Doug Barnes was researching his Caribbean project, he spent some time looking into this, because it was exactly the kind of threat he was worried about in the case of a data haven. Somewhat to his own surprise and relief, he concluded that it simply wasn't going to happen. "Cutting a submarine cable," Barnes says, "is like starting a nuclear war. It's easy to do, the results are devastating, and as soon as one country does it, all of the others will retaliate."

As the capacity of optical fibers climbs, so does the economic damage caused when the cable is severed. FLAG makes its money by selling capacity to long-distance carriers, who turn around and resell it to end users at rates that are increasingly determined by what the market will bear. If FLAG gets chopped, no calls get through. The carriers' phone calls get routed to FLAG's competitors (other cables or satellites), and FLAG loses the revenue represented by those calls until the cable is repaired. The amount of revenue it loses is a function of how many calls the cable is physically capable of carrying, how close to capacity the cable is running, and what prices the market will bear for calls on the broken cable segment. In other words, a break between Dubai and Bombay might cost FLAG more in revenue loss than a break between Korea and Japan if calls between Dubai and Bombay cost more.

The rule of thumb for calculating revenue loss works like this: for every penny per minute that the long distance market will bear on a particular route, the loss of revenue, should FLAG be severed on that route, is about $3,000 a minute. So if calls on that route are a dime a minute, the damage is $30,000 a minute, and if calls are a dollar a minute, the damage is almost a third of a million dollars for every minute the cable is down. Upcoming advances in fiber bandwidth may push this figure, for some cables, past the million-dollar-a-minute mark. [Link]

(Image source: http://www.flickr.com/photos/mundane_joy/2301368102/)