Importance of Innovation in Finance & BarCampBank

by Jesse Robbins | @jesserobbins | comments: 2

“Progress is not the mere correction of evils. Progress is the constant replacing of the best there is with something still better.” -Edward Filene

Two years ago, when we were organizing the first BarCampBank in the US, many people found it hard to believe that banks & credit unions could a place for meaningful grassroots innovation. Even crazier was the idea of organizing an unconference to begin bringing open source, transparency, identity, and community into the very closed world of banking & finance.

Since then the BarCampBank idea has turned into a movement. There have been over 14 events all over the world, and many of the ideas generated are beginning to turn into action.

To me, the global financial system is a platform that exists to “create more value than it captures”. Tim explained this in his Work on Stuff that Matters post, saying:

“A bank that loans money to a small business sees that business grow, perhaps borrow more money, hire employees who make deposits and take out loans, and so on. The power of this cycle to lift people out of poverty has been demonstrated by microfinance institutions like the Grameen Bank. Grameen is clearly focused on creating more value than they capture; not so the like of Fannie Mae and Freddy Mac, or WaMu, or many of the other failed financial institutions involved in the current financial meltdown.”

There has never been a more important time to bring meaningful innovation into the financial system, and there has never been more opportunity for our community to make it happen.

The next event is occurring this weekend (April 25-26, 2009) on Treasure Island in San Francisco.

After that, the following events are planned:

• BarCampBankVegas is set for May 2, 2009.
• BarCampBankCharleston2 is set for June 13, 2009
• BarCampBankGermany is set for October 23-25, 2009

Experience Syndication: Powered by Zappos

by Joshua-Michéle Ross | @jmichele | comments: 5

I have been thinking a lot about the new Powered by Zappos service.

According to Zappos:

Powered by Zappos (PBZ) is a feature Zappos.com offers to its partners where we design, host, fulfill and own a partners web site. Our goal is to provide Zappos customers as well as our partner's customers with the best possible service experience. By building partnerships through PBZ we can deliver great service to more people. Ultimately if you are purchasing through a PBZ site you are making a purchase from Zappos, your package with free shipping will even arrive in a Zappos.com box and you will receive all the great benefits Zappos has to offer.

For lack of a better term, I am calling this “experience syndication” since PBZ is essentially syndicating the value of the entire experience - not just one aspect such as content or business process or infrastructure. A quick Google search reveals that would-be competitors such as Clarks Shoes, Stuart Weitzman, Bostonian Shoes etc. are already utilizing PBZ.

I am of two minds on PBZ. As a business strategy I think it is a brilliant play. Zappos is syndicating the very thing that makes them great - the entire experience; from browsing to buying and especially post-sales support. In the hyper-competitive world of ecommerce, individual, mid-market brands like Clarks simply can’t compete with that so they better join. It also raises an interesting question. What other companies might look at syndicating their experience?

On the negative side: I am a big fan of Zappos but I am not blind to the fact that the more successful the PBZ offering gets, the more power they will wield over individual companies that depend on them for survival. In that sense I fear that Zappos may ultimately do to shoe companies what Amazon appears headed to do to book publishers - and what Walmart has already done to countless small brands - put them in a death grip and squeeze the life out of them. Let's hope I am wrong.

The U.S. iTunes App Store

by Ben Lorica | @dliman | comments: 7

With the iTunes App store now over a month old, I decided to look closely at data from the U.S. store over the last three weeks. While sales numbers are not publicly available, Apple publishes overall as well as category-level rankings. There are currently just over 1,800 (paid and free) applications in the App store, double what it was three weeks ago. Games is the largest category with about 500 applications (roughly 27% of all apps), up 87% from three weeks ago. Puzzles, Arcade, and Board games are the three largest Gaming subcategories:

The fastest-growing category, Education, more than tripled over the last three weeks.

The average price per paid app is around $5.50, with 94% of apps priced at $10 or less. Prices vary considerably by category with expensive apps skewing the average price in a category: a single application priced at $449 drove up the average price of Finance apps to more than $22. Excluding the top and bottom 1% priced apps, the average price of an iPhone application is about $5.20. Similarly, by removing the top and bottom priced app in each category, we get a more reasonable estimate of the average price per app within a category (click here for details).

The Book category is comprised mostly of ebooks and while there are over 150 such "apps", it was the only category not represented in the Top 100 rankings:

In contrast, more than 1 in 10 of all Music apps were among the Top 100 Paid Apps:

Looking beyond the Top 100 paid apps to all paid iPhone applications, the best-performing categories (in terms of popularity) are Music, Weather, Navigation, Lifestyle, and Entertainment (click here for details).

On average, app providers have slightly over one app each, with 25 (out of the close to 1,100) providers accounting for about 21% of all paid apps:

Most providers had 0 or 1 app listed in the Top 100 Paid Applications, with the following exceptions: Hottrix, Pangea Software, Inc., Phase2 Media, telience.com, and Electronic Arts all had 2 apps in the Top 100 list. For now, the cohort of web developers who dominate the Facebook application platform have been unable to make similar inroads in the iPhone platform. Perhaps it's time to brush up on Cocoa and Objective-C?

Kaminsky DNS Patch Visualization

by Jesse Robbins | @jesserobbins | comments: 4

Dan Kaminsky has posted the details of the widespread DNS vulnerability. Clarified Networks created this visualization of DNS patch deployment over the past month:

Red = Unpatched
Yellow = Patched, "but NAT is screwing things up"
Green = OK

Facebook Growth By Country and the Slowdown in App Usage

by Ben Lorica | @dliman | comments: 18

With the Facebook Developers conference slated for later this week, I thought it would be a good time to give a brief update of a previous post on Facebook demographics. What follows are recently published number of users by country and region, along with growth rates for select regions and countries. Over the last four weeks, the fastest growing regions were South America, Central America and the Carribean:

While Facebook grew double-digits in Asia it did so from a relatively small base (approx. 3.7 million users), in a region with hundreds of millions of potential users. Of the countries in South and Central America, Chile is worth highlighting (up 67.5% from four weeks ago). As several Radar readers predicted, Facebook has grown steadily in Chile where it now has over 2.2 million users (around 14% of the population). In other parts of the Americas, Hi5 and Orkut remain the largest social networks:

Looking closely at the top 30 countries, a few European countries have grown more than ten percent over the last four weeks (France, Spain, Germany, Italy), with France having the most number of users (approx. 2.5 million). Skyrock remains the largest social network in France. Norway saw a decline but is still home to more than a million Facebook users. We will continue to track how Facebook is doing vis-à-vis other leading regional social web sites and whether their disputes with other companies affect their growth rates.

As far as recent trends in the Facebook app platform (the subject of this week's f8 conference), we have detailed reports (here and here) on the subject. At the last Graphing Social Patterns conference, Roger Magoulas provided highlights of our most recent findings. The number of published apps continues to grow steadily (to over 32K) but total usage remains flat. Besides the fact that the top 10% of apps account for 98% of total usage, aspiring Facebook app developers should know that only about 6% of apps average at least 500 active users per day:

(For specific tips on how to launch and build successful Facebook apps, consult this O'Reilly Radar Report.) Finally, as I noted in a previous post, the most popular applications on the Myspace platform continue to account for slightly less users than their Facebook counterparts.

An ESB for the Web?

by Jim Stogdill | @jstogdill | comments: 14

I spend a great deal of my time encouraging "enterprise people" to think more like "web people." Focus on adoption, use platforms to enable emergent capability, build the "generative enterprise," and that sort of thing.

So, imagine my surprise when I saw the web acting a bit like the enterprise with the launch of Gnip.

As the web moves toward a network of widespread transactional API's, each with it's own vocabulary, it is starting to look a lot like a legacy enterprise writ large or maybe like an industry eco-system. So we shouldn't be surprised to see web developers turning to solutions that their enterprise colleagues would find familiar.

Anyone who has spent more than five minutes in the enterprise world talking about SOA in the last five years (or spent time building "trading platforms" for industry consortiums prior to that) has probably drawn a picture on a whiteboard that looks something like this (see, almost identical):

Whether you have integrated line of business applications inside the enterprise or connected trading partners within an industry, that N squared connection problem will resonate with your experience. Webs of poorly documented point-to-point integrations are expensive to build, expensive to maintain, and impossibly brittle when the business changes.

And now the N squared problem seems like it might be beginning to resonate with web developers too now that they have to integrate to an ever growing population of API's. Plus, on the web, the additional limitations of a port 80 based infrastructure add to the nightmare by throwing the expense of constant API polling into the mix.

So, what to do?

(continue reading)

Nokia to buy and open source Symbian

by Nat Torkington | @gnat | comments: 15

Nokia have announced their intentions to buy Symbian and open source it. It's being seen widely as a response to Google's Android, also an open source mobile operating system. I think it's easy to confuse "open source operating system" for something that will provide all the benefits of the Linux development model. As always, "open source" covers a wide range of development activities and licenses.

The license chosen for Symbian (the Eclipse Public License) is not the Linux kernel license (the GNU Public License v2.0). I suspect the EPL was chosen precisely for its terms so that handset manufacturers like Nokia are able to have their own proprietary extensions for which they do not have to give away the source. This is similar to Google's license for Android, the Apache license.

Both Google and Nokia are applications companies trying to build a mobile services platform, and they have remarkably similar assets. Google has a cloud computing strength that Nokia doesn't. They both have map information (Google drives the streets, Nokia bought NAVTEQ) and assisted GPS application ability. Nokia has hired some absolute geniuses from the ubiquitous computing world to bring network services into people's lives through the mobile phone, whereas Google's social acquisition, Dodgeball, was a catastrophe. Now they both have a handset platform. The difference is that Google's is built on modern technology and they had a chance to start with a clean slate. Symbian feels very 1990s in comparison.

The real question is what do they both hope to gain by having and open sourcing a handset operating system and its core applications? First, it's defensive (nobody wants someone else to "own the handset" and thus have a competitive advantage). But secondly, it's aggressive. They want handset manufacturers to be able to slap Android/Symbian onto the handsets, no royalties payable, you're welcome, and then ship those handsets to the carriers .... Android's default free web browser, of course, will point to Google. I imagine Nokia's strategy will be similar: they'll open source some compelling standard apps, which are the portals to get more Nokia apps and services onto the handsets.

There's a huge difference between Linux and the handsets, though, and I think it's an important one. Linux's license (the GPL) prevents people who ship Linux from including proprietary extensions. If you ship a modification to Linux, you must release the source. This means there are no privileged applications (the way Microsoft's apps used libraries that third-party apps couldn't), no proprietary competitive advantages in the kernel, and so the rate of improvement of every Linux distribution is maximized.

On Christmas Day 1914, the Germans and British soldiers on the World War I front stopped shooting each other, exchanged presents, and played football together. Essentially, the Linux kernel developer community is the Christmas Truce for the Unix platform developers--a place where they cooperate rather than compete ... because the license dictates that they do it.

Both Google and Nokia, however, have deliberately chosen licenses that don't encourage that kind of cessation to war. Proprietary competitive hardware and software can be put into any Android or Nokia phone at the appropriate level of the stack. I think this will slow down the success of their platforms and means neither will unlock the true potential of an open mobile platform. I believe true demilitarized openness is a necessary, but not sufficient, condition for open mobile platform success.

I'd love to know what you think. Am I off the mark? Have I missed a cunning strategic play? Is this, in fact, open source history being written?

Video of Rich Wolski's EUCALYPTUS talk at Velocity

by Jesse Robbins | @jesserobbins | comments: 1

Rich Wolski gave a truly impressive talk at Velocity about an open-source software infrastructure for cloud computing called EUCALYPTUS . The API is compatible with Amazon's EC2 interface, and the underlying infrastructure is designed to support multiple client-side interfaces. EUCALYPTUS is implemented using commonly-available Linux tools and basic Web-service technologies making it easy to install and maintain. Watch and learn...

You can see more videos from Velocity on Blip.tv.

Hyperic CloudStatus service dashboard launches at Velocity!

by Jesse Robbins | @jesserobbins | comments: 6

Javier Soltero just launched CloudStatus during his Hyperic sponsor session today at Velocity. CloudStatus is a public health dashboard for web services like Amazon's EC2/S3, and Google's App Engine.

Javier called to tell me about this last week after I declared that "Service Monitoring Dashboards are mandatory". This comes right after Amazon and Google had visible outages, and couldn't have happened at a better time. I'm really excited to see this idea take off, as it's something that is critical to the broad adoption of web services and cloud computing.

Service Monitoring Dashboards are mandatory for production services!

by Jesse Robbins | @jesserobbins | comments: 6

Google App Engine went down earlier today. GAE is still a developer preview release, and currently lacks a public monitoring dashboard. Unfortunately this means that many people either found out from their app and/or admin consoles being unavailable or from Mike Arrington's post on TechCrunch.

Google has a strong Web Operations culture, and there are numerous internal monitoring tools in use across the company, along with a smaller set available to customers. It's suprising that Google launched a developer platform without providing something beyond an email group, although they are by no means the first to do so.

Service Monitoring Dashboards are mandatory for production services and platforms!

• If you launch a platform that people pay you money for, you need to have a real time service dashboard. Ideally this should be decoupled from the rest of your infrastructure.
• Don't rely on platforms that lack service monitoring dashboards for production.

Many companies are initially reluctant to provide this kind of monitoring to the public, and only do so in reaction to an outage. However, it seems that every company that offers such a dashboard uses it as a source of competitive advantage.

The best example of this is trust.salesforce.com which they launched after series of outages in 2006. Amazon (eventually) launched a status dashboard for AWS, and added RSS feeds for specific services which I think is pretty cool.

Javier Soltero at Hyperic points out

1. The reports of service outages arrive long after anyone who depends on the services can possibly do anything to mitigate their effect.
2. The services themselves seem incapable of providing any visibility into the circumstances that might lead to future outages.

[...]Even TechCrunch points out that the Google Apps blog doesn’t even mention the outage. Other clouds rely on blogs such as this one, this one, or maybe even this one (from our good friends at Mosso). These are all places where outages can be discussed, but not the right means for people to find out whether it their application that crashed, or the cloud that it depends on.

(Updated:Niall Kennedy pointed out that GAE is still a preview release, and I agree that my original wording was wrong. My intent is to emphasize the importance of providing a public service dashboard and so I've edited accordingly.)

Two new open source projects at Velocity

by Jesse Robbins | @jesserobbins | comments: 3

At Velocity next week there will be two significant open source projects debuting. The first is the Jiffy: Open Source Performance Measurement and Instrumentation tool created by Scott Ruthfield and his team at Whitepages.com.

Most tools for measuring web performance come in two flavors:

• Developer-installed tools (Firebug, Fiddler, etc.) that allow individuals to closely trace single sessions


• Third-party performance monitoring systems (Gomez, Keynote, etc.) that will hit your site occasionally and report back component-level metrics (for a fee)

Neither of these tools give you real-world information on what’s actually happening with your clients—how long are pages really taking to load, what’s the real cost of client-side execution, and what’s the impact of your loading or dependency chain. This is even more important when you don’t host all of your own assets, such as when you load ads or JavaScript from third parties, for example, and you need to monitor their performance.

Thus we built Jiffy—an end-to-end system for instrumenting your web pages, capturing client-side timings for any event that you determine, and storing and reporting on those timings. You run Jiffy yourself, so you aren’t dependent on the performance characteristics, inflexibility, or costs of third-party hosted services.

The second is project is EUCALYPTUS, the Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems, presented by Rich Wolski from UCSB. This project has already started getting attention. (Many thanks to Surj Patel of Structure08/GigaOM for connecting us!)

Eucalyptus is an open-source software infrastructure for implementing "cloud computing" on clusters. The current interface to EUCALYPTUS is compatible with Amazon's EC2 interface, but the infrastructure is designed to support multiple client-side interfaces. EUCALYPTUS is implemented using commonly-available Linux tools and basic Web-service technologies making it easy to install and maintain.

The talk will focus on the design, the implementation tradeoffs we have identified in implementing Eucalyptus as an exploratory tool, and the ways in which we have chosen to address these tradeoffs in the first version of the software.

Bill Coleman to keynote Velocity

by Jesse Robbins | @jesserobbins | comments: 0

Bill Coleman has twice transformed our industry, and I'm excited to announce that he will keynote Velocity later this month. Bill is most famous for being the "B" in BEA and for leading the creation of Solaris while at Sun. He is now the CEO of his new startup, Cassatt, which "makes Data Centers more efficient".

Bill is awesome and I'm really looking forward to his keynote. He is changing the way we think about and manage Data Centers and the software that runs within them.

When we spoke earlier this week he explained how vacuum tubes created the fear of powering down servers, and how funny it is that that fear persists with people that have never seen them. (I've never made that connection as I'm "part of the problem" ;-)

At Velocity, Bill will likely talk about virtualization & efficiency, where he thinks we're headed, and the questions we need to be asking now to get there.

(Many thanks to Tim for suggesting this to Bill and making the introduction.)

Myspace/Facebook App Platforms & Total Installs

by Ben Lorica | @dliman | comments: 7

Within a few months, Myspace has quietly built an application platform with over twelve hundred applications. I previously posted a graph for Facebook app categories, in which I compared the categories using the number of active users. Unlike the older Facebook platform, Myspace only provides the number of installs:

It took a few months before Facebook started publishing active usage and I'm hopeful Myspace will follow suit.

At a comparable stage, roughly four months after launch, Facebook already had 4,300 applications. On the other hand, developers should know that the most popular Facebook applications attract a higher share of installs and active usage. After four months, the top 10% most installed Facebook apps accounted for 97% of all installs. The top 20% represented 99% of all installs. The corresponding numbers for Myspace were 91% and 96% respectively. Economists sometimes use the Lorenz Curve to visualize and measure inequality. In both application platforms, the top applications account for most of the installs, with the Myspace platform being slightly less unequal:

[Note: When I drew the preceding curves, I sorted from the most to the least installed applications.] One year after launching their platform, there are now more than 24 thousand Facebook applications! With more than 24K applications, the top 10 & 20 percent Facebook applications still have roughly the same share of total installs.

For more on the Facebook and Myspace platforms, Roger Magoulas of O'Reilly Research will present some of our most recent findings at the upcoming Graphing Social Patterns conference.

Structure and Velocity

by Jesse Robbins | @jesserobbins | comments: 4

Several people have asked me about the differences between Om Malik's Structure conference and our Velocity Web Performance & Operations conference.  Velocity is on June 23 & 24th at the SFO Mariott, and Structure follows on June 25th in San Francisco. 

The conferences are complementary: Structure discusses what is changing in internet infrastructure, and Velocity teaches how to make that change happen.

I've been recommending that anyone considering Structure make sure their engineering teams are going to Velocity.  For many technical leaders I think there is value in attending both, and I definitely plan on doing so.

The knowledge and skills learned at Velocity can be put to immediate use and will have significant impact on your business.  The reason for this is simple:

Faster, scalable, and highly available websites serve more pages to more customers in the same amount of time.

That's why we've worked hard to make Velocity the best resource for engineers to learn how to build and operate at web scale.  Here are a few examples:

Adam Jacob will give a step-by-step overview of Building an Automated Infrastructure, and then Luke Kanies will follow up with an in-depth session on Puppet.  This is the exact combination I used to explain how effective operations is a huge competitive advantage:

Luiz Barroso will describe Google's approach to energy-efficient datacenter design and management.  Applying these lessons can ultimately save millions of dollars, increase your operational agility, and decrease your environmental footprint.

Mandi Walls will teach how actionable logging can mean the difference between a 20-minute outage and a 2-hour outage while esoteric error codes are deciphered or developers are contacted to investigate.

Eric Lawrence, Program Manager for Internet Explorer, and Mike Connor, lead developer for Mozilla Firefox will explain how to optimize page performance for their respective browsers.  We'll also have demos of leading performance testing tools: HTTPwatch, Fiddler, AOL PageTest, and Firebug.

John Allspaw from Flickr will be be giving a talk about Capacity Management.  John's way of explaining both the problem and the opportunity is wonderfully straightforward:

You can check out the rest of the program and register on the Velocity site.  (Hint: You can use the code "vel08js" for a 20% discount.)  I'll be posting frequently as we add speakers and events.  I hope to see you at Velocity!

MySpace's Data Availability is not Data Portability

by David Recordon | @daveman692 | comments: 10

Yesterday MySpace, Yahoo!, eBay, Photobucket (also owned by News Corp), and Twitter announced the Data Availability Initiative. While I could write at length about how this shows the big companies have already realized how to diminish the DataPortability group's brand by linking anything they do "data portability," that isn't the point of this post. The crux of the announcement yesterday was that shortly MySpace would begin allowing third-parties to embed MySpace profile information within their own services in the name of "data portability". Unfortunately, the details around this remain buzzword-laden at best.

Their press release yesterday stated:

Additionally, rather than updating information across the Web (e.g. default photo, favorite movies or music) for each site where a user spends time, now a user can update their profile in one place and dynamically share that information with the other sites they care about. MySpace will be rolling out a centralized location within the site that allows users to manage how their content and data is made available to third party sites they have chosen to engage with.

At first glance this seems like a great thing. MySpace is partnering with Yahoo!, eBay, Photobucket, and Twitter to solve a pain point on the web; the inability to keep parts of your profile in sync around the web where you'd like them to be. The announcement didn't however offer any insight into how this would work beyond that, "the MySpace Data Availability initiative uses OAUTH [sic] and Restful APIs as its core technology underpinnings." After this announcement I had the pleasure of speaking with a reporter who was on the briefing call. He explained that MySpace said that due to their terms of service the participating sites (e.g. Twitter) would not be allowed to cache or store any of the profile information. In my mind this led to the Data Availability API being structured in one of two ways: 1) on each page load Twitter makes a request to MySpace fetching the protected profile information via OAuth to then display on their site or 2) Twitter includes JavaScript which the browser then uses to fill in the corresponding profile information when it renders the page. Either case is not an example of data portability no matter how you define the term!

To make this worse one of the pieces of profile information made available is a list of a MySpace user's friends. Once again there are two reasonable ways to do this: 1) MySpace provides a user's friends as a list of hashed email addresses to Twitter or 2) MySpace provides a user's friends as a list of MySpace usernames. While the hashed email route would certainly be simpler and easier for sites like Twitter to match against their own user database, I highly doubt this will be the implementation due to concerns around undesired account linking. Rather I think MySpace will choose to provide a list of other MySpace usernames. What this means is that in order for Twitter to make use of the information they must encourage all of their users to fill in their MySpace account on Twitter so that they can map a MySpace username to a Twitter username. Obviously in the best interests of MySpace to have more of their profiles linked to from around the web thus increasing page rank, visitors, and thus ad revenue.

At the end of the day it seems that MySpace is trying to become a large centralized profile repository on the internet. One where information might be available but certainly not allowed to be actually moved outside the network's walls. A good try, but just as no one would like Microsoft own identity for the entire web with Passport I fail to see how others will let MySpace own all of the profiles.

Update: Just got off a plane from London and realized that I missed a link to Chris Saad, DataPortability's co-founder, explaining yesterday that they "hope to see the MySpace “Data Availability” initiative evolve toward becoming a compliant implementation of the DataPortability Best Practices." While MySpace did not say in their release that Data Availability is a form of data portability, it certainly seemed to be interpreted that way.

The battle for the cloud

by Tim O'Reilly | @timoreilly | comments: 14

Andy Kessler has a great op-ed in the Wall Street Journal, The War for the Web:

Microsoft was smart to walk away (for now) from its $44 billion bid for Yahoo. It's never good to overpay. But the software giant - whose stock has flatlined for eight years - was onto the right strategy in looking to the Web for growth....

With the Microsoft/Yahoo deal breakdown, everyone assumes Google walks away with the prize. Not so fast. This contest is just starting. For Microsoft or Google or anyone else to win, they need four key elements of an end-to-end strategy:

- The Cloud. The desktop computer isn't going away. But as bandwidth speeds increase, more and more computing can be done in the network of computers sitting in data centers - aka the "cloud."...

- The Edge. The cloud is nothing without devices, browsers and users to feed it....

- Speed. - Speed. Once you build the cloud, it's all about network operations....

- Platform. ...Having a fast cloud is nothing if you keep it closed. The trick is to open it up as a platform for every new business idea to run on, charging appropriate fees as necessary....

Andy's analysis is all in those ellipses. Succinct, on-point, and refreshingly insightful about the true drivers of Web 2.0. And I can't help pointing out that the Wall Street Journal has now noticed the fundamental premise of our Velocity conference: "Once you build the cloud, it's all about network operations."

If Velocity were a movie, don't you think that quote might be on the movie poster?

Velocity preview at Web2.0 Expo

by Jesse Robbins | @jesserobbins | comments: 2

At the Web2.0 Expo this month we have a small preview of some of the topics and speakers at the Velocity Web Performance & Operations conference.  (Radar readers get a 20% discount by using "vel08js" as a discount code... and yes it works with the $300 early registration discount!).

Failure Happens
Friday @ 11:00 am, Room 2009

Artur Bergman and I will kick off the day with an entertaining/informative/eye-opening review of the year’s biggest failures, disasters, and painful lessons learned.

We'll review incidents by underlying root cause with a focus on what could have been done to prevent it. We promise not to be too harsh on anybody, although we will give special attention to particularly ironic failures or those that are "entertainingly coupled" to absurd marketing claims.

(Hint: Send your boss to this talk if they don't understand why you and your whole team need to go to Velocity.)

Even Faster Web Sites
Friday @ 1:30 pm, Room 2012

Steve Souders is the co-chair of Velocity and author of the bestselling book High Performance Web Sites. At the Expo last year Steve gave an incredibly popular talk on the 14 best practices he developed while working as the Chief Performance Yahoo!.

(continue reading)

What is Web Operations?

by Jesse Robbins | @jesserobbins | comments: 0

Theo Schlossnagle wrote a brilliant summary of one of the biggest challenges we discussed at the Velocity Summit in January:

What is this Velocity Summit thing? It was a bunch of web architects from highly trafficked sites sitting around talkin' smack. It was operated in Foo style. However, one thing that made me really appreciate this meet-up was the lack of self-importance displayed by attendees. Everyone was just there to talk -- not to make people understand how much they knew. We were talking about The O'Reilly Velocity Web Performance and Operations Conference: what it should be and why.

Two things that I walked away with were (1) a realization of the lack of a career path for people who do what we do (no standard titles, no standard roles and responsibilities and certainly a lack of sex appeal) and (2) a clear lack of terminology for the technology requirements that are so common in these environments. Terminology is easy, in my opinion -- you just argue until someone wins. Of course, arguing is a hobby of mine, so I have bias. On the other hand, defining a career path that is an industry accepted path is hard.

The term Web Operations was used a lot during this event. While it isn't awful, I really don't like this term. The hard part is that the captains, superstars, or heroes in these roles are multidisciplinary experts. They have a deep understanding of networks, routing, switching, firewalls, load-balancing, high availability, disaster recovery, TCP & UDP services, NOC management, hardware specifications, several different flavors of UNIX, several web server technologies, caching technologies, several databases, storage infrastructure, cryptography, algorithms, trending and capacity planning. The issue: how can we expect to find good candidates that have fluency in all of those technologies? In the traditional enterprise, you have architects which are broad and shallow and their team of experts which are focused and deep. However, in the expectation is that your "web operations" engineer be both broad and deep: fix your gigabit switch, optimize your MySQL database and guide the overall architecture design to meet scalability requirements.

I struggle with this. Not everyone can be a superstar. More importantly, no one can really start as a superstar. If we use an apprentice model (which is common in industries without institutional support) we limit the total number of able workers in this field. So, how do we (re)define the requirements for a junior web operations person?  [read more]

One of the reasons I'm excited about Velocity is that we're increasing the pool of great operations people.  We're getting inquiries from companies interested in sending groups of 30-40 people, and I expect more as we confirm speakers and sessions.  You can secure a spot now and get a $350 early registration discount.

Amazon improves EC2 (by embracing failure)

by Jesse Robbins | @jesserobbins | comments: 5

Amazon just announced two big improvements to EC2:

• Multiple Locations
  Amazon EC2 now provides the ability to place instances in multiple locations. Amazon EC2 locations are composed of regions and Availability Zones. Regions are geographically dispersed and will be in separate geographic areas or countries. Currently, Amazon EC2 exposes only a single region. Availability Zones are distinct locations that are engineered to be insulated from failures in other Availability Zones and provide inexpensive, low latency network connectivity to other Availability Zones in the same region. Regions consist of one or more Availability Zones. By launching instances in separate Availability Zones, you can protect your applications from failure of a single location.
  
  
• Elastic IP Addresses
  Elastic IP addresses are static IP addresses designed for dynamic cloud computing. An Elastic IP address is associated with your account not a particular instance, and you control that address until you choose to explicitly release it. Unlike traditional static IP addresses, however, Elastic IP addresses allow you to mask instance or Availability Zone failures by programmatically remapping your public IP addresses to any instance in your account. Rather than waiting on a data technician to reconfigure or replace your host, or waiting for DNS to propagate to all of your customers, Amazon EC2 enables you to engineer around problems with your instance or software by quickly remapping your Elastic IP address to a replacement instance.

Datacenters and geographic regions are Single Points of Failure (SPOF) too.  Failure Happens, and it's far better (and cheaper) to build services that are resilient to failure than to try to prevent them from happening.  This is a big step in the right direction.

Update: RightScale posted an excellent overview of how this works.

How Technology Almost Lost the War, but Should Do Better

by Jim Stogdill | @jstogdill | comments: 12

It was cool that ETech ventured into unexpected territory this year with Noah Shachtman's presentation on technology’s failure in Iraq. The talk was derived from his provocatively titled Wired article "How Technology Almost Lost the War: In Iraq, the Critical Networks are Social - not Electronic". In it he takes shots at the military’s infatuation with the bright shiny objects that support the big fight while missing the day-to-day realities of counter insurgency operations; a reality that revolves around people.

Leaving aside for the moment the fact that using technology to win the big fight gives one the luxury of discussing failures in the subsequent counter insurgency phase, Shachtman argues that the military's “Network-Centric” technology is the wrong tool for the counter insurgency job. Systems like Command Post of the Future (CPOF) are cool, but in this phase of conflict, they are like bringing an iPhone to a knife fight.

I can’t disagree, but I think the reasons are as much about a monoculture focused too long on the Fulda Gap as they are about technology's bells and whistles. But that’s a conversation for another day (and venue). An interesting question might be the one he doesn’t ask, what kinds of technology might help now in the midst of a counterinsurgency and how can we get them faster?

Released just before Shachtman’s talk, MIT’s Technology Review magazine covered DARPA’s Tactical Ground Reporting System (sorry, registration required), or TIGRnet. Where CPOF was designed for commanders fighting conventional battles, TIGRnet is for the patrolling sergeant and lieutenant fighting in a counter insurgency. While CPOF supports conventional ideas of command and control, TIGRnet gives troops on the ground new tools to share information horizontally (which might make it an accidentally subversive culture virus).

TIGRnet is interesting because it was built from scratch for the counter insurgency environment. This is no small thing in a one-size-fits all Army. However, it’s disappointing because it has been so long in the making.

(continue reading)