The Library of the Commons: Rise of the Infodex

by Mark Sigal | @netgarden | comments: 8

Somewhere between the realm of Personal and Shared media lies the realm of the Universal.

The realm of the universal is the Library of the Commons, a global repository of user-generated and crowd-sourced media and information.

Services that logically nest in the Library include: Amazon, Yelp, YouTube, Craigslist, Wikipedia, Flickr, Twitter tweets, Bit.ly items, Scribd docs, Expedia, Google News, Google Maps, TripAdvisor, iTunes, the App Store and any other services and/or information sources that 'just work.'

In other words, these are services that have defined the 'IT' to the point that we can now pretty much take their utility and availability for granted (typically via API access and/or embed codes with some form of customization wizard).

The Genesis of a Library

So how did we get to this place in the story? What gave birth to the Library of the Commons?

No one formally deigned it so, but from the countless me-too services borne of the dotcom and Web 2.0 land rushes, the above-referred services are the ones that cultivated the biggest audiences, grew the richest ecosystems and inspired the deepest engagement levels.

In Darwinian terms, these are the survivors, whose structures and workflows have been defined and refined by time/experience.

As such, they are generally well thought out, holistic and integrated, but more to the point, have large, engaged user bases.

Thus, the Commons presents a riddle. Almost as if inspired by Herman Hesse's 'The Glass Bead Game', the riddle is this.

If all of these services yield a smorgasbord of best practices, why not systematically emulate them so as to...FEDERATE them?

Put another way, what if a time came when people ceased trying to perennially re-create the wheel, and instead, started to 'decompose' these services; to empty their function sets from whatever nesting they were contained within; and to re-apply them into new contexts supported by a now federated data flow proxied within the Cloud.

Couldn't the composite feature set be exposed switchboard-style to enable any number of custom services and client apps?

To put some meat on the conceptual skeleton, consider the following exercise that I recently did:

A decomposition of Craigslist and TripAdvisor yields deep profiles that are accessorized and interconnected via context traversal flows, such as categorization routines, places, events, airfares, posts, pages, ratings, discussion threads, offers, jobs, businesses, products and personal listings.

Craigslist offers up 36 different sub-types of items For Sale; Services represent another 19 sub-types; Jobs 41 more; Discussions, another 72. And so it goes (including Housing, Personals and Community) across 175+ geo-locales.

TripAdvisor is an instance of this model that overlays a set of time-tested workflows specific to the relatively complex task of planning a vacation.

These workflows make it easy to match a travel plan to specific tastes, requirements and budget - regardless of the information traversal path you pursued to being ready to get pricing on desired travel dates.

Could these same workflows be re-purposed for researching and then purchasing other similarly complex products or services?

I will come back to that thought, in a moment.

The Rise of the Infodex

What is de-composed, can be re-assembled, and thus begins the Infodex.

The Infodex is a kind of next-generation Rolodex, with aspirations to grow into a real-time marketplace.

What exactly is the Infodex? It is comprised of three parts.

Part one is a listing tool for linking to content, creating a metadata wrapper around media items and encapsulating the above-referenced services (i.e., Yelp, YouTube, WIkipedia) into listing containers that define and expose the methods that one can interface to the media item (framework integrity stuff).

Part two is an indexing engine so that, once simple rules are defined, your media libraries and the information in the listings themselves becomes 'self-organizing.'

Named picture types (globes, animals, historic or famous images), for example, could be a federation of multiple picture services (Flickr, Photobucket, Getty Images) and 'discovered' pictures from past queries.

Looked at from this perspective, the goal, in part, is to establish a cloud-based, crowd-sourced Dewey Decimal System built around the outcome of facilitating better searching, compositing, cross-indexing, sharing, archiving, and analytics functions for specific media and information 'types.'

Part three of the Infodex is a unified runtime player that is congruent with the information flows of the mobile broadband age; namely, iPhone, Twitter, Facebook and Web (Javascript/Flash embeds/Adobe AIR) based viewing/playback environments.

One simple example of a basic type of function that might be propagated across all of these environments is the Three Item Topical List (e.g., Top Three Favorites or Three Most Related Items). Define once, propagate everywhere.

A core assumption of the model is that both the media player and the service integration layers are open-sourced. This ensures that the user experience is uniformly good across all of these services, and pushes proprietary-ness higher up the stack, thus raising the floor for all comers.

A final thought. Google became Google by indexing the web. Couldn't the next generation extend this approach by being federated, crowd-sourced and context-specific (i.e., media, information and service aware)?

Are their obvious best practices for The Commons? Obvious gotchas? What about the Infodex?

Related Posts:

1. Pattern Recognition: Makers, Marketplaces and the Library of the Commons


2. Envisioning the Social Map-lication


3. The Mobile Broadband Era: It's About Messages, Mobility and The Cloud

How NPR is Embracing Open Source and Open APIs

Daniel Jacobson Will Talk About the NPR Open API at OSCON

by James Turner | comments: 7

News providers, like most content providers, are interested in having their content seen by as many people as possible. But unlike many news organizations, whose primary concern may be monetizing their content, National Public Radio is interested in turning it into a resource for people to use in new and novel ways as well. Daniel Jacobson is in charge making that content available to developers and end users in a wide variety of formats, and has been doing so using an Open API that NPR developed specifically for that purpose. Daniel will talk about how the project is going at OSCON, the O'Reilly Open Source Convention. Here's a preview of what he'll be talking about.

James Turner: Can you start by explaining what NPR Digital Media is and what your role with it involves?

Daniel Jacobson: Sure. NPR is a radio organization, of course, and the Digital Media Group, of which I'm a part, handles, essentially as I describe it, everything that is publishable by NPR that does not go to a radio. So that includes the website, podcasts, API, mobile sites, HD radios, anything that has some sort of visual component to it. So Digital Media as a group is responsible for producing that content, producing all of those distribution channels, managing all of those relationships.

James Turner: And what is your particular role there?

Daniel Jacobson: I manage the application development team that is responsible for all the functional aspects of all of the systems, which includes our CMS, all of the templating engines for the website, for the API, for the podcasts, all of the engines that drive that.

James Turner: Now NPR is an organization that consists of a lot of member stations kind of flying in close formation. What's your relationship with the content producers? To what extent do they have their own stuff, and to what extent do you work together?

Daniel Jacobson: Those member stations are really exactly that; they are members of NPR. They essentially buy NPR programming. They're distinct organizations from us. NPR is a content producer and distributor. They buy our programming and broadcast it out to the world. They also have their own corresponding web teams that can take NPR content and also produce their own content and create their own websites. So in the Digital Media Team, we take a lot of pride and effort in providing services that help those member stations better serve their communities and their listeners and audiences, using NPR content and using their own content. We work with them to try and satisfy their missions. And to the extent that they need NPR services or content, we work hard to try and provide those. The API is one massive step, I think, in making it much easier for them to do what they need to do without a whole lot of intervention from us, where previously they would have to pull in content in much more arduous ways. So the API, I think, is a step in the right direction to make it more of a self-service model.

James Turner: Since you've mentioned the API, that's what you're going to be talking about at OSCON. We've already talked to the New York Times and the way they're opening up their content through APIs. What are you doing with yours?

Daniel Jacobson: Well, we launched ours formally at OSCON last year. And at that time, we essentially opened up our entire archive. So anything that you can get on npr.org is available through the API, to the extent that we have the rights to distribute it. There are some rights restrictions, for example, for receiving photos or stories from sources that we have not cleared rights to redistribute. Those are getting suppressed through a rights filtering engine on our API. Everything else that you can get on npr.org, you can get through the API. That includes full text. It includes images, audio, video, everything like that. Throughout the last year, we have added more features. We included the layer of "mix your own podcast", for example, which allows people to not only get the content in audio form, but also to download it as a podcast-type item. And all of that is available through search terms or totally customized queries. So what the API really does is it enables people to take the content, make widgets, or do whatever they want with essentially everything that is on npr.org and get to audiences that we are not getting to.

(continue reading)