Stop Giving the Newspapers Your Advice - They Don’t Need It

Speculation about the demise of the news business and advice about what they should do about it is everywhere. It makes for great, self-congratulatory sport but it won’t help the news industry.

Why?

Because the news industry doesn’t suffer from a shortage of ideas or possible revenue models, it suffers from a different but more acute malady: being an institution during a time of disruptive change.

While we have all been busy telling the newspaper institution what they should do differently we have missed one big point: Institutions are structured to precisely NOT do much of anything different.

The number one thing that ails newspapers? 70% of all costs lie in physical distribution and printing while readership and revenues have dramatically moved away from paper. This leads to a simple-minded but commonsense conclusion (and my superfluous piece of advice): maximize your online presence, build your online community, concentrate on journalistic talent, and jettison all costs associated with print; stop the presses.

Even if I you think I am wrong, just play along with me for a moment and, for the purpose of this exercise, assume I am right. If you can’t go that far substitute your own radical therapy (you know you have one!) in place of mine and answer the next question. Which major newspaper could have gone to its board anytime before 2009 and successfully proposed such a radical solution? The answer if you have ever worked in a large, “institutionalized” organization is zero. The scenario is so horrific, involves pains so great, outcomes so unknown and certain near-term revenue loss such that no institutional body would be capable of acting on it - much less restructuring around so medieval a remedy.

The failure of newspapers is not a failure of imagination or foresight nor is it a failure of individuals. This kind of failure is the hallmark of all institutions in the face of tectonic disruption. Institutions are a set of agreements that perpetuate a social order beyond individual intention or tenure. Changing those agreements is costly and time-consuming. So when the rate of change accelerates beyond the institution’s adaptive capacity - extinction follows.

The question is not “what should newspapers do?” but “how can a large institution effectively organize in response to disruptive change?” Taken thus, it is not only the fundamental question to ask of newspapers - but to ask of ourselves in relation to a host of big-ticket game-changers such as peak oil, environmental collapse and climate change that simultaneously require and defy our capacity for institutional response.

The stakes are much bigger than news. Let’s put our mind to that question instead of making more to-do lists. From the Radar audience I would like to ask for historical examples of institutions that have effectively responded to disruption? What are the lessons that we can draw from them?

Four short links: 21 August 2009

Moody Twitter, Future Geohistory, News Sucks, Whyless in Wonderland

1. TwitterMood -- using Twitter as a giant mood sensor for the world (see also temporal correlations, via kellan on delicious).
2. What Will Remain of Us -- The sea that brought trade to Dunwich was not entirely benevolent. The town was losing ground as early as 1086 when the Domesday Book, a survey of all holdings in England, was published; between 1066 and 1086 more than half of Dunwich’s taxable farmland had washed away. Major storms in 1287, 1328, 1347, and 1740 swallowed up more land. By 1844, only 237 people lived in Dunwich. Today, less than half as many reside there in a handful of ruins on dry land. (via blackbeltjones on Delicious)
3. The Three Key Parts of Stories You Don't Usually Get -- In reality, these longstanding facts provide the true foundation of journalism. But in practice, they play second-fiddle to the news, condensed beyond all meaning into a paragraph halfway down in a news story, tucked away in a remote corner of our news sites. Take a look at that WaPo page again. Currently, a link sits on the far right side of the page, a third of the way down, labeled “What you need to know.” Click on that link, and you’re taken here: a linkless, five-paragraph blog post from May. This basically captures our approach to providing the necessary background to follow the news.
4. Eulogy to _why -- a pseudonymous Ruby character, _why the Lucky Stiff, recently vanished from the net: all his sites and accounts were deleted. It's possible this is because someone tried to identify him, it's possible that his accounts were hacked. Either way, this is a touching tribute to him from John Resig. I for one would like to see more appreciation while the people are still around. Today, tell two good people that you enjoy what they do. You know you can.

Four short links: 18 August 2009

iPhone App Backstory, Cookie Resurrection, The Entrepreneuralism Lickmus test, and An Interesting Database

1. The Making of the NPR News iPhone App -- interesting behind-the-scenes look, with sketches and all. Station streams, however, presented a larger challenge. To begin with, NPR didn't have direct stream links for any of its stations, so we built a Web spider that identified and captured more than 300 iPhone-compatible station streams. After that first pass, we worked with our station representatives to manually test each stream. In the process they found enough new streams to double our database. All of these streams are delivered to the app from NPR's Station Finder API. (via mattb on Twitter)
2. You Deleted Your Cookies? Think Again (Wired) -- Flash keeps its own cookies, which are harder to delete. Several services even use the surreptitious data storage to reinstate traditional cookies that a user deleted, which is called ‘re-spawning’ in homage to video games where zombies come back to life even after being “killed,” the report found. So even if a user gets rid of a website’s tracking cookie, that cookie’s unique ID will be assigned back to a new cookie again using the Flash data as the “backup.” (via Simon Willison)
3. Would You Lick It? (Rowan Simpson) -- clever example of what it takes to be an entrepreneur.
4. FluidDB -- a shared "in the cloud" database built around tags: an object is a container for a set of tags which are name:value pairs, tag names have simple namespaces (e.g., "gnat/review" is the "review" tag in my namespace), all objects are world readable and writable but there are ACLs for tags, values can be any type (string, number, URL, Excel spreadsheet), and there's a simple query language. I'm curious to see what applications spring up around shared data. They're in limited alpha, controlling the # of users, so register now to play before everyone else.

How NPR is Embracing Open Source and Open APIs

Daniel Jacobson Will Talk About the NPR Open API at OSCON

News providers, like most content providers, are interested in having their content seen by as many people as possible. But unlike many news organizations, whose primary concern may be monetizing their content, National Public Radio is interested in turning it into a resource for people to use in new and novel ways as well. Daniel Jacobson is in charge making that content available to developers and end users in a wide variety of formats, and has been doing so using an Open API that NPR developed specifically for that purpose. Daniel will talk about how the project is going at OSCON, the O'Reilly Open Source Convention. Here's a preview of what he'll be talking about.

James Turner: Can you start by explaining what NPR Digital Media is and what your role with it involves?

Daniel Jacobson: Sure. NPR is a radio organization, of course, and the Digital Media Group, of which I'm a part, handles, essentially as I describe it, everything that is publishable by NPR that does not go to a radio. So that includes the website, podcasts, API, mobile sites, HD radios, anything that has some sort of visual component to it. So Digital Media as a group is responsible for producing that content, producing all of those distribution channels, managing all of those relationships.

James Turner: And what is your particular role there?

Daniel Jacobson: I manage the application development team that is responsible for all the functional aspects of all of the systems, which includes our CMS, all of the templating engines for the website, for the API, for the podcasts, all of the engines that drive that.

James Turner: Now NPR is an organization that consists of a lot of member stations kind of flying in close formation. What's your relationship with the content producers? To what extent do they have their own stuff, and to what extent do you work together?

Daniel Jacobson: Those member stations are really exactly that; they are members of NPR. They essentially buy NPR programming. They're distinct organizations from us. NPR is a content producer and distributor. They buy our programming and broadcast it out to the world. They also have their own corresponding web teams that can take NPR content and also produce their own content and create their own websites. So in the Digital Media Team, we take a lot of pride and effort in providing services that help those member stations better serve their communities and their listeners and audiences, using NPR content and using their own content. We work with them to try and satisfy their missions. And to the extent that they need NPR services or content, we work hard to try and provide those. The API is one massive step, I think, in making it much easier for them to do what they need to do without a whole lot of intervention from us, where previously they would have to pull in content in much more arduous ways. So the API, I think, is a step in the right direction to make it more of a self-service model.

James Turner: Since you've mentioned the API, that's what you're going to be talking about at OSCON. We've already talked to the New York Times and the way they're opening up their content through APIs. What are you doing with yours?

Daniel Jacobson: Well, we launched ours formally at OSCON last year. And at that time, we essentially opened up our entire archive. So anything that you can get on npr.org is available through the API, to the extent that we have the rights to distribute it. There are some rights restrictions, for example, for receiving photos or stories from sources that we have not cleared rights to redistribute. Those are getting suppressed through a rights filtering engine on our API. Everything else that you can get on npr.org, you can get through the API. That includes full text. It includes images, audio, video, everything like that. Throughout the last year, we have added more features. We included the layer of "mix your own podcast", for example, which allows people to not only get the content in audio form, but also to download it as a podcast-type item. And all of that is available through search terms or totally customized queries. So what the API really does is it enables people to take the content, make widgets, or do whatever they want with essentially everything that is on npr.org and get to audiences that we are not getting to.

(continue reading)

Walking the Censorship Tightrope with Google's Marissa Mayer

Google sometimes finds itself at a difficult crossroad of wanting to make as much information available to as many people as possible, while still trying to obey the laws of the countries they operate in. I recently had a chance to talk to Marissa Mayer, who started at Google as their first female engineer, and has now risen to the ranks of vice president in charge of some of Google's most critical product areas, such as search, maps, and Chrome. We talked about some of Google's future product directions, and also about how Google makes the decision as to when information has to be withheld from the users. Marissa will be delivering a keynote address at the O'Reilly Velocity conference next week.

James Turner: As VP of Search Products and User Experience, you're responsible for a vast swath of the Google product line, from search to maps to Google Labs. You were also the first female engineer at Google. Can you talk a little about how you came to Google and what brought you to where you are today?

Marissa Mayer: Sure. My background is when I was at Stanford, I was doing a symbolic systems degree in artificial intelligence. And I was always somewhat interested in search. I ended up getting an email [from Google] towards the very end of my job interview process. I came to Google and did the interviews. And I came here because I really wanted to put my AI background to use. For about the first year or so, I did. I did a lot of work on categorization, some work on search quality. And then interestingly, we sort of had a void around how the site looked and felt and how it worked. And we tried very hard to hire someone in UI. We thought we needed someone to do UI like one day a week, and do systems engineering the rest of the time. After a few months of failing to hire such a person, Urs Hölzle, our VP of Engineering, pulled me in and said, "Marissa, we've looked through all of the resumes and you have this background in your undergrad on cognitive psychology and philosophy and things. And would you mind dedicating one day a week to UI?" So I did.

I pulled together a volunteer team to help out one day a week while we all still worked on our various AI and systems work. And then, of course, one day became two days which became three days or four days or five days. And I was also programming at that point. I switched over from a lot of the AI work I was doing to programming the front end for Google, working on the Google web server because it was nice for me to be able to not only make decisions about the UI but also to implement them.

And then because I was implementing the changes to the front end, I would go and meet with Larry and Sergey. And they would say, "What's happening on the site this week?" And I would say, "Well, I coded a change that looked like this. And I coded a change that looked like that. And translated this page and it'll go here. And there'll be a pull-down over there for the number of results." And without even realizing it, I did project management before Google had project management, by specifying how things were and looked and communicating to the rest of the company about how these changes would manifest. And then when we got to about 200 or 300 people, Larry and Sergey discovered that most companies have this function, product management, which I didn't know about; they didn't know about prior [to this]. And we decided we should have such a department. They realized there were a few of us around the company that were doing product management even though that wasn't our title. So they started the product management group and got it started earlier. And so I became a PM.

First, I was the PM on Google.com, really broadly across the whole site, because there were only three of us. One of us did the site, which was me. Salar Kamangar did the ads. And Susan [Wojcicki] did the partners. And then as our teams grew, I became the director of consumer web properties, where I still did all of the consumer facing work of the website including branching into Gmail and tool bars and some of these other areas. And then as we progressed, eventually, we restructured so we had search, ads, and apps, because Gmail and the related space of calendar and docs became large enough that it made sense to spin it out. So then I kept the search piece and the properties we have that are more related to search.

James Turner: There's always been other companies trying to take a piece of Google's dominant position in search; how does Google plan to stay a step ahead in search, especially in light of new players like Wolfram and Microsoft's new emphasis on Bing?

Marissa Mayer: Well, we are very focused on search. We have a large team here that's really focused on it and working hard on it. And we're constantly trying to forge in new directions. So we were really excited about the launch of our search options page because we think that allows us to try a lot of new ways to slice and dice and filter results. We were also really excited about Google Squared, which attempts to do automated text extraction from the web and present comparison tables for different entities in response to queries. They're both new ways to search. How do you generate a timeline from a web search? How do you generate a comparison table? And some of our competitors are also looking at those same issues. So I think on the whole, right now, our search is a very healthy ecosystem. There's a lot of interest. There's a lot of activity, and there's a lot of new ground being forged.

James Turner: Google users want the most useful results, but content providers want to get their pages seen, sometimes it seems at any cost. How will Google continue to provide the most useful results in the world of increasingly sophisticated SEO gaming?

Marissa Mayer: Well, we generally -- we really want to be fair in these issues as well as be good to users. We do think that spam is very detrimental to the user experience. We do have an incentive to find spam and remove it from our results. But we want to do something in a way that's very scalable. The web is scaling at an incredible rate. And we don't think it's really viable to try and fight spam in a manual way. So we're always looking for new algorithmic ways to understand new spam techniques, to be able to detect them in an automated way and remove them from our results. And the nice side benefit that scalability has is it's also reasonably objective and fair.

(continue reading)

Four short links: 12 May 2009

Storage Superfluity, Data-Driven Design, Twit-Mapping, and DIY Biohacking

1. Lacie 10TB Storage -- for what used to be the price of a good computer, you can now buy 10TB of storage. Storage on sale goes for less than $100 a terabyte. This obviously promotes collecting, hoarding, packratting, and the search technology necessary to find what you've stashed away. Analogies to be drawn between McMansions full of Chinese-made crap and terabyte drive full of downloaded crap. Do we need to keep it? Are there psychological consequences to clutter? (via gizmodo)
2. In Defense of Data-Driven Design -- a thoughtful response to the "Google hates design!" hashmob formed around designer Douglas Bowman's departure from Google. When you’ve got the enormous traffic necessary to work out if miniscule changes have some minor, statistically significant effect, then sure, if you can do it quickly, why wouldn’t you? But that’s optimization that should happen at the very end of the design cycle. The cart goes after the horse. Put it the other way ‘round and you have a broken setup. It doesn’t mean horses suck. It doesn’t mean carts suck. Carts are not the enemy of horses. Optimization is not the enemy of design. Get them in the right order and you have something really useful. Get them the wrong way around and you have something broken.
3. Just Landed: Processing + Twitter + Metacarta + Hidden Data -- Jer searched Twitter for "just landed in", used Metacarta to extract the locations mentioned, and then used Processing to build visualizations.
4. Do It Yourself Genetic Sleuthing -- MIT is starting a hotbed of DIY biologists. The 23-year-old MIT graduate uses tools that fit neatly next to her shoe rack. There is a vintage thermal cycler she uses to alternately heat and cool snippets of DNA, a high-voltage power supply scored on eBay, and chemicals stored in the freezer in a box that had once held vegan "bacon" strips. Aull is on a quirky journey of self-discovery for the genetics age, seeking the footprint of a disease that can be fatal but is easily treated if identified. But her quest also raises a broader question: If hobbyists working on computers in their garages can create companies such as Apple, could genetics follow suit? It's unclear what those DIY-started "genetics" companies would look like--the potential is there, but it's yet to met the right problem. (via Andy Oram)

Just Landed - 36 Hours from blprnt on Vimeo.

Tim O'Reilly - Why Twitter Matters for News

Twitter has been used for a lot of different purposes, and one has been to report breaking news. But there's been some criticism of how Twitter deals with news, such as the Swine Flu outbreak. With that in mind, O'Reilly Week in Review talked to Tim O'Reilly himself, co-author of the new Twitter Book, about the role of Twitter in informing the public.

James Turner: Thanks for taking the time, Tim. You guys have just come out with Twitter Book. Right now, Twitter's getting a lot of volume because of the whole Swine Flu to-do. Do you think that it's been a net positive?

Tim O'Reilly: You made the comment that Twitter's getting a lot of play, but Twitter has so many uses. It's a little bit like saying, "Well, the internet's getting a lot of play," or, "The telephone is getting a lot of play," or, "Video is getting a lot of play." People are using this medium, and what's so interesting about Twitter is that it reflects all the many, many use cases, including spreading misinformation as well as being the first alert of new information. All of these are good functions. And I think some people reacted a little bit to the fact that Twitter seemed to increase the fear and hype level a little bit.

But I don't know that I would blame that on Twitter. Twitter just meant that the news spread faster. So anyway, I guess when I think about Twitter, the thing I think about the most is how powerful it is at reflecting actual human communications. I know people who are using it in hundreds of different ways. There are people who are using it for that wonderful original use case that Leisa defined so beautifully as ambient intimacy. I know people like me who are using it as a way of sharing my thought processes--what I'm learning, what I'm reading, what I'm caring about. I'm using it as a publishing medium really. And I've described my own work with Twitter as being the most minimal newspaper. And I mean that very seriously. When I gave a talk at the New York Times recently, I spoke about Twitter. I talked about the process of what a publisher does. A publisher pays attention to a community, whether it's a community of authors or a community of news makers. And they then curate it. They decide what's important. And then they share that with their community of readers. And then they presumably have feedback groups. And so I look at what I do with Twitter and I say, "Wow, I'm sharing the news that I'm finding interesting." I also have the ability to, in some sense, increase the status of members of my community.

I got this originally when I tweeted about the fact that I really liked the Venture Hacks site. And Babak Nivi, who started the site, had a post actually on FriendFeed, in which he posted a graph showing what happened to his Twitter follower account when I did that. And it doubled immediately. And I think that's a wonderful thing. I was able to find somebody that a lot of people who were following me didn't know about. I was able to call him out and say, "Hey, here's this great post on Venture Hacks. I find this a great site." And I did just what I do when I publish a book or just what I do when I pick somebody to be a speaker at a conference. I'm basically saying, "This person is worth paying attention to."

And, for me, that's just a very, very powerful use case that has made me so fond of Twitter. But then, of course, I go out there and I see all of these other ways that people are using it. And it's just exciting.

JT: One of the things I've noticed is that while what used to be traditionally considered blogs have stopped being web blogs, and they're more personal editorials, or personal rants. Twitter has really turned into, for a lot of people I think, the new blogging medium because with the short length and with using the tiny URLs, a lot of what I see is just a line about something interesting someone saw and a link.

TO: Yeah, it's funny. If you look back at the very early web blogs, you know, Dave Winer's UserLand blogs, it really was in some sense a tweet stream. Dave, literally in the course of the day, was doing tweet-length paragraphs saying, "This was interesting. This was interesting. This was interesting." And then occasionally, he would write a longer essay. And I think what we're seeing is the bifurcation of that function of blogs which was, "Hey, I saw this. I want to share it, "and the, "Hey, I want to have a longer conversation or opinion about this," into different tools.

I think they may need to be reunited in some future world, because I know I certainly find certain pieces that I wish that had a few more than 140 characters to comment. I find myself sometimes being pulled into conversations, in which case, I go, "Boy, I really should be doing this on FriendFeed." Certainly, there's the Tumblr-length blog posts. And, by the way, I think that's actually something that many people have not noticed. Many people are aware of Tumblr, but what they're not aware of is how the Tumblr-style blog post has actually become part of many other sites.

So, for example, if you look at Huffington Post, many of their little squibs, their little widget-sized article leads that they have on the homepage, are to Huffington Post articles, but many of them are to articles on other sites. So effectively, they've embedded a Tumblr-like blog, slightly more expanded than Twitter, within the context of what is a remaking of the blog into a newspaper. And, of course, the Wall Street Journal has always had a column of Tumblr -- almost Twitter sized -- entries which were usually pointers to more extensive articles elsewhere.

JT: Just to come back to news to finish things off, Garry Trudeau has fairly noticeably been parodying the process of news generation, or news reporting, by professional journalists on Twitter. What do you think the role might be for Twitter in news gathering and news reporting in general? I know that, for example, the Swine Flu "pandemic" that's going on, we had a case in New Hampshire and essentially all I was doing was repeating, "Wow, we just had a case in New Hampshire." So I don't know that I was adding any value to the conversation beyond kind of this echo chamber that it's being referred to. Is there a place for news?

TO: Absolutely. First off, I wouldn't say that you're adding no value because one of the values that occurs when somebody says, "Oh my gosh; we had a case in New Hampshire," is that they're spreading the word. The fact is the re-tweeting or commenting like that is a the equivalent of the news boy on the street corner, "Breaking news." You know? We don't have that anymore. We're just kind of passing on the word.

Now you might say, well, that's a less important function than breaking original news. Absolutely. And, yes, it sometimes can become drivel. But I also think we'll get better tools for separating the wheat from the chaff. And people will learn, "Oh, I'm not adding value anymore." And I certainly think that re-tweeting has gotten somewhat out of hand. And that's one of the reasons why in the Twitter book, I wrote a section about why "via" is better than RT. It was really around this idea that it's very easy and lazy to just say RT @ some user and pass on whatever they said. And it's much better to give your take on it and then give them credit with a via. And since I've really started trying to do that more, I find myself making sure that I pass on something of my own connection to the story or, in some cases, not passing it on at all.

JT: Well, Tim, thank you for taking the time to talk to us. Tim O'Reilly is, of course, the Founder and CEO of O'Reilly Media and also the Co-Author of The Twitter Book.

TO: Thanks very much, James.