Should Personal Genomics Be Regulated?

I read recently about the cease and desist letters sent to 23andme and other personal genomics companies selling tests directly to consumers. 23andme has responded, saying that they agree with the ultimate need for regulation, but that harnessing the consumer internet for personal genomics is a really valuable scientific tool.

I have to say I find myself doubtful about the urgency of this regulatory move. It smacks more of the hand of the AMA, an entrenched industry trying to make sure that the new tools of genetic testing remain under the thumb of doctors, than of true consumer protection. You have only to walk into Whole Foods to encounter a multi-billion dollar industry of supplements making all kinds of dubious health claims, which is completely unregulated. Why pick on personalized medicine, which has way more substance, and at least so far, way more care in the types of claims it makes?

I'm a happy 23andme customer, and it's hard for me to see how the information they provide consists in any way of medical advice that should be subject to regulation. They are very clear to mark the scientific status of any genetic studies they report on, and never begin to presume to diagnostics. In many ways, the service is a kind of "RSS for the genome," feeding you the results of the latest scientific studies that might be relevant to you. You can find these same studies simply by googling for them. 23andme simply says "this might be relevant" using data that you've provided about yourself via your genetic sample.

The benefits of what companies like 23andme are doing is enormous. Once you understand even a tiny bit about how genes affect our response to drugs, you realize just how flawed many clinical studies are. If people have different genetically programmed response to drugs, what is the right dosage? Is the drug effective? (What if it's 100% effective for 3% of the population, due to their genetics, while it's completely ineffective for the rest. Would we ever know?)

Ideally, every clinical study going forward would have a genetic screening component. And new studies should be done on old treatments to correlate their effectiveness with genetic data. But that's hugely expensive.

23andme (and presumably other similar services) have come up with a very clever hack that will vastly increase the available store of genetic information that can be used to cross check various medical studies. They recruit a large population of those who are merely curious. But once that genotype data is available, they can begin to do surveys of their user population to gather corresponding phenotype data. (That is, matching genetic data with observable characteristics.)

This will be a boon to science. As Linda Avey, one of the founders, wrote on the 23andme blog:

Our first mission is to enable personal access to genetic information and provide a look, through the prism of an individual’s genome, at the flood of research discoveries being published. Our longer-term goal is to utilize a web-based platform that gives individuals the ability to share details related to their personal traits-including diseases they have and how they respond to therapies-uniformly layered on their genetic profiles to start building the evidence needed to drive targeted diagnoses and treatments.

I hope that regulators will seize the opportunity provided by the consumer internet to open up the frontier of personalized medicine. This is our data, the most personal data we have. I don't see why it should be forced into the straight-jacket of the 20th century medical industry. We tried that with music, and what did that get us?

I'd say to let this area run for a while, and start thinking about regulation if and when it starts to go awry. Right now, the companies I'm aware of are being very careful not to promise anything that needs to be regulated.

Daylife's API for the News

Several years ago, my friend Upendra Shardanand tried to get me to join him in starting a company that would remake the way news is created and understood -- overturning the worst, ambulance-chasing tendencies of modern journalism, and building tools to help people track and understand the topics and people that shape their lives. I begged off in order to pursue my own startup, but it was the hardest "no" at which to arrive, since I respect Upendra so much and so admire what he was looking to build. Though we've chosen to pursue different topics, we have in common a desire to make the world better through entrepreneurial projects, and Upendra's effort definitely would have won me over had I not already started down my own road.

Happily, Upendra has built and launched a company, Daylife, around his ideas about the news industry, and I'm proud to be a Daylife advisor. There's an excellent article about Daylife in the current issue of BusinessWeek, talking about some of their early successes.

This month, Daylife is sponsoring a developer contest around its API, which provides a rich programming interface around news topics, people, and places. I'm one of the judges for the contest, along with Brian Behlendorf, Clay Shirky, Jeff Jarvis, and others. It makes me very happy to see some of the API samples, many of which remind me of ideas I heard kicked around back when Google News first launched. (Coincidentally, there's an interesting article about the stagnation of Google News in today's New York Times.) Daylife has also put together a list of Lazyweb ideas for the contest, my favorite of which is this design for a tracker of news about evil dictators.

I'm looking forward to seeing what people come up with for the contest, and I'd encourage you to check it out and submit a project. I started playing around tonight and quickly came up with three ideas for Daylife API projects that would help my startup. It won't take too many people doing the same before Upendra's idea of changing the way news works starts to take shape in the world.

WordSpy as Collective Intelligence

I've long been a fan of WordSpy, Paul McFedries' site that features definitions and first use of new words and phrases. It's a great trendspotting tool. The words we use give surprising insight into popular consciousness. Many of them, like junk sleep, silent disco, free-range kid, or Blackberry prayer illustrate new social trends, while others like phantom load or quake lake are terms of art that have entered popular consciousness, and still others, like naked street are like deli.cio.us links to things you've never heard of, but could end up being important or useful.

Paul used to offer the service by email, but discontinued it years ago, and as a result my use of the site declined. I was delighted to see that he recently re-instituted email delivery of the new daily WordSpy entry. And of course, there's RSS, and I just talked Paul into adding a WordSpy twitter feed as well.

I twittered about this, and Erin McKean pointed to another great word site, doubletongued.org.

Inside Innovation at Xerox PARC

We were part of a group of journalists and bloggers invited to hear presentations from 10 different research groups within various parts of Xerox, PARC, and Fuji-Xerox. The format was similar to a science fair or a poster session in an academic conference with small groups moving around to hear presentations from the different projects. While other research labs use a large auditorium and parade different researchers in, I thought the smaller, science fair format made for better interactions between the visitors and the researchers.

We saw early prototypes created by the researchers themselves, so the user interfaces were far from polished. Here are some of the highlights from our visit:

Seamless Document Viewer

A J2ME application designed to help solve the problem of viewing documents on small screens (cell phones and other mobile devices), this app automatically segments a document into blocks and displays the keyphrase for each block. The keyphrases are intended to help users navigate to sections of interest quickly. The cell phone demo we saw used a fairly intuitive touchscreen interface that included an interesting way to pan and zoom in and out of sections of a document. Because documents viewed through the application need to be processed and analyzed in advance, it is better suited for viewing PDF's and static documents, not frequently updated web pages.

Hybrid Categorization
Categorizing documents automatically is an old topic in information science. Most tools rely only on the text portion of documents and use a combination of Natural Language Processing and Machine Learning. I was looking forward to this presentation because we use text-only automatic classifiers to help organize some of our data sources.

Hybrid categorization uses both the text and images contained in documents. It isn't clear how scalable their hybrid categorizer is, the results we saw were based on small numbers of documents. Precision measures the accuracy of a categorizer and judging from the results of an academic competition, Xerox' hybrid (text +images) approach may hold some promise.

Erasable Paper
"Reusable paper" refers to paper coated with special materials and a custom printer that shoots UV light onto it. The resulting printed document is designed to fade within 24 hours and the paper can be reused and fed into the printer multiple (10+) times. The printer can even erase the printing on the specially-coated papers, and print an entirely new document on the same sheets of paper. We raised the possibility that a sheet of paper that has nominally erased itself can be reverse engineered to reveal sensitive content: think security agencies or dumpster-diving identity thieves. Surprisingly, the researchers had not seriously investigated the possibility of "recovering erased documents".

The cost of the specially-coated paper is projected to be only 2-3 time the cost of normal paper, while the accompanying printer will cost about the same as a laser printer. Since paper can be reused multiple (10+) times, the obvious environmental benefits also lead to savings. Further savings come from the design of the printer itself: since the printing is done with light (UV LED bar), the printer does not use ink or toner.

Intelligent Redaction
Redaction is the process of removing sensitive information from documents. Popular examples include government/intelligence documents released to the public and medical records. Text redaction is normally a tedious manual process that requires staff possessing significant domain expertise. As an example, privacy rules governing medical records in the U.S. requires redaction of terms associated with HIV/AIDS, mental health and drug/alcohol problems. In the demo we saw, the software tool examined a corpus of documents, automatically came up with terms/phrases associated with the listed illnesses, and redacted them from every document in the corpus.

Other Notables

SpongeBob SquarePants Supports O'Reilly Research Finding

In O'Reilly Radar's recent reseach report, Virtual Worlds: A Business Guide, we contend that virtual worlds will go mainstream. The most powerful data point supporting our argument is that the most active and popular virtual worlds nowadays tend to be those populated by children. The next generation is growing up playing virtual worlds.

And now one of the biggest purveyors of virtual worlds for children, Nickelodeon (which owns Neopets), is going in deeper. It's adding more virtual world features to Neopets and developing a virtual world around its SpongeBob SquarePants franchise. Companies not paying attention to virtual worlds are not paying attention to where the market is going.

Wattzon.org - How much energy we consume and what to do about it

Saul Griffith has published a version of his talk at ETech as a website, wattzon.org.

Saul's key points: Solving global warming is an engineering problem. We know the connection between greenhouse gases and global warming, and can determine just how much carbon we're allowed to put into the atmosphere to give us the temperature we can live with. The answer isn't pretty. He looks at it from both a personal point of view (how do I need to change my lifestyle to use only my fair share of the global carbon allowance) and from a global policy point of view (what are the available sources of clean energy, how big are they, and what is the scale of the industrialization effort required to harness them?)

From Wattzon:

The average American uses 11400 Watts of power continuously. This is the equivalent of burning 114 x100 Watt light bulbs, all the time. The average person globally uses 2255 Watts of power, or a little less than 23 x100 Watt light bulbs.

What are the consequences of us all using this much power?

What is the implied challenge of global warming in terms of how we produce power?

What are the things we do as individuals in terms of using power that we might change?

Wattzon.org hosts a document that gives us a framework for thinking about these challenges, and how we might change our behaviours as individuals as well as our collective behaviour as societies and global citizens, if we are to meet the great challenge of the 21st century - how to live in a world where we increasingly understand the resources to be finite, and the consequences of our actions complex & inter-twined.

What temperature do we set climate change at? What CO2 concentration does this imply we need to aim at? How much power can we get from fossil fuels while still meeting this goal? How much power do we need to install and produce from non-carbon technologies? What does this mean for countries, corporations, and individuals?

Click a lightbulb to continue.

(See also Ethan Zuckerman's great summary of Saul's talk and the video interview done by TechwebTV.)

Laptop penetration in Brazil, rising developer count in China

Interesting email from Paul Kedrosky:

You'll find this interesting: The only place I have been where I see as many open laptops in the audience as at O'Reilly conferences is here in Brazil. Really fascinating.

In a related note, I talked the other day with Stephanie Martin, the head of IBM DeveloperWorks. She noted that the number of Chinese developers active on the site is approaching the number from the US, and that she expects it to pass the US numbers before long. This is as much a reflection of the decline in the developer population in the US as it is the rise of the developer population in China. (More details in this article.)

I wonder, though, how many of the developers in the O'Reilly emerging tech ecosystem are being counted. IBM's focus on enterprise software development may cause them to miss all the ferment coming from cross-over, self-taught developers who started in some other field (say design, or engineering, or finance) and just had an itch to scratch. I certainly see a lot of vibrancy in the US tech community, and I just don't buy the doom and gloom that people seem to be spreading about the declining number of CS degrees. Most of the tech entrepreneurs I know didn't start in CS anyway. They are self-taught.

New O'Reilly Radar Report: a Business Guide to Virtual Worlds

Virtual worlds, particularly Second Life, have generated much excitement -- and much skepticism. In Virtual Worlds: A Business Guide, the newest O'Reilly Radar report, Ben Lorica, Roger Magoulas, and the O'Reilly Radar team get past the hype (and the anti-hype), detail what is happening in Second Life and other virtual worlds, and lay out what businesses need to do to succeed in these worlds, now and in the future. It examines business opportunities, evaluates what has been successful and what hasn't, and what trends are starting to emerge. Those interested in the present will finds an in-depth study of Second Life; those interested in the future will be most interested in what we've learned about children's engagement in virtual worlds. Indeed, the most active users of virtual worlds aren't adults yet. Regardless of how Second Life ends up as a business, there are plenty of reasons to be bullish about virtual worlds as a business category. This report shows why and what to do about it.

Order the report.

See all the O'Reilly Radar reports.

@ETech: Tuesday Morning Keynotes

Saul Griffith started the day with a sober, but ultimately hopeful, talk about energy literacy. The subtitle of the talk was "know what you can do, do what you can," and the core of his talk (we'll point to the slides when we get 'em) was the steps we need to take, individually and collectively, to be able to have a rational conversation about energy.

1. We need to understand the link between CO₂ and climate
2. Based on that understanding, make a temperature choice. The planet is warming. The question is how much. Where do you want to stabilize the earth temperature? He set the dial at different levels and sketched out the different consequences we have to accept at each level.
3. Based on the temperature we choose, we have to decide just how much carbon that allows us to release. He pointed out that temperature stabilization can take 100-300 years.
4. Based on the amount of carbon we choose to release, we have to decide how much fossil energy we can use.
5. Based on the usable fossle energy we can use, we have to decide what clean energy sources we need to supplement the fossil fuel.
6. Based on what clean energy sources we have access to, we have to determine a new energy mix -- and how we'll engineer.
7. Then -- and Griffith acknowledged that this "might be the hardest part" -- we have to turn off our use of existing carbon fuels."

He then showed how he is trying to change his lifestyle based on his decisions during those seven steps. He pointed out what we know already -- that even reasonable, moderate people in the developed world have a big carbon footprint -- and something I, for one, didn't know -- that public carbon-footprint calculators give low estimates.

As he listed the changes he's trying to make, Griffith noted that the things he wants to do to lower his carbon footprint are things he wants to do already (eat less, travel less, etc.). If you're optimistic about your ability to change, you can be optimistic about how we gets to his new life -- and how we can.

Then Megaphone founders Jury Hahn and Dan Albritton delivered a fascinating phone-game demo. Their combinations of tiny mobile devices with simple games those devices play on a big, communal screen were both technically interesting and fun to play. Albritton promised us something "really, really weird," and he delivered. You really haven't lived until you've sat next to someone next to you in a dimmed conference room standing up, yelling "ribbit" like a frog, and looking to see if his perfect match responds.

Eric Rodenbeck, CEO of Stamen Design walked through some of his firm's more high-profile visualization projects. Trulia Hindsight maps homes over time, but also reveals more (like where pollution is); Oakland Crimespotting reveals both patterns of crime -- and patterns of crime enforcement; a project for mySociety shows how multiple variables -- home prices, commute time -- can be elegantly combined in a single interactive visual.

Rodenbeck spent some time showing how information visualization, while it may be hot right now, nothing new. He displayed some century-old pre-computer infovis examples that were"both beautifully arranged and scientifically valuable." Then, as now, the best information visualizations are those where cool and useful overlap, where story and headline overlap. This dovetailed nicely with an obvservation Griffith made in his earlier talk, when he cited an 1896 article by one Svante Arrhenius that linked carbon with warming. We keep discovering the same things!

Sun Microsystems chief gaming officer Chris Melissinos, was there to talk about the company's J2SE-based, open source (via GPL v2) game-development platform Project Darkstar, but he had plenty of provocative one-liners and observations:

* "If we calculated the carbon footprint for World of Warcraft, we'd all vomit."
* "This is the first generation of gamers raising gamers."
* "Women over 35 comprise the largest segment of online game players."

Finally, Elizabeth Churchill, of Yahoo Research, reported on studies she made of public multi-touch displays, emphasizing how the real and the virtual interact. In particular, I was taken by her descriptions of how people learn to be part of communities of practice by watching -- or as the online world calls it, "lurking," an activity that is often dismissed. But Churchill maintained, "Lurking is an important practice. What we reveal in the virtual serve as the icebreaker for real life."

I'm only giving a quick taste of a strong, diverse session, but I want to get back into the breakout sessions, which have started already...

@TED: Best of Day 4 and a Wrap-Up

The last day at TED is a combination of exhaustion, anxiety, and wistfulness: exhaustion because we've been neglecting our sleep, anxiety because we remember how much work awaits us after the event is over, and wistfulness because we realize we can't live like this all the time.

Perhaps because the programmers knew that we'd be pulled in multiple directions, the last half-day of TED was stuffed with talks that demanded our full attention. Here are some of the best moments from today:

* Johnny Lee Chung showed his jaw-dropping Wii Remote hacks, which create, for less than $50, an interactive whiteboard. (Jesse Robbins covered it previously on Radar.) When comparing it to a real, multi-thousand-dollar interactive whiteboards, Chung said of his project, "you get 80% of the way there for 1% of the cost."

* Paul Collier, author of The Bottom Billion, looked to a historical analogy to consider how to get countries out of poverty. The last time the rich world did something serious about developing another region, he stated, was in the late 1940s, when the U.S. needed to develop a devastated Europe. We can learn from what the U.S. did then -- providing aid, reversing a protectionist trade policy, moving its security policy from isolation to engagement, and abandoning some notions of national sovereignty to create the United Nations -- as a model for what we need to do now.

* Al Gore, who debuted his "Inconvenient Truth" presentation at TED two years ago, delivered a run-though of a new talk he's developing. This time out, he seems less focused on alerting us to climate change -- he's done that already -- and more on what to do about it, not only at a personal level but at a national level. As he put it, "changing the laws is more important than changing light bulbs." During his talk, I kept creating an alternate history in my head. Regardless of your political point of view, there's no doubt that the world -- and the U.S.'s place in the world -- would be far different now if we were in the last months not of the second Bush administration but the second Gore administration. Perhaps Gore did more good outside of the White House than he could have inside.

There were two other talks from yesterday that I wanted a bit more distance from before I tried to write about them.

The first was from MIT's Tod Machover, who spoke in the abstract about how music has power in people's lives -- and then proved it. His talk peaked when he brought out , Dan Ellsey, a longtime cerebral palsey patient from Tewksbury Hospital, outside Boston, to show how even someone with a profound disability can create music. Then Machover moved from talking about composition to talking about performance, and Ellsey, thanks to a system developed at the MIT Media Lab, was able to "play" one of his songs. Strapped to chair, imprisoned by his illness, Ellsey and his work were completely, miraculously alive. My words can't express the drama of the moment. Perhaps when this talk makes it to the TED website, you can see for yourselves.

It's also hard to describe what conductor Benjamin Zander achieved last night. The Boston Philharmonic conductor also spoke of music, connected his notions of music to inspirational ideas about leadership (and, it seemed at the time, everything else in the universe), and led a devastating experiment that had the entire audience singing Beethoven's "Ode to Joy" in phonetic German, with him urging us to take our amateurish singing higher and higher. I won't even try to describe what he accomplished (again, the filmed talk will do that for itself), but I do want to share with you -- despite this being a family blog -- the term he introduced that everyone was talking about last night: BTFI.

Zander told of a cellist auditioning to be in an orchestra. Zander, helping the musician, told him he was holding back: he had to do more. The cellist didn't get the job because, he reported to Zander, he was still holding back. But then the cellist said "fuck it," made himself audition for a better job at another orchestra, and got it. Zander's lesson: to excel, you have to get BTFI, "beyond the 'fuck it'." It's an idea he and his wife Rosamund Stone Zander explore in their brief and wonderful The Art of Possibility. (You can read the Google Book Search excerpt from his book, on the origin of BTFI, here.)

Most of the speakers at TED were people who had seen problems -- in themselves, in an industry, in society -- and had decide to get BTFI. That encapsulated TED and it's an appropriate note to go out on. Until next year...

@TED: Best of Day 3

The joke among TEDsters is that, around the third day, it becomes an endurance sport. It's one thing to be in a room listening to spectacular insights for a few hours. It's another to be doing so for half a week. Nonetheless, part of the experience you get from being at events like TED is that feeling of being overwhelmed: someone just said what feels like the smartest thing you ever heard -- and then the next speaker says what feels like the smartest thing you ever heard -- and then ... well, you get the idea. It's intellectually exhausting, but it's also thrilling.

Here are some of the best moments of Day 3:

* The "What's Out There" series of talks was pretty, er, out there. Particle physicist Brian Cox walked us non-particle-physicists confidently through the importance of the Large Hadron Collider, soon to open near Geneva. It was one of several talks in that section in which the enthusiasm of the speaker was so great that we all thought about dropping our careers and becoming particle physicists.

* The "What's Out There" panel was especially strong, but there were two talks that knocked me out. Joshua Klein, an animal behaviorist, talked about species that have adapted to human behavior. In particular, he talked about crows. He described his project to build a vending machine for crows and he showed short films about ways crows can take advantage of civilization: for example, dropping nuts onto boulevards so cars can crack them open. And author Richard Preston reported on his (and others') adventures high in the Redwoods, the unexpected ecosystems discovered up there, and the ecological threats they face. I don't want to simplify his rich argument -- his book The Wild Trees is a must-read -- but he does a great job of connecting the fate of the Redwoods and the fate of us.

* During the "What Will Tomorrow Bring?" session, the other Chris Anderson spoke not about his provocative "Free" thesis (on the cover of the new Wired) but about his less-than-$100 blimp, which he showed off, the product of his desire to make a "minimum unmanned aerial vehicle." Chris and the blimp will also be at ETech on Tuesday.

* Peter Schwartz argued that Wikipedia is a leader in the battle against poverty: it brings knowledge and possibility to places not getting them any other way.

The series of talks that most engaged me was the first one of the day, entitled "How Do We Create?"

* Designer Yves Behar urged us to question basic assumptions in out work. "Why do we have a CapsLk key on our computers"? he asked. "As a designer, I don't want to just slapping a new skin over existing technology." He certainly didn't do that when he designed the XO Laptop, which, among other delights, doesn't have a CapsLk key.

* Robert Lang, origami artist and mathematician, talked about how creativity depends on learning from those who came before us, even if we're taking lessons in ways the originators never intended. His best of many aphorisms: "the secret to productivity is letting dead people do the work for you."

* There was one particularly moving presentation during that session, from MIT's Tod Machover, which I'll give its own post after I have a bit more distance from it. (Benjamin Zander's great climax last night will get a shout-out, too.)

About that that anti-TED meme flying around the blogosphere that I mentioned last time...

Some of it has been cranky (such as a high-profile tech blog publishing the attendee list and vetting it), but at least one post -- from the usually very thoughtful Umair Haque, deserves to be addressed. His argument is that TED does more harm than good. I disagree. There are some things a reasonable person could argue against TED -- the inevitable elitism that comes with the high entry fee and the occasional self-congratulatory tone come to mind -- but by adopting a web-centric "ideas worth spreading" meme, I believe curator Chris Anderson has worked hard to make the ideas expressed at TED as available as possible. Most TED talks are available for free on the TED website. An official blog is reporting the events of the conference in near-real-time, as are a dozen or so unvetted ones. Haque's argument seems to be that TED does more harm than good because it hasn't saved the world (he denies that in the comments, but it seems to be his argument). Sure, it hasn't. No mere conference can. (Indeed, the even more elite Davos conference this year seemed to be full of leaders just throwing up their hands.) But while I suspect Anderson's goal is transformational change, the simple act of sharing transformational ideas -- first to a room of elites and then to the growing percentage of the world with Internet access -- lets them take root in unexpected places. That is definitely far more good than harm.

And now, off to the final day...

@TED: Best of Day 2

It was a day of extremes at TED, ranging from an extended session examining the pervasiveness of evil to an evening celebration of some of the most life-affirming ideas possible. It also ranged from the sober (how to survive a nuclear attack) to the self-referential and self-congratulatory (a brief sit-down with TED's originator, Richard Saul Wurman). Here's a quick rundown of some of the long day's many high points:

* There was a late-morning series of talks on the topic "Is beauty truth?", but it was in another session that we saw how truth brings with it some sense of beauty. Alisa Miller, president of CEO of Public Radio International, used the remarkable information visualizations of Worldmapper to illuminate what news stories get covered and what don't. (You can see her slides and hear her talk here.) In the end, she notes, "covering Britney is cheaper" than considering the more important stories.
* While interviewing Craig Ventner, TED curator Chris Anderson remembered an exchange they had a few years back. Anderson had asked, "Can you be accused of playing God?" Ventner's reply: "We're not playing." It was a joke, but Ventner took is as an opportunity to deliberate on how we're "supposed to use our knowledge to improve humanity." He also contrasted the optimism of TED with the pessimism of Davos
* In a brief talk, Dean Ornish gave his standard talk on why "our genes are not our fate," detailing the importance of lifestyle change. It was an interesting complement to Ventner and others showing how our fate could be in creating new genes.
* Philip Zimbardo, top researcher of the classic 1971 Stanford prison study and an expert witness for Abu Ghraib guard, showed some photos from the Iraqi prison that were more graphic and troubling than what's been in the mainstream press, and hammered on the parallels between his landmark study and what is happening today. Countering the "bad apple" theory of people who behave terribly when placed in unsuperivised positions of complete power over others, he stated, "it's the barrel that's bad, not the apples." As intense as that session was, it kicked off with a brief clip about the just-ended next-generation DVD wars that managed to be both hilarious and offensive.
* While accepting their TED prizes, novelist David Eggers delivered a tour-de-force tour through his 826 writing and tutoring project, and Neil Turok shared his double life as a physicist (he's not so sure the Big Bang started everything) and as a founder of the African school for math and sciences.

And that doesn't even include Samantha Power's stirring talk about diplomat Sergio Vieira de Mello ... or Susan Blackmore's sharp application of Darwinian principles to everything ... or Isaac Mizrahi's star turn ... or 100 other fascinating moments. And the best news is that this event is only half over.

Finally, I've noticed an anti-TED meme flying around the blogosphere this morning. I'll address that tonight, when I write my post about today's events.

Teaching design to businesspeople

The "D" in TED stands for "design," and it's become a truism that design is a crucial element of business success. Ask Apple. But the conventional wisdom still maintains that design is a "soft" art, not worthy of attention by serious businesspeople. It's for the designers, the marketing people. And when top executives insert themselves into the design process, the results are often unintentionally hilarious. I remember a decade ago, when I made my living in part by building websites for large media companies, one meeting in which a client CFO rejected an iteration of a site because he didn't like the shade of mauve in the background. (Philip Greenspun joked about these sort of people -- go to this page and search for "mauve" -- but it really happened.)

There's a growing literature detailing how businesspeople can build and sell better products if they think like a designer. Jane Fulton Suri's Thoughtless Acts?: Observations on Intuitive Design is a tiny classic that does nothing less than teach businesspeople a new way of looking at everything. And this month O'Reilly is publishing Subject To Change: Creating Great Products & Services for an Uncertain World, in which the Adaptive Path team shows how important design is, in terms businesspeople can understand.

And now these ideas are filtering into education. In one high-profile example, the Stanford "d.school" is doing much to connect business and design. Andy Oram, in an internal O'Reilly list, pointed us to a fascinating article by Terry (Bringing Design to Software) Winograd of Stanford in the recent Interactions called "Design education for business and engineering management students: a new approach." Winograd reports on the insights that have come to Donald Norman as he has sought new ways to infect nondesigners with design smarts. The author of The Design of Everyday Things and Emotional Design has developed a "design track" in the Master of Manufacturing and Management program at Northwestern. It's still early on, and Winograd's perspective in the article is prospective, but it's another data point that the next generation of businesspeople may be able to think at a deeper level than shades of mauve.

@TED: Best of Day 1

If nothing else, TED is a trip. The veteran conference has gone through many permutations. Under curator Chris Anderson, TED is still full of technology, entertainment, and design, but it has really lived up to the change-the-world rhetoric that was always a bit more under the surface during Richard Saul Wurman's ace stewardship. Al Gore's talk about global warming turned into An Inconvenient Truth after a movie producer saw him deliver the talk at TED; Pangea Day, an ambitious attempt to create a world-wide one-day film festival (it's coming May 10) came out of TED as well. And this week E.O. Wilson is debuting the first iteration of his Encyclopedia of Life, funded by a TED grant. Indeed, the change-the-world attitude is so great that the only truly negative feedback I heard at last year's event was over how wasteful the opulent gift bags were. So this year the bags are constructed from 100-percent post-consumer recycled beverage bottles by Rickshaw Bagworks.

The conceit of this year's TED, now in its final installment in longtime home Monterey before a move to Long Beach, is "The Big Questions." I'll chronicle some of the high points of the conference here. For more detailed coverage, the official TED blog is offering blow-by-blow coverage. And the event, while aiming to be iconoclastic, has become so iconic (and expensive and exclusive) that it has inspired its own barcamp alternative, as noted by Jerry Michalski.

Some of the choice moments of the first day:

* Third generation paleoanthropologist Louise Leakey explaining what you need to do to if you want your remains to be found as a fossil (there's lots of luck involved if you want to be preserved for millenia), and how "technology removes barriers to population growth"
* Priceline founder Jay Walker demonstrating how it takes a lump of coal to transmit a megabyte of data across the Net
* Photographer Chris Jordan talking us through "Running the Numbers," a series of dramatic information visualizes focusing on consumption in the U.S. He learned that, for example, one million is the number plastic cups used on planes in the U.S. every hour and delivered a devastating visualization of that data point.
* * Guitarist Kaki King channelled Preston Reed and the ghost of John Fahey in a surprising and thrilling manner
* Roy Gould and Curtis Wong debuted Microsoft's WorldWide telescope, which may, as Gould put it, "change the way we do astronomy." Simply, it allows us to see the sky -- and what lurks beyond the sky -- in an entirely new way (the talk is already on the TED site).
* The presentation from the day that burned itself immediately into my long-term memory came from Jill Bolte Taylor, a neuroanatomist who showed us a real human brain -- and spent most of her talking describing her own stroke and what she felt and thought while her brain was going wild, from the borderline-metaphysical ("I can't define where I begin and where I end") to the borderline-hilarious ("I'm a busy woman. I don't have time for a stoke"). Her description of her time in that strange state, caught between two worlds, the rare neouroanatomist who has been able to chronicle a brain-changing event from the inside, was astonishing.

And that's just cherrypicking from the first day, a half day of talks. There are 2-1/2 more days coming...

(There are always fun interstitial film clips between talks. You can't beat a harmonica-playing Darth Vader.)

The Dangers of Predicting the Future

The instant-analysis business is a tricky one. None of us have working crystal balls; any attempt to predict the future, even the five-minutes-from-now future, is risky.

For example, on January 31, mere hours before Microsoft made its unsolicted $44 billion-plus offer for Yahoo, Forrester Research, my alma matter, posted a research note with the following headline and deck:

Microsoft Will Make Small Acquisitions
Its Size, Visibility To Antitrust Bodies, And Strategy Rule Out Big Deals

I'm not pointing this out to embarrass a particular firm. Most prognosticators play the same dangerous game. Indeed, Gartner Group, the leader in the tech-research space, has been known to quantify its predictions in its reports. When it expresses 90 percent confidence, it means that there's a one in 10 chance that its preimum-priced look into the future is out of focus. They're not sure.

We all want to know what comes next. It would be great to know in advance if buying that stock, taking that job, or marrying that person is the right idea. But we can't. As Salman Rushdie wrote during his years on the run from the fatwa, "Our lives tell us who we are." We can't know for sure how things will turn out until they happen.

But we can surmise the context in which the future will occur. We can aggregate early signals and make smart decisions based on them. Rather than calling outcomes, we're here to call trends, to cut through conflicting signals and discern the most powerful ones. We don't know which company will become the leader in multitouch devices, for example, but we've seen plenty of signals to support a contention that multitouch will be huge in the years to come. That's why I'm so excited about two just-around-the-corner events, TED and our own ETech: They give you a peek into the future, straight from some of the places where it's happening already.

The LiveScribe Pulse Smartpen

The coolest thing I saw at our Tools of Change for Publishing Conference last week was the LiveScribe SmartPen. This amazing pen includes a microphone and an optical sensor that synchronizes the audio with any notes you take on special microdot paper using the pen. Touch the appropriate point on your notes to replay the relevant section of the audio stream. The quality of the recording was good, and it was really uncanny to point the pen at words written on the paper and hear what actually happened. The 1 gigabyte model can record 100 hours; the two gigabyte model can record 200 hours.

This device is one more sign of what I've been calling ambient computing, the interpenetration of computing with the physical world via sensors. I've tended to focus on cloud computing ambience, though, and this is a personal device. It's also a fulfillment of projects such as Gordon Bell's MyLifeBits, the idea that decreasing storage costs will eventually mean that we'll have the ability to record our entire lives in digital form.

It's also a sign that the physical computing revolution, whose alpha-geek stage we've been documenting in Make:, is entering its next phase, in which entrepreneurial opportunities emerge. It's so wonderful to see an invention that is so much "on trend" that in retrospect it seems inevitable, yet in its first appearance is so unexpected and remarkable!

I've put in my pre-order. This is the coolest device I've seen in a long time. Especially cool that they are opening it up as a platform, with an SDK for developers to build new applications for the device.

Understanding the undersea cable cuts... (updated: "fifth cable cut")

The Fiber Cuts in the Middle East are getting a lot of attention. The economic damage is real and the geopolitical issues are extremely complex (which is why I edited my earlier post).

From an operations perspective these kinds of outages are nothing new, and underscore why having "many eggs in few baskets" is such a problem. I believe we will see similar incidents when we have the first multi-datacenter failures where multiple providers lose significant parts of their infrastructure in a single geographic area. (Remember: location is a basket too!)

To really understand the current issue, I recommend Neal Stephenson's incredible (and lengthy) Wired article from 1996 entitled "Mother Earth Mother Board":

[...] It sometimes seems as though every force of nature, every flaw in the human character, and every biological organism on the planet is engaged in a competition to see which can sever the most cables. The Museum of Submarine Telegraphy in Porthcurno, England, has a display of wrecked cables bracketed to a slab of wood. Each is labeled with its cause of failure, some of which sound dramatic, some cryptic, some both: trawler maul, spewed core, intermittent disconnection, strained core, teredo worms, crab's nest, perished core, fish bite, even "spliced by Italians." The teredo worm is like a science fiction creature, a bivalve with a rasp-edged shell that it uses like a buzz saw to cut through wood - or through submarine cables. Cable companies learned the hard way, early on, that it likes to eat gutta-percha, and subsequent cables received a helical wrapping of copper tape to stop it.

[...] There is also the obvious threat of sabotage by a hostile government, but, surprisingly, this almost never happens. When cypherpunk Doug Barnes was researching his Caribbean project, he spent some time looking into this, because it was exactly the kind of threat he was worried about in the case of a data haven. Somewhat to his own surprise and relief, he concluded that it simply wasn't going to happen. "Cutting a submarine cable," Barnes says, "is like starting a nuclear war. It's easy to do, the results are devastating, and as soon as one country does it, all of the others will retaliate."

As the capacity of optical fibers climbs, so does the economic damage caused when the cable is severed. FLAG makes its money by selling capacity to long-distance carriers, who turn around and resell it to end users at rates that are increasingly determined by what the market will bear. If FLAG gets chopped, no calls get through. The carriers' phone calls get routed to FLAG's competitors (other cables or satellites), and FLAG loses the revenue represented by those calls until the cable is repaired. The amount of revenue it loses is a function of how many calls the cable is physically capable of carrying, how close to capacity the cable is running, and what prices the market will bear for calls on the broken cable segment. In other words, a break between Dubai and Bombay might cost FLAG more in revenue loss than a break between Korea and Japan if calls between Dubai and Bombay cost more.

The rule of thumb for calculating revenue loss works like this: for every penny per minute that the long distance market will bear on a particular route, the loss of revenue, should FLAG be severed on that route, is about $3,000 a minute. So if calls on that route are a dime a minute, the damage is $30,000 a minute, and if calls are a dollar a minute, the damage is almost a third of a million dollars for every minute the cable is down. Upcoming advances in fiber bandwidth may push this figure, for some cables, past the million-dollar-a-minute mark. [Link]

Update Feb-06 @ 08:52 GMT: I am aware of five cable segments that are experiencing problems, including one that was reported on January 23rd which had a repair already underway. I don't think this is a "fifth cut" as some people are starting to report, and I'll post an update if that changes.

A lot of needless confusion and worry could be avoided if FLAG Telecom and the other carriers involved would provide timely and useful updates on their website. It appears that they are doing a good job of restoring connectivity, but they are terrible job of telling an increasingly concerned public exactly what is going on. This kind of confusion resulted in false reports that "Iran was completely offline", which was corrected by the Renesys blog team after the story spread to influential blogs, Slashdot, Digg, and the mainstream media.

CNN Goes Multitouch

We've been noting more and more signals that multitouch screens are catching on beyond the iPhone and iPod Touch. We've recently seen a hint from Microsoft and a Walt Mossberg overview in the Wall Street Journal. And there's a high-profile one going on right now.

CNN is reporting on the unfolding Super Tuesday presidential primary results and on-air personalities are using, in addition to standard graphics, large Jeff Han-invented Multitouch Collaboration Walls to interact with real-time voting data. The visualizations aren't necessary useful (pie charts based on 3% of precincts reporting are stupid), but the screens are being used to make data come alive.

"It's a stupendous way to explain a lot of complicated data," David Bohrman, chief producer of CNN's political coverage, told the Wasington Post today. "Fundamentally, our job is to explain things to people, and we need to do it visually. This lets us do it naturally, without a keyboard or mouse getting in the way."

It's way too early to predict that multitouch will supplant point-and-click the same way point-and-click made most text-only interfaces obsolete, but tonight signals how much traction multitouch is getting already.

For more Radar insights on multitouch, see Tim O'Reilly's pinpointing what the Kindle is missing and Nat Torkington's report on Jeff Han's talk at ETech 2006.

PMOG as a kind of Augmented Reality

Mike Arrington did a good overview post this past weekend entitled Play A Multiplayer Online Game While Surfing The Web about O'Reilly AlphaTech Ventures' portfolio company GameLayers, whose first product, a "game" called PMOG, is due out of private beta in a few weeks.

In this post, I want to connect what GameLayers is doing to the theme I wrote about the other day, augmented reality. A key concept that we all need to wrap our head around is the idea of multiple information layers. The narrow view of augmented reality is the overlay of generated images onto, say, a real-time video stream, but I think of it as any technique that creates additional layers of information on top of the primary layer we're interacting with. Google Maps' blended view with street lines overlaid on satellite imagery is thus a kind of primitive augmented reality. Adding 3D images of buildings would be a further augmentation. As would being able to twitch sideways to a view of data about the companies or people residing in those buildings, or downwards to view the subsurface infrastructure of sewers, water pipes and cable runs.

Taken in this broad sense, now consider what GameLayers is doing. Via a Firefox plug-in, they provide a platform for building additional layers across websites. As Michael explains, this could be used to create virtual tours of related sites, quests and other game-like activities, all happening on a layer that is invisible to many of the people coming to those sites, but all too real to those inhabiting the shared consensus reality of the new layer.

GameLayers is thus a bold experiment in searching out new possibilities in cyberspace. We invested in the company because of the long view that the web is only the first level of electronic augmentation of the world we live in. There are many more levels to be built, until, as Wallace Stevens said:

One might have thought of sight, but who could think
Of what it sees, for all the ill it sees?
Speech found the ear, for all the evil sound,
But the dark italics it could not propound.
And out of what sees and hears and out
Of what one feels, who could have thought to make
So many selves, so many sensuous worlds,
As if the air, the mid-day air, was swarming
With the metaphysical changes that occur,
Merely in living as and where we live.

Augmented Reality: A Practical Guide

I'm jealous. The Prags have just published a book I wish I'd thought of first: Augmented Reality: A Practical Guide.

I've been talking quite a bit about augmented reality lately, especially when people ask me about what I think might represent a discontinuity significant enough to represent a paradigm shift of the scale of the PC revolution or Web 2.0. Sensors instrumenting the world and driving collective intelligence applications that provide new information layers in our everyday experience is one big element of this next revolution.

The topic also comes up whenever people ask me about Second Life, because I'm much more fascinated by the possibility of SL to create additional information layers on top of this world than I am about the idea of it as a complete alternate reality. I usually point to an SAP project I learned of last year, in which SAP is working with a Swiss property management firm to build instrumented models of their buildings in Second Life. That is, you open a door in the building, a door opens in the SL model. The building catches fire, so does the SL model. And of course, that's why I was so excited about Google's acquisition of Sketchup. It seems to me to be a really important long-term play in the mapping space. After all, so much of the built world we interact with isn't represented at all on the maps we use. An address on the 37th floor of a building looks just the same to our mapping system as one on the first floor. But does it need to be that way? Not in a future where we've populated our maps (at first perhaps Google Earth, but eventually web-based maps as well) with additional layers representing the human-built world.

Augmented reality is also coming at us in the news, especially forward looking news outlets (hint: "News for nerds. Stuff that matters.") Take a look at these recent Slashdot headlines and think about them as all part of an emerging augmented reality trend: Smart 'Lego' Set Conjures Up Virtual 3D Twin, Cellphone App Developed that Could Allow For 'Pocket Supercomputers', Stanford's New Website Converts Your Photos to 3D, and The Coming Wave of Gadgets That Listen and Obey. Add in the recent Radar posts The Future of Cell Phone Headsets and More on the Virtual Reality Audio Headset. Season with a dash of Nintendo Wii and innovative cell phone games like Mobzombies (Radar post.)

We're clearly careening towards a world in which virtual worlds are overlaid on the real world, bits interpenetrated with atoms.

I should be clear that this broad-strokes definition of augmented reality isn't what's covered in the Prags' new book. They are focused on a more traditional definition: "to create the sensation that virtual objects are present in the real world." They provide some first tools for developers to explore interfaces and techniques for doing so, with an emphasis on overlaying rendered objects onto real time digital video. This is a subset of the big picture I'm drawing in this post, but an important one. And perhaps even more to the point, this book will help to socialize the idea and to get people started building the new skills that will be required as augmented reality interfaces go mainstream.