Burning Man Gets an API (and a Whole Lot More)

by Brady Forrest | @brady | comments: 17

An API! SMS! Foursquare! An iPhone app! They are all coming to Burning Man this year. Will the festival be the same?

The annual tech-art festival in the Nevada desert, starts on Sunday. Normally the attendees leave their phones and laptop behind, but this year that may not be the case. As I ride from Seattle to Black Rock City, NV I am getting SMS from friends on the playa. In anticipation of wifi and possible data connections Foursquare has rolled out Black Rock City as a city (@sfslim is already the Mayor of The Man). If AT&T's service doesn't work then attendees may be able to take advantage of OpenBTS's local SMS project. Most of the attendees aren't there, but the tech is already making its presence known.

Burning Man is dismissed as a party by many people (attendees and non-attendees alike), but for many it is a unique opportunity to try out new software. Geohackers in particular find it to be a great playground. Black Rock City is a full city complete with a fire department, stores (where you can buy coffee, tea or ice bags), a Main drag and 40,000+ residents. However, since it is only around for a week each year (and is always in a new location) there is not time (or profit) for commercial companies to map it. The process falls to the community and they take advantage of the opportunity (and sites like Flickr use the resulting commercial-grade data).

This year the Burning Man organization is assisting with the launch of an API. With the API you get access to descriptions and locations of the Streets, Art, Camps and Events. When combined with a map this is everything you need for a local city guide. And that is exactly what the iPhone app does (it's not available in the app store; if you want it head to the Burning Man Earth Camp next to Media Mecca -- be nice). It maps all of those entities, will geolocate you and let you mark favorites (see the screenshot from my iPhone). You can learn more about the API project here. Burning Man still has its Virtual Playa project online.

There is also a move to take advantage of Flickr's machine tags. For example if you take a picture of Area 47 (with the online directory entry: http://earth.burningman.com/brc/2009/themecamp/2234/) then use burningman:camp=2234. The photo will appear on that locations page. We will see how many photos end up using these machine tags. I suspect that V2 of the iPhone app will add a camera that can apply those tags automatically and that we'll see more uptake then.

Burning Earth team member, Tom Longson, sent me the following.

Burning Man's theme this year is evolution which is fitting as Burning Man Earth launches an online directory, API, and a beta iPhone App. The group of artists, geo-wankers, and software developers are rapidly deploying systems, both off and on the Black Rock Desert playa to help participants find each other, schedule events, find theme camps, and artwork. It is a digital project aimed at providing better maps, and an online space to describe the community and art.

The open source webapp, named "Earth", builds upon Open Street Map, GeoDjango, and Pinax to create an easy to use, mapping interface for the event. Coupled with Jeffrey Johnson's prior work with aerial photography, and Andrew Johnstone's virtual playa 3D modeling, the platform is rapidly evolving to become an important part of the organization of the event.

Burning Man's API now opens the door for developers and artists alike to remix and reuse data about the event. For example, you could plot all the events in the next hour, build an Arduino belt that vibrated in the direction of the closest piece of artwork, or a web service for rating theme camps.

In addition, Mikel Maron is championing machine-tags to allow the project to couple Earth's database with other websites, such as Flickr. By integrating machine tags, people can say on Flickr what art installation their photo is of, and Earth will automatically pull up that photo. Likewise, Flickr will provide a link to the page describing the artwork itself.

Beyonds enabling mashups, the APIs are the foundation for the new beta iPhone app, which serves as both a directory and enhanced GPS designed for Burning Man. A small number of participants will get to try out the app, which will be in full production next year.

While it may sound like fun and games, the harsh conditions of the Black Rock Desert make the system a perfect testbed for mapping temporary places, people, and things. In this same way, these tools may just be the next best thing for helping disaster hit regions react and respond. Burning Man Earth is more than just an attempt at radical self-expression, self-reliance, and community building. It may just be a tool for tomorrow.

This is Burning Man at its best. Letting people create something just for the festival and its attendees. The question becomes how will the larger Burning Man community, expecting a cellphone free vacation, react to intrusions from the real-world?

BTW, If you are on the playa you may be able to find me at my group art project Steve the Robot H.E.A.i.D.

Is intimate personal information a toxic asset in cloud datacenters?

by Carl Hewitt | comments: 15

Guest blogger Carl Hewitt, Emeritus at MIT in the Electrical Engineering and Computer Science department, is known for his research on strongly paraconsistent logic, privacy-friendly client cloud computing, norms and commitments for organizational computing, and concurrent programming languages, models, and theories.

Aggregators (Google, Yahoo, Microsoft, Facebook, etc.) tend to believe that personal information is a valuable asset for several reasons. It is valuable to advertisers because it enables greater relevance for their ads. It is valuable to users because it can be used to enrich their lives. And it is valuable to aggregators because they can use personal information to make more money by selling (anonymous?) versions and by using it to bring together advertisers and customers. Recency and intimacy can add value to information. Current and recent information tends to be more relevant than older information. Intimate psychological, physiological, sociological, geographical, medical, etc. information can be used to personalize interactions.

Intimate current personal information is also valuable for government security because it can be critical to taking security counter measures. Already in the UK, the previous two years of everyone's email, web browsing, and telephone calls are becoming available to government officials at varying levels of detail. For example, detectives will be required to consider accessing telephone and internet records during every investigation under new plans to increase police use of communications data.

But that's only the beginning. As Jim Gray noted in "Distributed Computing Economics" (MSR-TR-2003-24) there is a growing imbalance between the computation power of billions of cores in aggregator datacenters and the relatively feeble fiber optic communications coming out of aggregator datacenters. This problem has now become so severe that Amazon has been forced to introduce a commercial service that lets users of their cloud import and export data through the post--as in, put it on storage devices and ship it by land, sea, or air. Soon even this stopgap will become impractical for government security agencies because whole shipping containers would have to be transferred--the functional equivalent of shipping large pieces of an aggregator datacenter. Consequently, to be effective, future government security software will have to be tightly integrated with aggregator datacenters. The most effective security measures will require aggregator datacenters to be heavily regulated, i.e., analogous to nuclear power plants.

Semantic Integration, an emerging technological capability to bring together all kinds of information in a semantic engine, will greatly intensify all of the above issues (see "A historical perspective on developing foundations for privacy-friendly client cloud computing: The Paradigm Shift from 'Inconsistency Denial' to 'Practical Semantic Integration^TM' " ArXiv 0901.4934). The following kinds of information can be semantically integrated: calendars and to-do lists, email, SMS and Twitter archives, presence information (including physical, psychological and social), maps (including firms, points of interest, traffic, parking, and weather), events (including alerts and status), documents (including presentations, spreadsheets, proposals, job applications, health records, photos, videos, gift lists, memos, purchasing, contracts, articles), contacts (including social graphs and reputation) and search results (including rankings and ratings).

Two critical technologies are the foundation of Practical Semantic Integration: The first is Lightly Structured Natural Language^TM interfaces that allow information to be easily found and organized. The second is many-core semantic engines (see "ActorScript^TM: Industrial strength integration of local and nonlocal concurrency for Client-cloud Computing""; ArXiv 0907.3330) that rapidly process information in ways that are tolerant of inconsistency (see "Common sense for concurrency and strong paraconsistency using unstratified inference and reflection" ArXiv 0812.4852).

To be effective, government security Semantic Integration systems will need to be joined with those of aggregators. Thus Semantic Integration of personal information on aggregator datacenters will require additional government regulation of aggregators. Will government regulation prove toxic to the ability of aggregators to innovate?

This is a future that we expect most readers would find distasteful. There is an alternative: A client cloud is a local cloud controlled by a client, e.g., a family cloud might consist of the cell phones, computers, security cameras, home entertainment centers, Wi-Fi access points, etc. of a family. Semantic Integration could be performed in clients' clouds so that clients by default store their information in cloud datacenters in a way that it can be decrypted only by using a client';s secret key.

Semantic Integration using clients' clouds has some important advantages. Client responsiveness can be faster by not requiring communication with datacenters. Aggregator capital, operating and communication costs can be lower because Semantic Integration is performed in clients' clouds instead of aggregator datacenters.

By performing Semantic Integration in clients' clouds, aggregators can make tons of more money than now by doing an even better job of matching up customers with merchants in a way that is more pleasing to both. Aggregators can provide software that runs in the clients' clouds (although it may have to be audited by 3rd parties). The aggregator's software can volunteer high level information to the aggregator's datacenters about the kind of merchant information that might be relevant. Within clients' clouds, the merchant information can then be tailored to the specific requirements of clients.

For reasons above, an aggregator can do better by performing clients’ Semantic Integration using their clouds rather than relying entirely on the aggregator's cloud. And using clients’ clouds could lessen the degree of government regulation because the government would have to subpoena clients to obtain their most intimate personal information. If the information in an aggregator’s datacenters is sufficiently anonymous, then it would not become necessary for government security agencies to regulate them so heavily.

The question is: "What are the aggregators going to do about intimate personal information?" If one of them initiates a project to develop a Semantic Integration product that operates in clients' clouds, then the others will rapidly follow suit.

The Government Blocks Twitter No It Doesn't

by Mark Drapeau | @cheeky_geeky | comments: 11

In a recent CSPAN interview, White House spokesman Robert Gibbs noted that, “for some reason, Twitter is blocked on White House computers,” which created a minor frenzy among tech-savvy journalists ranging from UPI to The Hill. Later, news upstart Mediaite uncovered that the New Media team in the Old Executive Office Building could indeed access Twitter, but other people working on White House staff do not necessarily share the same privileges. This is all very interesting, but this story is far bigger than the White House, because it serves as a metaphor for rules governing social media tool use for the thousands of employees working throughout the Federal government.

Decisions about which social media sites are allowed in the Executive Branch are somewhat inconsistent, as I pointed out in a Department of Defense research paper earlier this year. Often without explanation or transparency, different agencies and even offices within agencies have different policies about use of social media platforms on the Web. Additionally, even when public affairs employees are allowed to use tools like Twitter and YouTube to communicate, they are sometimes blocked by different authorities at work from using them. So, in a gray area, they employ workarounds using personal laptops, iPhones, and the like.

Such internal contradiction cannot last long. Eventually there will have to be consistent, widely-known policy guidance about what sites can be used, and by whom, and why. And as the workforce age structure changes, and lines between professional and personal increasingly blur, employees will demand access to these sites more. Some sites may legitimately be blocked, but currently, there are a hodgepodge of rules that are often confusing, and possibly make the overall situation worse. Here, I propose two arguments for not blocking most social media sites on most government computers.

One, blocking social media sites does little for safety and security. The statement “Twitter is blocked” typically means that the domain Twitter.com is rendered inaccessible from a government Web browser. The downside to blocking sites this way is that there are simple mechanisms for alternatively accessing the underlying software (Twitter.com can be accessed from TweetGrid.com, YouTube.com from videos.Google.com, and so forth). Hence, official computers can access the same sites through different portals. Employees may also turn to nearly ubiquitous personal devices like BlackBerries to use social media during work hours. Finally, there are many "clones" of sites like Twitter and YouTube; are Identi.ca, Plurk, and similar microsharing sites also blocked? Thus, some employees effectively use the same social networks to send and receive the same information, with all of it being harder to monitor. This is not a recipe for good cyber-security of government systems or employee information.

Two, blocking social media platforms does little for government efficiency, transparency, and citizen engagement. True, when used poorly, sites like Twitter and YouTube are a distraction from official duties and a time-sink. But the same can be argued about phones, email, and even the cafeteria. When used responsibly, however, social media provides real-time information about critical news, helps employees working on similar topics within the government find and communicate with each other, allows the discovery of work-related conferences and other events, helps people better understand how technology is influencing overseas incidents like the Iranian election protests, conversing with citizens about microniche issues related to the office one works in, and countless other worthwhile applications. Blanket social media bans empower information to fall through the cracks rather than get to people who could use it.

Three reasonable steps should be taken. First, top-level government information assurance analysts need to determine what security risks various common social media websites pose to the government; they should be “binned” into categories like “Use only on non-military computers” or “Not for government system use.” Second, policies need to be transparent, consistent, and well-publicized across the government; employees will frown on radically different policies being applied in different buildings on Independence Avenue, or on different Army bases in Virginia. Third, employees and contractors working in government facilities need to be educated about the positive and negative aspects of using social media websites, just as they are about other aspects of cyber-security and other government procedures.

These three steps should counteract possibly less secure employee workarounds, and go a long way towards the more open, transparent, and participatory government that the President proposed in the first memo he disseminated after taking office. Interestingly, while the U.S. debates whether or not certain computers can and cannot access Twitter, across the pond the U.K. has released an official government template for how to use Twitter - it's a 20-page document offering practical advice, and uploaded online using Scribd for the entire world to see. Just as we look to other countries for ideas about how we can improve transportation, health care and the like, we might include social media on that list.

Bantamweight Publishing in an Easily Plagiarised World

by Mark Drapeau | @cheeky_geeky | comments: 10

Even professional writers are prone to infrequent accidental plagiarism. But in the world of novels, newspapers, and college exams, there are rules about bootlegging others’ work that are well-established - most everyone agrees on what behaviors are unacceptable and what the consequences are. In bantamweight publishing, however, the rules are not so clear.

In order for the British Army to raise more units during the First World War, it created battalions of otherwise healthy men with lowered minimum height requirements. In this way, short, powerful miners and similarly swarthy individuals were able to contribute to the war effort. These soldiers were called bantams (a term now heard most commonly in boxing, bantamweight). Similarly, in a Web 2.0 environment, the short powerful bursts of searchable, findable, and sharable data emitted from personal electronic devices are a form of bantamweight publishing in which persons outside the regulated publishing industry can contribute to the information sharing effort.

Bantamweight publishing comes in many forms. Twitter is certainly in this category, but there are a steadily increasing number of ways to share small bits of information with the world. From updating your Facebook Wall to Yammering inside your enterprise to updating your LinkedIn status to commenting on people’s BrightKite locations, everyone is doing it. But in an easily plagiarized world, who owns your sentences once you publish them? It’s not really clear. And in a murky environment where someone might get a macropublishing book deal by popularizing someone else’s creative hashtag, bantamweight publishing runs the risk of serious future problems.

Oh, bantamweight publishing has its customs. Self-policing crowds ensure that most people who lift someone else’s excellent quote or funny picture or news link give credit to the originator using the “retweet” (RT) convention followed by a username. But there is little downside to cheating relative to being expelled from college or fired from your newspaper. As is well known in animal behavior circles, it can be temporarily advantageous for cheaters to infiltrate a system like this.

To be sure, quoting someone’s original haiku verbatim and making it appear as if it were your own is an infraction of bantamweight publishing customs. But what if someone tweets an Abraham Lincoln quotation - must the re-tweeter cite the originator? The custom seems less pressing in this case, mainly because of a lack of intent to deceive and arguable "fair use" of a well-known statement by a famous person. One can imagine altruistic plagiarism as well, where people repeat memes to raise money for charity, or virally make people aware of an immediate Amber alert. Further, who could fault someone for copying information about a charity onto their Facebook Wall without citing the originator? In the bantamweight publishing world, information sharing can easily supersede attribution. There are gradations of citations.

Bantamweight publishing is popular among those who feel brevity is a virtue. But when an entire work of art is bounded in 140 characters, even brevity has its limits. Sometimes, squeezing in a proper attribution through editing content can change the original meaning, when the edits unwillingly shift from cosmetic to substantive. And what happens when you run out of space when attempting to retweet someone who retweeted someone who tweeted an important quotation from the Washington Post? To a large degree, a work of bantamweight publishing is like a painting with an upper weight limit, where the novelty is the canvas and the attribution is the frame; most viewers would choose to appreciate the canvas without the frame if given the hard choice.

Another major difference between regular publishing and bantamweight publishing is the lack of research and editing standards. Sometimes people attribute flawed information properly. It is obvious that excellent curators of information like NYU professor Jay Rosen and publisher Tim O’Reilly are exceptions to the rule, based simply on the phenomena of Rick Rolling, #moonfruit, and celebrity death hoaxes. To many, bantamweight publishing is not an micro-investigatory piece to be researched, sourced, edited, and spread, but rather a form of enhanced social chatter and gossip spreading. And according to the rules of gossip, it doesn’t really matter where it comes from; gossip is fun.

Few would argue that the British bantam units were a bad idea, and likewise bantamweight publishing has many virtues. But there are also pitfalls to this in an easily plagiarized world, particularly when money comes into play. Who’s looking out for the intellectual property of a winning hashtag that becomes a book, or a stream of haikus that becomes a blog that companies advertise on? At some point, bantamweight publishing will no longer be a lawless frontier territory; what will it look like next?

How I Learned to Stop Worrying and Love the Bot

by Mark Drapeau | @cheeky_geeky | comments: 2

Web technologies often allow you to scale things that weren't scalable before. Unfortunately, that list of scalable things includes spam. From unsolicited phone calls to unwanted emails to unnecessary tweets, it can seem like we're getting progressively overloaded with information we don't necessarily want. One group blamed for the increase in online spam are Twitter bots - Twitter accounts created to automatically perform certain behaviors like following anyone who mentions "candy" or retweeting any mention of the the word "fashion."

Some people find such bots to provide annoying and useless clutter. I, on the other hand, have come to love the bot. In an age of information overload and filter failure, good bots can act as an initial filter for discovering pertinent things within the real-time information ecosystem. Unless you're fanatical about a subject, why follow 100 military bloggers or 250 marketing gurus or 85 fashionistas when one or two bots can collate their best stuff and simplify your life? Who has time to find all these accounts, track up-and-comers, and listen to everything they say? I tend to be a one-bot kinda guy. And when I find a good bot, I hang onto her.

Bots love to exchange gifts with you, too. Generally, my tweets are relatively long and filled with informative nouns - and bots appreciate my efforts. Every instance when I mention "celebrity" or "baseball" or "journalism" to the world is like giving a tiny gift to a bot that keeps it relevant. And in exchange, the bot ensures that my information automatically gets to a wider audience of people that I don't know yet. Around the clock, bots are selflessly recruiting my next generation of fans. And my love affair with bots is just beginning; because they're inherently unjealous creatures, I can use as many bots as I want, whenever I want, however it pleases me.

Brands, on the other hand, always want to be my soulmate, even though they don't often love me back. They don't tell their friends what they heard from me, and they don't share their best gossip with me. Usually when I meet a brand, I find them to be a very distant anti-filter that talks only about themselves. They're rarely chivalrous. Unlike bots desperately seeking my attention, brands only want me when I talk to them first. Sometimes they thank me for the compliments, and sometimes they're sorry they hurt me, but either way I always feel a bit empty after talking with a brand. Brands are just not that into me.

Brands tend to be very jealous and are always checking to see if I'm being faithful. Yet while I sit around hoping they'll get in touch, they always seem to be busy talking to someone else. One brand that's on my mind all the time is Comcast, but how often do they ever think about me? According to my diary, @comcastcares wrote me four times, and @comcastbonnie just once. Frank and Bonnie (and Scott, too) never suggest novel things I might like to watch based on shows I tweet about, never give me the latest news about high-speed Internet connections, and they don't even try to sell me on the digital phone service I don't have. This brand only tries to make me happy after they've hurt me.

I'm not the only one getting his feelings hurt. Unlike bots I love who share my information and give me some in return, Comcast rapidly narrowcasts in a multiplexed Kabuki dance designed to cheer us up when we're feeling blue about them. Frank and his colleagues send messages to specific people 97% of the time, and retweet what they say less than 1% of the time. And its not like other brands are thinking about me a lot either - even my beloved Starbucks only retweets fans like me about 1% of the time. Sure, I'm on cloud nine during an occasional encounter with a brand I really like, but they always seem to roll over and ignore me afterwards.

Developing relationships in a socially networked era is difficult because there's less old-fashioned courtship and more emphasis on "hanging out." It's hard to find a truly generous brand nowadays. That's why when it comes to brands, I like to spend my nights with non-profits that friends set me up with. A new article points out a lot of great reasons to develop relationships with non-profit brands: they're member-driven, promote community participation, create value in people's lives, and engage audiences by speaking to their primary interests.

Developing a relationship with a non-profit brand in this economy is hard, though - they always want me to pay for everything. Bots, on the other hand, are happy to go Dutch. Sure, most Twitter bots aren't great at engaging in conversation, but I think they can be thought of as stripped-down non-profits. The reason I have learned to stop worrying and love the bot is because they're created by passionates to collate knowledge from people they don't know, and share it with other people they don't know. Call me Dr. Strangelove, but I'd rather have a good one-night stand with a generous bot than a bad long-term relationship with a selfish brand.

Twitter is Not a Conversational Platform

by Mark Drapeau | @cheeky_geeky | comments: 54

Perhaps the most common reason given for joining the microsharing site Twitter is "participating in the conversation" or some version of that. I myself am guilty of using this explanation. But is Twitter truly a conversational platform? Here I argue that the underlying mechanics of Twitter more closely resemble the knowledge co-creation seen in wikis than the dynamics seen with conversational tools like instant messaging and interactions within online social networks.

Wikis are causally thought of as platforms for "collaborative" document creation. But on Wikipedia, while many people share knowledge to co-create pages, the process is not formally collaborative in the sense that contributors are not cooperating with each other ways that form group identity (to paraphrase Clay Shirky from his book Here Comes Everybody). To the contrary, passionate experts write the majority of text, and a long tail of other contributors offer relatively few, small edits. Many users contribute nothing. Through this process, Wikipedia pages often become valuable repositories of knowledge.

Brian Solis recently posited the dichotomy of whether Twitter is a conversational or broadcast platform. New data bears on this. According to a Harvard Business School study, about 10% of Twitter users contribute roughly 90% of its content. Anecdotally, these 10% are subject-matter experts, passionates, mavens, and thought leaders who break news, write strong opinions, and tell jokes. Like on Wikipedia, most users merely read this information, and a modest number of people in the long tail use the information in the form of re-tweets, comments, corrections, and alternative opinions or links.

So while an individual user may use Twitter primarily as a conversational tool or a broadcast medium, in its totality, Twitter operates a lot like a wiki: as a knowledge-sharing, co-creation platform that produces content and allows its consumption. Conversation is perhaps the most simple and obvious form of collaboration, but would anyone claim that Wikipedia is a conversational platform? Despite the presence of information sharing, co-creation of an end product, and even discussion pages, Wikipedians on the whole aren't having conversations.

According to this argument, Twitter is no more a conversational platform than Wikipedia is. But is it a social networking platform? New HBS data showing that men have 15% more followers than women and being twice as likely to follow another man than a woman also bear on this to some extent. Authors Bill Heil and Mikolaj Piskorski state: "On a typical online social network, most of the activity is focused around women - men follow content produced by women they do and do not know, and women follow content produced by women they know. Generally, men receive comparatively little attention from other men or from women."

As in the case of the conversational platform, it seems that Twitter is also no more a social network than Wikipedia is. Wikis have user accounts and discussion pages, and it is possible for relationships to form. Twitter has user handles and direct messaging, and relationships can form. But social relationships on Wikipedia and Twitter are not a prerequisite for satisfaction and success (inasmuch as that can be defined). For instance, the popular and useful account @BreakingNews has hundreds of thousands of followers but participants in effectively zero engagement. There are many Twitter users who contribute large amounts of useful information and engage in relatively little conversation. And it is not common for people to describe Wikipedia as a social network.

Andrew McAfee notes that two useful Twitter traits are its asynchronous and asymmetric nature. These two traits are also critical to Wikipedia, but importantly much less so within popular social networking platforms like Facebook and MySpace. Thus, entities that are clearly social networking platforms can be but are not necessarily knowledge co-creation platforms, and entities that are clearly asynchronous knowledge co-creation platforms can be but are not necessarily social networks.

If microsharing tools resemble wikis more than conversational tools and social networks, this has huge implications for how people and organizations approach use of this emerging technology. Solis suggests, I think rightly, that "sometimes it's effective to...maintain a presence simply by reading, listening, and sharing relevant and timely information without having to directly respond to each and every tweet." The strategy of being a "lethally generous" member of a community would seem to be more worthwhile in this context, contrasted with the individual-level customer service approach of (for example) @ComcastCares.

This framework for thinking about microsharing platforms as knowledge co-creation enablers also puts Nielsen's recent data on Twitter's "user retention and loyalty" in a new light. When the average user is a consumer of the content produced by subject-matter experts and passionate mavens, how much does it matter if the majority of use is infrequent spectating (particularly when the information is archived for asynchronous retrieval)? As Shirky recently noted in his talk at the IAC/ACT Management of Change Conference that I attended in Norfolk, VA, such an imbalance of contribution is not a condition of failure for the platform or its users.

Finally, if microsharing is equated with knowledge co-creation, rules for attribution becomes an important consideration. But while the wiki attribution process has generally been worked out, attribution on Twitter is like the wild west - there are no rules; only conventions that are commonly accepted in some circles but not others. In addition, it is relatively easy to cheat the system, hard to catch someone doing it, and difficult to determine what the consequences are of such behavior. This problem will be a lasting one, requiring careful consideration by not only the user community, but also Twitter itself.

Geeks Invade Government With Audacious Goals

by Mark Drapeau | @cheeky_geeky | comments: 16

Guest blogger Mark Drapeau is the Co-Chair of the Gov 2.0 Expo Showcase in Sept 2009 and the Gov 2.0 Expo in May 2010, both in Washington, DC. He holds the title of Associate Research Fellow at the Center for Technology and National Security Policy at the National Defense University, a professional military educational school run by the Joint Chiefs of Staff. Mark is also co-founder of Government 2.0 Club, an international platform for sharing knowledge about the intersection between technology and governance.

When one thinks about important problems facing the United States, and indeed people all over the world, it is difficult to not come up with the laundry list that every talking head seemingly has on the tip of their tongue: jobs, education, health care, national security, poverty. There are so many problems to solve, with so many constraints on spending money, and such a short supply of manhours to get the job done. Many government employees spend a lot of time working on the issue or crisis of the day (or the hour) rather than thinking about long range planning and strategy.

This might be Alexander Hamilton's fault. One of the first things I was indoctrinated with after moving to Washington, D.C. was that the U.S. system of federal government was not designed to make good decisions; rather, it was designed to not make horrible ones. This is counterintuitive, perhaps, but mainly true. And this flies in the face of ideas about using technology to make government more efficient, mainly because the purpose and organization of government is quite different from that of business.

Nevertheless, more and more people from the private sector are interested in playing a role in government, thanks in no small part to the excitement surrounding the Obama election and inauguration, in which social media technologies and information sharing were showcased at their best - massive fundraising from many small donors, empowering people to self-organize locally, and direct public relations that circumvented a mainstream media lens. Now, people enamoured with emergent social technologies want to know how they themselves can revolutionize not only politics, but also governance.

For those who don't follow fashion trends in Washington, D.C., allow me to present the new and increasingly popular species of talking head - The Geek. (The Geek is distinguished from The Wonk, studious, preppy, bespectacled types that run Washington policy, know exactly what intersection Brooks Brothers is on, and enjoy cocktail parties for "networking," and The Nerd, the type of scientist or other fastidious pointy-head rarely seen outside a laboratory or professorial tower, with nary an interest outside their own peculiar and narrow slice of life.)

The prototypical Geek is a different breed of talking head, one that usually lacks media training, one that often hails from Silicon Valley, San Francisco, Seattle, New York, Austin, Boston, St. Paul, or Boulder, one who likely knows more about the inside of a computer than the average person does about the inside of their fridge, a well-read introvert shy in real life but outgoing on Twitter and in the blogosphere, who is erudite enough to have always felt there was a better way to run the government but feeling entirely disconnected from the apparatus.

No longer. When I speak about Government 2.0 to audiences around D.C. I am fond of telling them about the very smart and motivated outsiders (i.e., The Geeks) who think that they can run the government better than the government can. I enjoying dropping the line, "The government can no longer afford to work at the pace of government," because people never really know what to say in response as they mull it over. That statement is somewhat tongue-in-cheek and not entirely fair to hundreds of thousands of hard-working government employees; but of course, my role as a speaker is usually to provoke thought and get a point across, not to be fair. And gradually, through my efforts and those of many other Government 2.0 enthusiasts, people inside the Beltway are understanding that new ideas and new technologies can bridge gaps between government and the citizens (and that outsiders are starting to utilize such technologies whether the government gives permission or not).

(continue reading)

Tweenbots: Cute Beats Smart

by Brady Forrest | @brady | comments: 17

If you wanted to build a robot that could go from one end of Washington Square Park to the other without your help how would you do it? How expensive in time and money would it be? Would you build or buy a navigation system? Construct a sensing system to detect obstacles? Or would you decide to take a different tact and use cute as your primary tool?

ITP student Kacie Kinzer created a 10-inch smiling robot called a Tweenbot that can only go straight. For each journey Kacie would give the robot a destination and clearly label it. Given the obstacles in its way and lack of navigation or steering systems the expectation was that the robot would not make it. However the robot's avoidance of the uncanny valley and clearly written goal helped it out. Humans would redirect the Tweenbot so it successfully reached its destination. Below is a map of one Tweenbot journey:

Mission 1: Get from the Northwest to the Southwest Corner of Washington Square Park / time: 42 minutes / number of people who intervened: 29

As Kacie describes on the site:

Over the course of the following months, throughout numerous missions, the Tweenbots were successful in rolling from their start point to their far-away destination assisted only by strangers. Every time the robot got caught under a park bench, ground futilely against a curb, or became trapped in a pothole, some passerby would always rescue it and send it toward its goal. Never once was a Tweenbot lost or damaged. Often, people would ignore the instructions to aim the Tweenbot in the “right” direction, if that direction meant sending the robot into a perilous situation. One man turned the robot back in the direction from which it had just come, saying out loud to the Tweenbot, "You can’t go that way, it’s toward the road.”

So why do people help out the tweenbot? Personally I would not be able to resist assisting the anthropomorphized little robot. The smile signals its innocent intentions and the Tweenbot's label makes it clear how to help. It's something for designers and technologists to remember; sometimes cute and clever can get the job done much cheaper and in less time than smart and expensive.

There are more Tweenbots coming so if you happen to see any friendly robots around your town lend a hand. Here are some of the prototypes that are currently in development.

via Hacker News

Startup Marketing Isn't Rocket Science, So Don't Hire the Ph.D Too Soon

by Darren Barefoot | @dbarefoot | comments: 7

Guest blogger Darren Barefoot is a writer, marketer, technologist, and co-founder of Capulet Communications, a web marketing firm that specializes in high-tech and sustainability clients. He is the co-author of a forthcoming book about social media marketing for No Starch Press entitled "Friends with Benefits." Darren's personal blog is DarrenBarefoot.com.

.

A couple of weeks ago, my partner and I met with a potential client, the newly-hired Vice President of Marketing at a technology startup. She was a heavyweight, having spent the last decade in charge of marketing at a major Canadian corporation. She'd been craving a change of pace, and so accepted an offer to oversee the marketing at this eight-person startup. She'd only been with the company for a couple of weeks, so I asked about her immediate plans. She replied, "I'm going to start by putting some policies and procedures in place--we could really use more structure."

Her response highlighted one of the most common mistakes we encounter when working with early-stage startups: the founders hire too much marketing talent too early.

Why does this happen? I'm not sure, but I wonder if it's because many founders have a technical background. As such, they're unfamiliar and sometimes a little intimidated by the challenges of promoting their startup. To assuage their concerns, they bring in a senior marketer with plenty of credentials.

In theory, this looks like a rational decision. After all, the more experienced the executive, the better. Practically speaking, things aren't quite that simple.

These types are usually great strategists. That's what got them high on the ladder at their previous corporate or agency job. They're accustomed to devising a strategic plan and overseeing a team that implements that plan. That's fine, because, a startup obviously needs strategy.

By their nature, though, they're thinkers, not doers. They're great at wrestling with thorny messaging problems or rejigging the corporate branding, but how often have they run a CPC advertising campaign, or how good are they at grokking Google Analytics?

The work they're accustomed to delegating or outsourcing is now work that they must do themselves. They have some 'doing' skills, but they're often atrophied or antiquated. Your average startup doesn't have a graphic designer or a copywriter or an SEO expert on staff. The VP of marketing at a bootstrapped startup must be all of these people, and we find that the average corporate escapee can't fill those roles.

What's the answer? A startup still needs strategic thinking. It just doesn't need it 40 hours a week with medical benefits and a corner office. Instead, founders should contract a senior marketing consultant with expertise in their field for a fixed number of hours a month. Have them weigh in on the hard decisions, draft launch plans and so forth.

Then, saving on the cost of an executive's salary, contract out the tactical marketing or employ a couple junior staffers to do the legwork. Marketing isn't rocket science, after all, so hold off on hiring that marketing Ph.D. until you're certain that you need them.

ETech Preview: On The Front Lines of the Next Pandemic

by James Turner | comments: 0

With all of the stress and anxiety that humanity deals with on a daily basis--confronting the dangers of global warming, the perils of a financial system in meltdown and the ever-present threat of terrorism--the fact that there's yet another danger lurking out there ready to destroy mankind: the threat of a global pandemic, may be easy to forget. But although you and I may have driven thoughts of Ebola and the like from our minds, Dr. Nathan Wolfe worries about them every day. Dr. Wolfe founded and directs the Global Viral Forecasting Initiative which monitors the transfer of new diseases from animals to humans.

He received his Bachelor's degree at Stanford in 1993 and his Doctorate in Immunology and Infectious Diseases from Harvard in 1998. Dr. Wolfe was awarded the National Institute of Health International Research Scientist Development Award in 1999 and a prestigious NIH Directors Pioneer Award in 2005. He'll be speaking at the O'Reilly Emerging Technology Conference in March. His session is entitled, "Viral Forecasting." Thank you for joining us.

Dr. Nathan Wolfe: My pleasure.

James Turner: So why don't we start by talking about the Global Viral Forecasting Initiative. How is it different from the work that the CDC and the WHO and similar organizations do monitoring disease spread?

NW: Well, what we do is we actually focus on the interface between humans and animal populations. When we looked back and investigated the ways in which disease got started, the ways that pandemics really originated, what we found is that really the vast majority of these things are animal diseases. So rather than monitoring for illness, at which point it could potentially be too late, we've taken it one step backward. We actually focus on people who have high levels of contact with animals. And we set up large groups of these individuals and monitor the diseases that they have, as well as the diseases in the animal population. So the idea is to be able to catch these things a little bit earlier.

JT: The last disease that really made a big splash with the media was Ebola, earlier in this decade. But we really haven't heard much recently. Have things calmed down as far as new and novel diseases? Or are we just hearing less about outbreaks these days?

NW: Well, I mean I think we've had really substantive important pandemics. If you take a look at SARS, for example. SARS really only infected about 1,200 individuals, but its impact was tremendous. It was billions of dollars of economic impact all throughout the world. Even in a place like Singapore, where you had a small number of cases, you had an incredibly substantive financial impact. And then, of course, right now we have H5N1 which is -- they call it the bird flu. Actually, most influenzas are bird influenzas, so it's a little bit of a misnomer. But H5N1 is a virus which is spreading around the world in birds. And if it does make a transition into humans, which some bird flu will over the next 20 to 30 years, it could be incredibly devastating. So I think that these are kind of constant and present dangers. They're things that are increasing over time simply because of the way that we're connected as a human population.

(continue reading)

ETech Preview: Creating Biological Legos

by James Turner | comments: 22

If you've gotten tired of hacking firewalls or cloud computing, maybe it's time to try your hand with DNA. That's what Reshma Shetty is doing with her Doctorate in Biological Engineering from MIT. Apart from her crowning achievement of getting bacteria to smell like mint and bananas, she's also active in the developing field of synthetic biology and has recently helped found a company called Gingko BioWorks which is developing enabling technologies to allow for rapid prototyping of biological systems. She will be giving a talk entitled Real Hackers Program DNA at O'Reilly's Emerging Technology Conference, March 9-12, in San Jose, California. And she's joining us here today. Thank you for taking the time.

RESHMA SHETTY: No problem. Happy to be here.

JAMES TURNER: So first of all, how do you make bacteria smell nice, and why? I get an image of a commercial, "Mary may have necrotizing fasciitis, but at least her hospital room smells minty fresh."

RS: Well, the original inspiration for the project was the fact that for anybody who works in a lab, who works with E. coli, when you grow cultures of the stuff, it just smells really bad. It smells really stinky, basically. And so our thought was, "Hey, why don't we reengineer the smell of E. coli? It'll make the lab smell minty fresh, and it's also a fun project that gets people, who maybe aren't normally excited about biology, interested in it because it's a very tangible thing. I can smell the change I made to this bacteria."

JT: So what was the actual process involved?

RS: So the process was, you basically take a gene, we took a gene from the petunia plant, which normally provides an odor to the flower, and you place that gene into the E. coli cell. And by supplying the cell with an appropriate precursor, you make this minty smell as a result. So it's fairly straightforward.

JT: Your degree, biological engineering, is a new one to me. How is it different from biochemistry or microbiology or genomics or any of the other traditional biotech degrees?

RS: Well, biology and biochemistry, and so on, are concerned with studying the natural world. So I'm going to go out and figure out how the natural world works. Biological engineering, instead, is really all about saying, "Hey, we have this natural world around us. Biology is, in some sense, a new technology through which we can build new engineered biological systems." Right? So the idea is, what's the difference between physics and electrical engineering? Electrical engineers want to go build. So in biological engineering, we're interested in going and building stuff, too. But using biology, rather than physics, as the underlying science of it.

(continue reading)

ETech Preview: Why LCD is the Cool New Technology All Over Again

by James Turner | comments: 7

In an early test of the OLPC XO in Nigeria, the student users dropped every laptop several times a day. Despite the laptops' rugged construction, they occasionally needed fixing, and a group of six-year-old girls opened up a "hospital" to reseat cables and do other simpler repairs. Mary Lou Jepson, One Laptop Per Child project's CTO, had this response: "I put extra screws underneath the battery cover so that if they lost one, they could have an extra one. And kids trade them almost like marbles, when they want to try to get something fixed in their laptop."

Mary Lou led the development of the OLPC's breakthrough low-power transflective display, combining a traditional backlit color display with a black and white display that could be used outdoors. She left OLPC to form Pixel Qi, and bring the revolutionary engineering used in the XO to the broader consumer market. In this interview, she discusses lessons learned from OLPC and shares her vision of "cool screens that can ship in high volume, really quickly, at price points that are equivalent to what you pay for standard liquid crystal displays."

At ETech, Mary Lou's keynote presentation delves further into Low-Cost, Low-Power Computing.

JAMES TURNER: I'm speaking today with Mary Lou Jepsen, Founder and CEO of Pixel Qi. Dr. Jepsen previously served as chief technology officer for the One Laptop per Child program where she was an instrumental player in the development of the OLPC's revolutionary hybrid screen. She also previously served as CTO of Intel's display division. Dr. Jepsen was also named by Time Magazine recently as one of the 100 most influential people in the world for 2008. She'll be speaking at the O'Reilly Emerging Technologies Conference in March, and we're pleased she's taken the time to talk to us. Good evening.

MARY LOU JEPSEN: Hi. Nice to speak with you tonight.

JT: So in some ways, you're kind of uniquely qualified to comment on the current travails of the OLPC since you've been in highly influential positions both in the OLPC effort itself and at Intel, who some believe tried to sabotage the OLPC. Do you think that the OLPC would've had wider acceptance if the Intel Classmate wasn't competing against it?

MLJ: It is interesting. I think the OLPC, and I haven't seen the latest numbers, sold a lot more than the Classmate. I think head-to-head there's no comparison which is the better machine, and I'm not saying that just because I'm the architect. But what's really happened has been extraordinary. I think OLPC's impact in sort of spearheading the movement to Netbooks is fairly undisputed, although OLPC is not the best selling Netbook; 17 million Netbooks shipped in 2008 and that's through companies like Acer, Asus, MSI, HP, Dell. And that impact on the world is starting to be felt.

JT: What were the factors that led you to leave the OLPC program and start Pixel Qi?

MLJ: You know, I started OLPC with Nicholas in his office in the beginning, in January of 2005. And at that point, right after that Bill Gates, Steve Jobs, Michael Dell, all said it was impossible. So it became my job to sort of take that, create an architecture, invent a few things, convince the manufacturers to work with me to develop it, get a team together, and take it into high-volume mass production. And then it got to the point where my days were spent getting safety certifications for various countries.

And I just realized, it's time for me to continue doing this; this is the best job I've ever done, but to keep going, why not make these components that are inside of the XO and let everybody buy them rather than just exclusively making and designing them for the OLPC laptop. If you make more of something, you can sell it for less. So rather than just serving the bottom of the pyramid, why not take the fantastic technology that we developed at OLPC and serve the whole pyramid? Everybody wants their batteries to last a lot longer. Everybody wants screens that are e-paper-like and high resolution and sunlight readable. So why not make these for the whole world?

(continue reading)

The Kindle and the End of the End of History

by Jim Stogdill | @jstogdill | comments: 24

This morning I was absentmindedly checking out the New York Times' bits blog coverage of the Kindle 2 launch and saw this:

“Our vision is every book, ever printed, in any language, all available in less than 60 seconds.”

It wasn't the main story for sure. It was buried in the piece like an afterthought, but it was the big news to me. It certainly falls into the category of big hairy audacious goal, and I think it's a lot more interesting than the device Bezos was there to launch (which still can't flatten a colorful maple leaf). I mean, he didn't say "every book in our inventory" or "every book in the catalogues of the major publishers that we work with." Or even, "every book that has already been digitized." He said "every book ever printed."

When I'm working I tend to write random notes to myself on 3x5 cards. Sometimes they get transcribed into Evernote, but all too often they just end up in piles. I read that quote and immediately started digging into the closest pile looking for a card I had just scribbled about an hour earlier.

I had been doing some research this morning and was reading a book published in 1915. It's long out of print, and may have only had one printing, but I know from contemporary news clippings found tucked in its pages that the author had been well known and somewhat controversial back in his day. Yet, Google had barely a hint that he ever existed. I fared even worse looking for other people referenced in the text. Frustrated, I grabbed a 3x5 card and scribbled:

"Google and the end of history... History is no longer a continuum. The pre-digital past doesn't exist, at least not unless I walk away from this computer, get all old school, and find an actual library."

My house is filled with books, it's ridiculous really. They are piled up everywhere. I buy a lot of old used books because I like to see how people lived and how they thought in other eras, and I guess I figure someday I'll find time to read them all. For me, it's often less about the facts they contain and more about peeking into alternative world views. Which is how I originally came upon the book I mentioned a moment ago.

The problem is that old books reference people and other stuff that a contemporary reader would have known immediately, but that are a mystery to me today - a mystery that needs solving if I want to understand what the author is trying to say, and to get that sense of how they saw the world. If you want to see what I mean, try reading Winston Churchill's Second World War series.

Churchill speaks conversationally about people, events, and publications that a London resident in 1950 would have been familiar with. However, without a ready reference to all that minutiae you'll have no idea what he's talking about. Unfortunately, a lot of the stuff he references is really obscure today and today's search engines are hit and miss with it - they only know what a modern wikipedia editor or some other recent writer thinks is relevant today. Google is brilliant for things that have been invented or written about in the digital age, or that made enough of a splash in their day to still get digital now, but the rest of it just doesn't exist. It's B.G. (before Google) or P.D. (pre digital) or something like that.

To cut to the chase, if you read old books you get a sense for how thin the searchable veneer of the web is on our world. The web's view of our world is temporally compressed, biased toward the recent, and even when it does look back through time to events memorable enough to have been digitally remembered, it sees them through our digital-age lens. They are being digitally remembered with our world view overlaid on top.

I posted some of these thoughts to the Radar backchannel list and Nat responded with his usual insight. He pointed out that cultural artifacts have always been divided into popular culture (on the tips of our tongues), cached culture (readily available in an encyclopedia or at the local library) and archived culture (gotta put on your researcher hat and dig, but you can find it in a research library somewhere). The implication is that it's no worse now because of the web.

I like that trichotomy, and of course Nat's right. It's not like the web is burying the archive any deeper. It's right there in the research library where it has always been. Besides, history never really operates as a continuum anyway. It's always been lumpy for a bunch of reasons. But as habit and convenience make us more and more reliant on the web, the off-the-web archive doesn't just seem hard to find, it becomes effectively invisible. In the A.G. era, the deep archive is looking more and more like those charts used by early explorers, with whole blank regions labeled "there be dragons".

So, back to Bezo's big goal... I'd love it to come true, because a comprehensive archive that is accessible in 60 seconds is an archive that is still part of history.

A Climate of Polarization

by Gavin Starks | comments: 10

Guest blogger Gavin Starks is founder and CEO of AMEE, a neutral aggregation platform designed to measure and track all the energy data in the world. Gavin has a background in Astrophysics and over 15 years Internet development experience.

We're all aware of the emotive language used to polarize the climate change debate.

There are, however, deeper patterns which are repeated across science as it interfaces with politics and media. These patterns have always bothered me, but they've never been as "important" as now.

We are entering an new era of seismic change in policy, business, society, technology, finance and our environment, on a scale and speed substantially greater than previous revolutions. The sheer complexity of these interweaving systems is staggering.

Much of this change is being driven by "climate science", and in the communications maelstrom there is a real risk that we further alienate "science" across the board.

We need more scientists with good media training (and presenting capability) to change the way that all sciences are represented and perceived. We need more journalists with deeper science training - and the time and space to actually communicate across all media. We need to present uncertainty clearly, confidently and in a way that doesn't impede our decision-making.

On the climate issue, there are some impossible levers to contend with;

1. Introducing any doubt into the climate debate stops any action that might combat our human impact.
2. Introducing "certainty" undermines our scientific method and its philosophy.

When represented in political, public and media spaces, these two levers undermine every scientific debate and lead to bad decisions.

Pascal's Wager is often invoked, and this is entirely reasonable in this case.

It is reasonable because of what's at stake: the risk of mass extinction events. If there is a probability that anthropogenic climate change will cause the predicted massive interventions in our ecosystem, then we have to act.

The nature of our actions must be commensurate with both the cause and the effect. The causes are many: population, production, consumption - as are the effects: war, poverty, scarcity, etc.

Our interventions will use all our means to address both cause and effect, and those actions will run deep.

Equally, we must allow science to do what it's designed to do: measure, model, analyse and predict.

From a scientific perspective we must allow more room for theories to evolve, otherwise we'll only prove what we're looking for.

However, if we ignore the potential need to act, the consequences are not something anyone will want to see.

It's not something we can fix later (for me, "geo-engineering" is not a fix, it's a pre-infected band-aid).

Given the massive complexity of the issues, and that - really - anthropogenic climate change is only one of many "peak consumption" issues that we face, there is no way we can accurately communicate all the arguments that would lead to mass understanding.

However, the complexity issues are no different from those we face in politics. They are not solvable, but they are addressable.

We can communicate the potential outcomes, and the decisions that individuals need to make in order to impact the causes.

Ultimately it's your personal choice.

My choice is based on my personal exposure to the science, business, data, policy, media, and broader issues around sustainability. That choice is to do my best to catalyse change as fast as I possibly can.

We all need to actively engage in improving communication, so that everyone - potentially everyone on Earth - can make informed choices about the future of the planet we inhabit.

--
Recommended reading:

http://www.realclimate.org/ is a great resource.

Today, the UK Government launched a campaign "to create a more science literate society, highlighting the science and technology based industries of the future"

What Will Change Everything?

by Brady Forrest | @brady | comments: 16

Regular Radar contributor Linda Stone sent this in to be posted today.

What game-changing scientific ideas and developments do you expect to live to see? The Internet, television, antibiotics, automobiles, electricity, nuclear power, space travel, and cloning - these inventions were born out of dreams, persistence, and imagination. What game-changing ideas can we expect to see in OUR lifetimes?

As each year winds to a close, John Brockman, a literary agent representing some of the finest minds in science and technology and the founder of Edge (a 501c3), poses a provocative question to an international community of physicists, psychologists, futurists, thought leaders and dreamers. Brockman is a master convener, both online and in real life. This year’s annual Edge question, What will change everything?, generated responses from Freeman Dyson, Danny Hillis, Martin Seligman, Craig Venter, and Juan Enriquez, to name a few. Here are a few highlights.

Venter imagines creating life from synthetic materials and expects that our view of life, itself, will be transformed.

Nobel Laureate, Frank Wilczek, believes everything will continue to become smaller, faster, cooler, and cheaper -- with its implications of an Internet on steroids and exciting new designer materials.

Several neuroscientists wrote about everything from direct communication of feelings and thoughts from brain to brain to electrical brain stimulation for the treatment of mood disorders to cheap cryonic suspension of brains, to ways to control brain plasticity. Mihaly Csikszentmihalyi and Verena Huber-Dyson, see science evolving beyond analytical focus and including a sense of synthesis. Huber-Dyson envisions the end of fragmentation of knowledge.

George Dyson, science historian, author, kayak-designer and builder, looks toward the stars - or here on earth, suggesting, “the detection of extraterrestrial life, extraterrestrial intelligence or extraterrestrial technology will change everything.”

Discover Magazine editor in chief, Corey S. Powell, offers a list of possibilities from synthetic telepathy to genetically engineered kids.

Eleanor Roosevelt once said, “Great minds discuss ideas; Average minds discuss events; Small minds discuss people.”

Tap your greatness as we welcome 2009!

Enjoy reading responses to the Edge annual question and please comment on the question yourself: What will change everything? What game-changing scientific ideas and developments do you expect to live to see?

The State of Transit Routing

by Jim Stogdill | @jstogdill | comments: 12

My brother called me a week ago and during the course of our conversation mentioned that he made the trek to the Miami Auto Show. He was complaining that he really wanted to take Tri-Rail (the commuter rail that runs along Florida's South East coast) but it was just too hard to figure out the rest of the trip once he got off the train. "One web site for train schedules, another for buses, and another for a city map to tie it all together. It was just too much trouble to figure out, so I drove. I just want to go online and get directions just like I do for driving, but that tells me which train, which bus, etc."

Coincidentally, later in the day I downloaded the iPhone 2.2 upgrade with the new walking and public transit directions. So far, at least where I live, it's useless. The little bus icon just sits there grayed out, taunting me. I guess because SEPTA (our local transit authority for bus and regional rail) isn't giving data to Google?

My brother hadn't heard of Google Transit, but It turns out to have some coverage in Miami. Their coverage at this point seems to be transit authority centric and doesn't seem to have great support for mixed mode or stuff that crosses transit system boundaries. I am curious though, is it being used? Let me know in the comments if you are using it to good effect.

Anyway, my brother's call on the same day as the iPhone update piqued my interest in the current state of the art for mixed mode transit routing. After some mostly fruitless web searches for I reached out to Andrew Turner. I knew he'd know what was going on in. This is what he had to say:

Routing is definitely one of the emergent areas of technology in the next generation applications. So far, we've done a great job getting digital maps on the web, mobile locative devices, and comfortable users.

One problem for awhile has been the lack of data. You can have a great algorithm or concept, but without data it's useless. Gathering this data has been prohibitively expensive - companies like NAVTEQ drive many of the roads they map for verification and additional data. Therefore if you wanted to buy road data from one of the vendors you had to have a large sum of money in the bank and know how you were going to monetize it. This stifled experimentation and creating niche applications.

Now that the data is becoming widely, and often freely, available innovation is happening at an increased pace.

For one example, consider the typical road navigation. The global OpenStreetMap project has always had topology (road connectivity), but the community now adding attribute data to ways such as number of lanes, stop lights, turn restrictions, speeds, and directionality. Anyone can download this data to use with a variety of tools such as pgRouting. As a result people are rethinking standard routing mechanisms that assume travel from A to B via the fastest, or shortest, route. What if a user wants to take the "greenest" route as determined by lowest total gas mileage, or the most scenic route based on community feedback.

An area that has strongly utilized this idea has been disaster response. Agencies and organizations deploy to areas with little on the ground data, or data that is now obsolete due to the disaster they're responding. Destroyed bridges, flooded roads, new and temporary infrastructure are just some of the aspects that are lost with typical navigation systems. However, given the capability of responders to correct the data and instantly get new routes is vital. And these routes may need to be based on attributes different from typical engines - it's not about the fastest, but which roads will handle a 5-ton water truck?

This scheme was deployed in the recent hurricane response in Haiti in conjunction with the UNJLC, CartOng, OpenStreetMap and OpenRouteService.

Beyond just simple, automotive routing, we can now incorporate multi-modal transit. With 50% of the world's population now living in urban areas, the assumption that everyone is in a car is not valid. Instead people will be utilizing a mixture of cars, buses, subways, walking, and bicycling. This data is also being added to OpenStreetMap as well as other projects such as Bikely or EveryTrail. GraphServer is one routing engine that will incorporate these various modes and provide routes.

And we're interfacing with all these engines using a variety of devices: laptop, PND (Personal Navigation Device), GPS units, mobile phones, and waymarking signs. PointAbout recently won an award in the Apps For Democracy for their DC Location Aware Realtime Alerts mobile application that displays the route to the nearest arriving metro.

What's also interesting is the potential of these routing tools beyond actual specific individual routes. Taken in amalgamation the routing distances form a new topography of the space. Given a point in the city, how far can I travel in 20 minutes? in 40 minutes? for less than $1.75? This type of map is known as an isochrone. Tom Carden and MySociety developed London Travel Time Maps that allow users to highlight the spots in London given a range of house prices and travel times.

Despite these apparent benefits, there is a large hurdle. Like road data, there has been a lack of openly available transit data to power applications and services. Providers like NAVTEQ and open projects like OpenStreetMap are possible because the public roads are observable and measurable by any one. By contrast, the many, varied local transit agencies own and protect their routing data and are reluctant to share. Google Transit has made great strides in working with transit authorities to expose their information in the Google Transit Feed Specification - at least to Google. This does not mean the data has to be publicly shared, and in many cases this is exactly what has occured.

However, not even the allure of widely admired Google Transit can induce transit authorities to share their prized data. The Director of Customer Service of the Washington Metro Area Transit Authority (WMATA) plainly states that working with Google is "not in our best interest from a business perspective."

Hopefully, this situation will change, first through forceful FOIA requests, but later through cooperation. One step in this direction have been TransitCamps. And Portland's TriMet is a shining example with a Developer Resources page detailing data feeds and API's.

These experimentations are just the beginning of what is being pushed in the space. Routing is one of those features that users may not realize they need until they have it and then they'll find it indispensable. The ability for a person to customize their valuation of distance to assist in making complex decisions and searching is very powerful.

For more projects and tools, check out the OpenStreetMap routing page, Ideas in transit and the OGC's OpenLS standards.

DIY Appliances on the Web?

by Jim Stogdill | @jstogdill | comments: 9

Or, My Enterprise is Appliancized, Why Isn't Your Web?

I wrote a couple of posts a while back that covered task-optimized hardware. This one was about a system that combined Field Programmable Gate Arrays (FPGA's) with a commodity CPU platform to provide the sheer number crunching performance needed to break GSM encryption. This one looked at using task-appropriate efficient processors to reduce power consumption in a weather predicting super computer. In these two posts I sort of accidentally highlighted two of the three key selling points of task-specific appliances, sheer performance and energy efficiency (the third is security). The posts also heightened my awareness of the possibilities for specialized hardware and some of my more recent explorations that focused on the appliance market in particular got me wondering if there might be a growing trend toward specialized appliances.

Of course, specialized devices have been working their way into the enterprise ever since the first router left its commodity Unix host for the task-specific richness of specialized hardware. Load balancers followed soon after and then devices from companies like Layer 7 and Data Power (now IBM) took the next logical step and pushed the appliance up the stack to XML processing. These appliances aren't just conveniently packaging intellectual property inside commodity 1U blister packs, they are specialized devices that process XML on purpose-built Application Specific Integrated Circuits (ASICS), accelerate encryption / decryption in hardware, and encapsulate most of an ESB inside a single tamper proof box whose entire OS is in firmware. They are fast, use a lot less power than an equivalent set of commodity boxes, and are secure.

Specialization is also showing up in the realm of the commodity database management systems. At last year's Money:Tech Michael Stonebraker described a column-oriented database designed to speed access to pricing history for back testing and other financial applications. In this case the database is still implemented on commodity hardware. However, I think it's interesting in the context of this conversation on specialized computing because it speaks to the inadequacy of commodity solutions for highly specific requirements.

A device from Netezza is also targeted at the shortcomings of the commodity DBMS. In this case the focus is on data warehousing, but it takes the concept further with an aggressive hardware design that is delivered as an appliance. It has PostgreSQL at its core but it takes the rather radical step of coupling FPGA's directly to the storage devices. The result, for at least a certain class of query, is a multiple order of magnitude boost in performance. I think this device is noteworthy because it puts the appliance way up the stack and is perhaps a harbinger for further penetration of the appliance into application-layer territory.

While appliances are expanding their footprint in the enterprise, it seems like the exact opposite might be happening on the web? Maybe the idea of a closed appliance is anathema to the open source zeitgeist of the web, but in any case, the LAMP stack is still king. Even traditional appliance-like tasks such as load balancing seem to be trending toward open source software on commodity hardware (e.g. Perlbal).

I can't help but wonder though, at the sheer scale that some web properties operate (and at the scale of the energy cost required to power them), can the performance and cost efficiency of specialized hardware appliances be ignored? Might there be a way to get the benefits of the appliance that is in keeping with the open source ethos of the web?

If you've ever uploaded a video to Youtube and waited for it to be processed you have an idea of how processor hungry video processing is on commodity hardware. I don't know what Google's hardware and energy costs are for that task but they must be significant. Same goes for Flickr's image processing server farm and I would guess for Google's voice processing now that its new speech services have launched. If the combination hardware and electricity costs are high enough, maybe this is a good place to introduce specialized appliances to the web?

But how to do that in a way that is consistent with the prevailing open source ethos and that still lets a firm continue to innovate? I think an answer might be sort of DIY writ large; a confluence of open source and open hardware that works like an undocumented joint venture based on the right to fork. Think Yahoo and the Hadoop community or JP Morgan and friends with AMQP but with hardware and you get the idea. Such a community could collaborate on the design of the ASICS and the appliance(s) that hosted them and even coordinate production runs in order to manage unit costs. Perhaps more importantly, specifying the components openly would serve cost sharing across these companies while still supporting flexibility in how they were deployed and ultimately, generativity and innovation for future uses.

There are probably a bunch of reasons why this is just silly speculation, but Google's efforts with power supply efficiency might be seen as at least a bit of precedent for web firms dabbling in hardware and hardware specifications. In fact, Google's entire stack, from it's unique approach to commodity hardware, to software infrastructure like GFS, might be thought of as a specialized appliance that suits the specific needs of search. It's just a really really big one that "ships" in a hundred thousand square foot data center form factor.

Apps for Democracy

by Jim Stogdill | @jstogdill | comments: 3

Vivek Kundra, the District of Columbia's CTO, isn't just talking about transparent government and participative democracy, he's working hard to make DC's massive data stores transparent and inviting your participation.

I first heard about Vivek's push for transparency when he spoke at an Intelligence Community Conference in September (I just happened to be speaking on a panel thanks to a twitter-induced serendipitous introduction to one of the conference organizers - @immunity). He was there in sort of "one government entity to another" role to demonstrate that data could be shared and that it is valuable to do it.

I was impressed with the risks he was taking to push hard for the "democratization of data" and for what he was calling The Digital Public Square. What came through really clearly was that he didn't just view this as a technology exercise, but as a way for citizens to participate in real ways and to increase government accountability. It was an engaging and refreshing talk.

It's not exactly news at this point that he came up with $20,000 to offer prizes for the best applications to be built on top of the district's data. After all, the submissions have been long closed and the winners have already been announced. However, I thought it might be worth pointing out that you still have until tomorrow to vote for the two people's choice award winners.

I thought it was kind of fun to just poke around in the list of submissions and see what people came up with. As you can imagine many of them are geo-spatial display's of district stores, but there are some other ideas in there as well. Take a look, see what you think, and get your people's choice vote in.

And just because the contest is over doesn't mean it's too late to build something. Take a look at the catalog of data and see what comes to mind. This is just the beginning (Mayor Nutter of Philly, I'm looking at you...).

My Apple Holiday Wish

by Jim Stogdill | @jstogdill | comments: 20

I've been searching for a personal backup solution that doesn't suck for, well, pretty much since I got my first computer in the 80's, and I'm still looking.

A few years ago I was cleaning out old crap and ran across boxes and boxes of 800kb floppies labeled "1988 backup disk x." The trash / recycling picker uppers got those along with a pile of zip disks, various CD's, DVD's, a USB drive or two, and a couple of bare SATA drives that I was too cheap to buy housings for. Oh, and there was even a pile of tapes in some long forgotten format in there.

After a few years of manually copying stuff to multiple USB drives, last year I was completely seduced by the "it's like RAID but you don't need identical drives" beauty of the Drobo. Three failures later (including one with smoke), a nasty virtual tinnitus that comes and goes as its disks transition through a perfect cabinet-resonating frequency, incompatibility problems with Time Machine and Airport Extreme, and access speeds that are too slow to serve Final Cut, and screw it. Now it mostly just sits there powered down making a Drobo-shaped dust-free spot on my desk. It's too buzzy to listen to but too expensive to Freecycle.

Next up, Time Capsule. Still (even more) useless for Final Cut and that sort of thing, but it's doing an ok job with backups - at least of the straight Time Machine variety. There are still a few issues though...

First off, I don't really trust that single spinning platter. It will die some day. Plus, it's in my house about ten feet from where my laptop is usually parked so my eggs are all in a single fire / theft / flood basket.

Apple's Mobile Me and the Backup program that comes with it theoretically provide a solution to this issue, but unfortunately it sucks. It's slow, much slower than a local time capsule backup because it is relying on an Internet connection. Also, it effectively requires my machine to be running all the time so that it can conduct it's backups in the middle of the night when I won't be competing for bandwidth or CPU cycles.

Even worse, it fails all the time. I don't know why, but it's finicky. A brief connectivity hiccup (or whatever) and I wake up the next day to find that my multi-hour backup died. Finally, It's too small to be useful for more than a few key critical files. I have a few hundred gigabytes of data I'd like to secure and my mobile me account is limited to twenty.

So Apple, I don't usually resort to begging, but here's your chance to fix backup for me once and for all. Just update the firmware in my Time Capsule so that my fast Wi-Fi-based local backups can be incrementally streamed to either an expanded Mobile Me account or to a separate S3 account (or whatever) whenever it's sitting at home with my network connection to itself.

I can't leave my laptop connected for the days it would take to stream all those hundreds of Gig, but Time Capsule is just sitting there with my Internet connection doing nothing while I'm at work anyway, so give it something to do. This way I'll have the best of both worlds, fast reasonably secure backups to my local Wi-Fi connected Time Capsule when I'm home and don't-need-to-think-about-it remote storage that can take its time when I'm not. At the risk of way over reaching, it could even work in both directions so that if I'm on the road for an extended period, Time Machine could backup critical changes directly to Mobile Me which could then in turn incrementally stream that back to my Time Capsule.

Ok, that's it. A simple idea I think. Can I have it by Christmas?

By the way, if the thought of all those gigabytes in your Mobile Me data centers makes you blanche (and the idea of using S3 is anathema to Apple's do it all culture), how about a Time Capsule-based distributed hash overlay network? If every Time Capsule shipped with the option of turning on a separate partition representing about 1/3 of the disk, you could put a Planet Lab-like distributed file system in there. My files would be split into chunks, encrypted, and distributed around to other people's Time Capsules while some of their stuff was on mine. Sort of an inverted Bit Torrent for backups, no data center required.

That would be cool but I know you won't do it. And, from the category of "things you are even less likely to do," if you opened up the Time Capsule firmware to third parties someone else probably would.

PICNIC Network 2008

by Brady Forrest | @brady | comments: 4

The week of September 22nd I am going to be flying across the Atlantic for the third PICNIC Network. Ever since i heard about the conference last fall I've wanted to attend. My friends' stories last year focused on the many RFID-enabled art pieces. As discussed in this interview these were developed by Mediamatic, a digital art lab.

Last year Mediamatic offers PICNIC delegates some fun and intriguing services. An RFID-tag was added to the PICNIC badges, and linked to the delegates profile on the PICNIC network. A team of top notch hackers, developers and dreamers got involved and came up with cute, fun and relevant new services. Remind us!.

"Last year we had the first version of the hackers camp where we built some cool physical interfaces for the PICNIC social network. We had the Photobooth, Badger, The Friend-Drink station, I-tea and of course a whole range of inspiring designs that we only could realise later."

Mediamatic is going to be hosting another hacking workshop and I'll be volunteering. Additionally there are a number of sessions and day long minitracks:

Surprising Africa has a great selection of content. I've never been to Africa, but there is an increasing amount of tech heading there. Google, Nokia and Vodafone are going to be describing their initiatives. Ethan Zuckerman will be discussing citizen journalism in Africa; Ethan gave an amazing talk at last year's ETech on this topic (Radar post).

Visible City is exploring the data available from cities. As their page states "What if an entire city could be visible from above, like we see it from an airplane? Not simply buildings and squares, but also the aggregation of people who populate it, outdoor as well as indoor. We could detect public gatherings and traffic jams, estimate which neighborhoods are most crowded, reconstruct commuting patterns during the day." This is very relevant for this year's ETech.

In the Internet of Things speakers from SAP, OpenSpime, and ThingM (amongst others) will be discussing how they are moving devices and objects online. This session makes me think of the soon-to-be released Fitbit, a personal web-enabled activity tracker. Soon it won't be just things online, but our passive data as well.

Perhaps the most significant event of the week be the Green Challenge. The winner will get €500,000 from Sir Richard Branson. Last year's winner was Qurrent, an energy consumption monitoring company.

If you're going to be there drop me a line.