Four short links: 20 November 2009

Social Network Search for Morons, Bulking Up Bio Data, Better E-Mail, Better Standards

1. Spokeo -- abysmal indictment of society, first prize in mankind's race to the bottom. Uncover personal photos, videos, and secrets ... GUARANTEED! Spokeo deep searches within 48 major social networks to find truly mouth-watering news about friends and coworkers. PS, anybody who gives their gmail username and password to a site that specializes in dishing dirt can only be described as a fucking idiot. (via Jim Stogdill, who was equally disappointed in our species)
2. Biologists rally to sequence 'neglected' microbes (Nature) -- The Genomic Encyclopedia of Bacteria and Archaea is project to sequence genomes from more branches of the evolutionary tree of life. Eisen's team selected and sequenced more than 100 'neglected' species that lacked close relatives among the 1,000 genomes already in GenBank. The researchers reported earlier this year at the JGI's Fourth Annual User Meeting that even mapping the first 56 of these microbes' genomes increased the rate of discovery of new gene and protein families with new biological properties. It also improved the researchers' ability to predict the role of genes with unknown functions in already sequenced organisms. (via Jonathan Eisen)
3. Mail Learning: The What and the How (Simon Cozens) -- a few things that a really good mail analysis tool needs to do. I hope that my mail client and server does these out of the box in the next five years.
4. Introducing the Open Web Foundation Agreement -- The Open Web Foundation Agreement itself establishes the copyright and patent rights for a specification, ensuring that downstream consumers may freely implement and reuse the licensed specification without seeking further permission. In addition to the agreement itself, we also created an easy-to-read "Deed" that provides a high level overview of the agreement. Applying the open source approach to better standards.

Four short links: 11 November 2009

Participation Tools, Open Data Requests, Go Programming Language, Why Open Source is Better

1. ParticipateDB -- database of online tools for public participation. Closed alpha now, with 32 tools and 15 projects in the database. (via Sara Winge)
2. DataTO -- like data.gov, but it's where users request data sets. (In this case, from the Toronto municipal government)
3. Go -- new language from Bell Labs and Unix central figures Rob Pike and Ken Thompson, who now work at Google. Bits of C, bits of Google, it compiles to native binaries and runs nearly as fast as C. Built with concurrency and memory management as central figures. Not used in production at Google yet, but grew from a 20% project to something worthy of public release.
4. On Commit Bits (Jacob Kaplan-Moss) -- that day-one-commit-bit is one of the starkest differences between the corporate and the open source development model. [...] Granted, Django’s very conservative when it comes to granting that commit bit, but I’m not aware of a single open source project under the sun that’d give out a commit bit on a contributor’s first day. I’ve seen developers who’ve been hired to work full time on open source work for months without commit access to the project they’re paid to develop! One of several posts that Jacob's made about why open source makes for (on average) better software.

Four short links: 20 October 2009

Politics in The Age of Social Software, Ethernet Patents, Free Book Fear, Programming Exercises

1. Poles, Politeness, and Politics in the Age of Twitter (Stephen Fry) -- begins with a discussion of a UK storm but rapidly turns into a discussion of fame in the age of Twitter, modern political discourse, the "deadwood press", and The Commons in Twitter Assembled. There is an energy abroad in the kingdom, one that yearns for a new openness in our rule making, our justice system and our administration. Do not imagine for a minute that I am saying Twitter is it. Its very name is the clue to its foundation and meaning. It is not, as I have pointed out before, called Ponder or Debate. It is called Twitter. But there again some of the most influential publications of the eighteenth century had titles like Tatler, Rambler, Idler and Spectator. Hardly suggestive of earnest political intent either. History has a habit of choosing the least prepossessing vessels to be agents of change.
2. Apple and Others Hit With Lawsuit Over 90s Ethernet Patents -- unclear whether the plaintiff is 3Com (who filed the patents) or a troll who bought them. "We strongly believe that 3Com’s Ethernet technologies are being regularly infringed by foreign and some US companies," said David A. Kennedy, Chief Executive Officer of U.S. Ethernet Innovations. "We believe that the continued aggressive enforcement of the fundamental Ethernet technologies developed by 3Com against the waves of cheap, knock-off, foreign manufactured equipment is a necessary step in protecting the competitiveness of this American technology and American companies in general." (via Slashdot)
3. The Point -- someone's publishing Mark Pilgrim's "Dive into Python", which was published by APress under an open content license. Naturally this freaked out APress (it's easy to imagine many eyelids would tic nervously should such a thing happen with one of O'Reilly's open-licensed books). Mark's response is fantastic. Part of choosing a Free license for your own work is accepting that people may use it in ways you disapprove of. There are no “field of use” restrictions, and there are no “commercial use” restrictions either. In fact, those are two of the fundamental tenets of the “Free” in Free Software. If “others profiting from my work” is something you seek to avoid, then Free Software is not for you. Opt for a Creative Commons “Non-Commercial” license, or a “personal use only” freeware license, or a traditional End User License Agreement. Free Software doesn’t have “end users.” That’s kind of the point.
4. Programming Praxis -- programming exercises to keep your skills razor-sharp, with solutions.

Four short links: 19 October 2009

YouTube Bandwidth, RFID Visualization, Social Software Arms Race, Google Voice to the Laptop

1. YouTube's Bandwidth Bill is Zero (Wired) -- they buy dark fibre and peer with the major ISPs.
2. Immaterials: The Ghost in the Text (Vimeo) -- visualising RFID fields. See also the blog post about the work by Timo Arnall from Touch and Jack Schulze from BERG.
3. The Commercial Speech Arms Race (Bruce Schneier) -- Whenever you build a security system that relies on detection and identification, you invite the bad guys to subvert the system so it detects and identifies someone else. Sometimes this is hard ­-- leaving someone else's fingerprints on a crime scene is hard, as is using a mask of someone else's face to fool a guard watching a security camera ­-- and sometimes it's easy. But when automated systems are involved, it's often very easy. It's not just hardened criminals that try to frame each other, it's mainstream commercial interests. Bad actors game systems, and social software is just another system to be gamed. It's very difficult to create a system with no incentive to misbehave or to accuse others of misbehaving.
4. A SIP of the Future (Tim Bray) -- he connected Google Voice with Gizmo5 so his Google Voice number forwards to his laptop. FTW.

Social Networking is the Means to Achieve Workplace Collaboration

Yesterday I live-blogged a bit from the terrific Government 2.0 event produced by FedScoop.com at the Newseum in Washington, DC. I wrote a post about how collaboration was not the means, but rather an end made possible by the means of social networking tools. You can read my original writing and some initial comments here. Below, I expand a bit on these ideas.

My post was initially inspired by one speaker's (WFED's Chris Dorobek) notion, shared by some others (Justin Houk commented that, "Taxpayers don't want to think about those in government sitting around on Twitter all day even thought that might be an effective way to collaborate."), that social networking tools come across as too social or "fun" and that being social is not what people are truly doing (in the government) when they use them - they're collaborating. Thus, when marketing Government 2.0 to wider audiences, he feels that a term like "collaboration tools" is more appropriate.

In my opinion, while this might sound better to a more traditionalist, untrained ear, I think it is factually wrong to say that things like Facebook or Intellipedia are collaboration tools. True, collaboration often happens with these tools. And perhaps one could argue that collaboration is mainly what people hope to accomplish with them in the workplace. Fair enough. But I think that collaboration is the end result of leveraging social networks, which is in actuality what the social networking tools are for.

In other words, social networks are a means by which to accomplish something. This something might very well be collaboration. It might also be putting together an office softball team, or a study group of employees all learning Arabic. Is arranging players on a softball team "collaboration"? I don't think so. Is it an important part of a coherent, productive workplace? Perhaps. There are many important things that happen in workplaces based around social networks that are not strictly collaboration on work projects.

One big thing I've been thinking about lately is "leveraging social networks to accomplish important stuff" and no one can deny that personal relationships can influence collaboration. How well you know someone, how much you identify with them, how much you trust them, their level of reliability or transparency - all of these are values derived from social networking that then, when leveraged, can influence collaboration. Collaboration is not an end in itself, of course - it is a means to accomplish some end (finishing a draft report, etc.). So, social networking is a means to collaboration, which is a means to achieving some work or personal goal.

I also reject the notion that there is something wrong with having some fun at work. The idea that having fun with social software shouldn't be allowed in serious workplaces is ridiculous. And of course, anyone who's ever passed around a joke-of-the-week email, celebrated a colleague's birthday with a cake in the break room, or ended work at 4pm for an informal happy hour with the office (i.e., effectively every government and corporate employee) would surely agree with me on this. Work can be fun and be productive, too. The director of the Office of Personnel Management recently visited Google for a reason.

So, briefly, I think social networking tools are not necessarily collaboration tools. They are social software that allows social networks to be leveraged to accomplish things you find important. That might be collaboration on a National Intelligence Estimate (protecting America, earning your paycheck), or arranging a carpool with people in your agency (getting to work on time, being more green), or finding a racquetball partner (staying healthy, living well, bonding) - all of which postitively influence the workplace, in government and in the private sector as well.

As Fred Wellman commented on my original post, "I can't help but wonder if Chris [Dorobek] is seeking a more politically correct or business sounding name of the same tools with the goal of breaking down barriers to implementation and usage as opposed to a lack of understanding of the power of social networking applications in the business of government." I think there's a lot of truth to that. But I also think that, as an academic, this is actually not what we are doing.

This may sound a bit esoteric, but from an academic standpoint I think pointing out that using social networks - online and off - is at the very core of what we are doing is an important thing to point out. When we are "collaborating," we are leveraging social networks to accomplish important stuff.

Four short links: 9 October 2009

Negative Karma, Wal-Mart TQI, Idiot Airlines, and Native iPhone Apps in Lua

1. Don't Display Negative Karma -- A fascinating insight for those building social software, whether for collective intelligence or otherwise: There can be no negative public karma-at least for establishing the trustworthiness of active users. A bad enough public score will simply lead to that user's abandoning the account and starting a new one, a process we call karma bankruptcy. This setup defeats the primary goal of karma-to publicly identify bad actors. Assuming that a karma starts at zero for a brand-new user that an application has no information about, it can never go below zero, since karma bankruptcy resets it. Just look at the record of eBay sellers with more than three red stars-you'll see that most haven't sold anything in months or years, either because the sellers quit or they're now doing business under different account names. (I love finding articles like this, thinking "they should write a book for us!" and then realizing "oh, they already are!") (via Hacker News)
2. Information Wants to be Free, Even At Wal-Mart (Pete Warden) -- an interesting piece on the value of opening up data, sharing information in negotiations so the best outcome can be reached. I'd argue that this trust argument is usually a cop-out, hiding worries about turf and control. In most cases it's clear that it's not in the other party's best interest to screw you over, and if it is, why are you dealing with them at all? The worst cases I saw were between departments within the same company, often we shared more information with competitors than the guys down the hall. The other reason I see people not sharing is shame: many companies (and individuals) work hard to present a facade of competence and quality that facts belie.
3. The Forest, The Trees, and the Bag Fees -- The bean counters can't track the revenue dilution of all these new fees. They don't want to. We miss the forest for the goddamed trees all the time. And the CEO acts as if fees are found cash. Meanwhile, no one asks why our overall revenue is plunging and we're losing money quarter after quarter. Everyone acts as if one thing has nothing to do with the other. A reminder to watch the important numbers, e.g. cash in bank, profit, customer satisfaction. (via Bryan O'Sullivan)
4. Native iPhone Apps Written in Lua -- open source port of Lua with Cocoa bindings for the iPhone. This is a tutorial showing you how to install and get past Hello, World. Apple have already approved one app written using it.

Four short links: 5 October 2009

Bozo Cloud Talk, Annotation Fail(ish), Python MySQL Slash, and Infinite Books

1. Brown Cloud Marketing -- advertorial "interviewing" GM of a company offering "DNS in the cloud". This might be a worthwhile service, but the way he markets it (by saying open source is "freeware" and the market leader is "legacy") reveals a rich vein of bozo. Freeware legacy DNS is the internet's dirty little secret (actually, it's the reason we have a functioning DNS), Nominum software was written 100 percent from the ground up, and by having software with source code that is not open for everybody to look at, it is inherently more secure. (security through obscurity is equating clothing with being naked yet blind). The Internet kindly did the poor man's homework: screenshot of a cross-site scripting vulnerability in their customer portal, a Nominum security advisory from 2008, and the Nominum web server is running Linux, Apache, and PHP (all legacy freeware yet apparently not the Internet's dirty little secret). (via Bert Hubert and Securosis)
2. Public Annotations on Healthcare Bill -- using technology from SharedBook, Congressman Culberson hoped to get citizens marking up the healthcare bill. They're using the software but many are just commenting on page 1--turning the hosted annotation platform into a forum with an odd user interface. It's a UI challenge: designing a way to let focused people comment on specific things, while also permitting impatient unfocused people to comment on the general topic. It's like asking for a SmartCar that seats 80. See also OpenCongress and their annotation system which also has hundreds of comments on the first few lines of the bill (including 39 on the one line "111th Congress"--apparently more contentious than you'd think!).
3. MyConnPy -- pure-Python MySQL client library, useful because it requires no C compilation to install (and thus can work on systems without C compilers installed, e.g. mobile). (via Simon Willison)
4. The Infinite Book -- design concept for an ebook reader (not a product you can buy yet). Sexy. (via Gizmodo)

Four short links: 25 September 2009

On Wheel Reinvention, Research Visualization, New Comments, and Defective Congressional Data

1. Diesel: A Case Study In That Thing I Just Said -- a new asynchronous I/O library in Python, which earned this fabulous review from Glyph Lefkowitz who wrote the granddaddy of all asynch libraries in Python, Twisted. Again, I don't want to dump on Diesel here; for what it is, i.e. an experiment in how to idiomatically structure asynchronous applications, it's all right. For that matter Twisted has its fair share of bugs too, which would be pretty easy to lay out in a similar post; you wouldn't even need to do the research yourself, just go look at our bug tracker. But both Diesel and Tornado make the mistake of attempting to replace the years of trial-and-error, years of testing discipline, and years of portability and feature work that Twisted has accumulated with a few oversimplified, untested hacks.
2. Eigenfactor -- ranking and mapping scientific knowledge. Visualizations and analyses from when geeks attack scientific publishing.
3. Washington Post Develops Visual, Web-like Commenting System -- WebCom displays comments in a dynamic web instead of a traditional list. As new comments come in, the web gets bigger. The web, however, is not organized by chronology. King and his team believe that the most valuable comments are those that are rated highly by peers and those that spur responses. WebCom uses those criteria to organize the web. (via The Evolving Newsroom)
4. Congressional Data is Defective By Design -- You should have better access to this info! You should have — at your fingertips — immediate, unrestricted digital access to the full text of any piece of legislation the very moment it’s released publicly by Congress. This is punishingly ridiculous. Congress could immediately take steps to make all publicly-relevant legislative data comply with the community-derived Eight Principles of Open Government Data.[...] That is to say, bill info from Congress could and should be available today in real time, free of charge, open-source, and licensed openly, via such open-standards technologies as XML, API’s, and regular bulk data downloads. We're entering a time where the tools and methods that make good software can help make good laws. (via timoreilly on Twitter)

Four short links: 3 September 2009

Smarter Eyes, Urinal Protocol Efficiency, Petabytes on a Budget, and LocaLondon

1. Many Eyes Make All Bugs Shallow, Especially When The Eyes Get Smarter (David Eaves) -- Mozilla released bug submission data, and David realizes with some minor investment (particularly some simpler vetting screens prior to reaching bugzilla) bug submitters could learn faster. For example, a landing screen that asks you if you've ever submitted a bug before might take newbies to a different page where the bugzilla process is explained in greater detail, the fact that this is not a support site is outlined, and some models of good "submissions" are shared (along with some words of encouragement). By segmenting newbies we might ease the work burden on those who have to vet the bugs.
2. Urinal Protocol Efficiency (xkcd blog) -- geeks are pattern-matching creatures that can count. This leads us to a question: what is the general formula for the number of guys who will fill in N urinals if they all come in one at a time and follow the urinal protocol? One could write a simple recursive program to solve it, placing one guy at a time, but there’s also a closed-form expression. If f(n) is the number of guys who can use n urinals, f(n) for n>2 is given by: [...] The protocol is vulnerable to producing inefficient results for some urinal counts. Some numbers of urinals encourage efficient packing, and others encourage sparse packing. (via Hacker News)
3. Petabytes on a Budget: 67Tb for $7,867 -- DIY cloud hardware. (via timhaines on Twitter)
4. LocaLondon (Chris Heathcote) -- informative, ingenious, and replicable (like all that Chris does), it's a Twitter feed of art exhibitions in London (when they open, when there's a week left, and on the last day) and a glorious horizontal touchscreen-friendly meta-reviews site so you can quickly see at a glance what's on now and what people think of it.

The Library of the Commons: Rise of the Infodex

Somewhere between the realm of Personal and Shared media lies the realm of the Universal.

The realm of the universal is the Library of the Commons, a global repository of user-generated and crowd-sourced media and information.

Services that logically nest in the Library include: Amazon, Yelp, YouTube, Craigslist, Wikipedia, Flickr, Twitter tweets, Bit.ly items, Scribd docs, Expedia, Google News, Google Maps, TripAdvisor, iTunes, the App Store and any other services and/or information sources that 'just work.'

In other words, these are services that have defined the 'IT' to the point that we can now pretty much take their utility and availability for granted (typically via API access and/or embed codes with some form of customization wizard).

The Genesis of a Library

So how did we get to this place in the story? What gave birth to the Library of the Commons?

No one formally deigned it so, but from the countless me-too services borne of the dotcom and Web 2.0 land rushes, the above-referred services are the ones that cultivated the biggest audiences, grew the richest ecosystems and inspired the deepest engagement levels.

In Darwinian terms, these are the survivors, whose structures and workflows have been defined and refined by time/experience.

As such, they are generally well thought out, holistic and integrated, but more to the point, have large, engaged user bases.

Thus, the Commons presents a riddle. Almost as if inspired by Herman Hesse's 'The Glass Bead Game', the riddle is this.

If all of these services yield a smorgasbord of best practices, why not systematically emulate them so as to...FEDERATE them?

Put another way, what if a time came when people ceased trying to perennially re-create the wheel, and instead, started to 'decompose' these services; to empty their function sets from whatever nesting they were contained within; and to re-apply them into new contexts supported by a now federated data flow proxied within the Cloud.

Couldn't the composite feature set be exposed switchboard-style to enable any number of custom services and client apps?

To put some meat on the conceptual skeleton, consider the following exercise that I recently did:

A decomposition of Craigslist and TripAdvisor yields deep profiles that are accessorized and interconnected via context traversal flows, such as categorization routines, places, events, airfares, posts, pages, ratings, discussion threads, offers, jobs, businesses, products and personal listings.

Craigslist offers up 36 different sub-types of items For Sale; Services represent another 19 sub-types; Jobs 41 more; Discussions, another 72. And so it goes (including Housing, Personals and Community) across 175+ geo-locales.

TripAdvisor is an instance of this model that overlays a set of time-tested workflows specific to the relatively complex task of planning a vacation.

These workflows make it easy to match a travel plan to specific tastes, requirements and budget - regardless of the information traversal path you pursued to being ready to get pricing on desired travel dates.

Could these same workflows be re-purposed for researching and then purchasing other similarly complex products or services?

I will come back to that thought, in a moment.

The Rise of the Infodex

What is de-composed, can be re-assembled, and thus begins the Infodex.

The Infodex is a kind of next-generation Rolodex, with aspirations to grow into a real-time marketplace.

What exactly is the Infodex? It is comprised of three parts.

Part one is a listing tool for linking to content, creating a metadata wrapper around media items and encapsulating the above-referenced services (i.e., Yelp, YouTube, WIkipedia) into listing containers that define and expose the methods that one can interface to the media item (framework integrity stuff).

Part two is an indexing engine so that, once simple rules are defined, your media libraries and the information in the listings themselves becomes 'self-organizing.'

Named picture types (globes, animals, historic or famous images), for example, could be a federation of multiple picture services (Flickr, Photobucket, Getty Images) and 'discovered' pictures from past queries.

Looked at from this perspective, the goal, in part, is to establish a cloud-based, crowd-sourced Dewey Decimal System built around the outcome of facilitating better searching, compositing, cross-indexing, sharing, archiving, and analytics functions for specific media and information 'types.'

Part three of the Infodex is a unified runtime player that is congruent with the information flows of the mobile broadband age; namely, iPhone, Twitter, Facebook and Web (Javascript/Flash embeds/Adobe AIR) based viewing/playback environments.

One simple example of a basic type of function that might be propagated across all of these environments is the Three Item Topical List (e.g., Top Three Favorites or Three Most Related Items). Define once, propagate everywhere.

A core assumption of the model is that both the media player and the service integration layers are open-sourced. This ensures that the user experience is uniformly good across all of these services, and pushes proprietary-ness higher up the stack, thus raising the floor for all comers.

A final thought. Google became Google by indexing the web. Couldn't the next generation extend this approach by being federated, crowd-sourced and context-specific (i.e., media, information and service aware)?

Are their obvious best practices for The Commons? Obvious gotchas? What about the Infodex?

Related Posts:

1. Pattern Recognition: Makers, Marketplaces and the Library of the Commons


2. Envisioning the Social Map-lication


3. The Mobile Broadband Era: It's About Messages, Mobility and The Cloud

Four short links: 1 September 2009

Social Investigative Journalism, Mozilla Service, Gov Data, Video Fun

1. Help Me Investigate -- find other people who want to investigate the same things you do ("on which streets in my town are the most parking tickets issued?", "why is there a giant unused TV screen in the downtown of this city?", "how much does this city council spend on PR?"), work together to resolve it, and leave a record of the answer for others. It's a different angle on MySociety's What Do They Know.
2. Mozilla Service Week -- We believe the Internet should make life better. Join us the week of September 14-21, 2009, as we take action to make a difference in our communities, our world, our Web. (via MySociety)
3. Open Government Data: Starting to Judge Results -- mall, tangible, steps that turn published government data into cost savings, measurable service improvements, or other concrete goods will "punch above their weight" : not only are they valuable in their own right, but they help favorably disposed civic servants make the case internally for more transparency and disclosure. Beyond aiming for perfection and thinking about the long run, the volunteer community would benefit from seeking low hanging fruit that will prove the concept of open government data and justify further investment.
4. Three Frames -- small fun. I love that there are still small fun things to do. (via pleaseenjoy on Twitter)

Four short links: 11 August 2009

1. The Slowing Growth of Wikipedia and More Details of Changing Editor Resistance -- researchers at PARC analysed Wikipedia and found the number of new articles and number of new editors have flattened off, and more edits from first-time contributors are being reverted. This is a writeup in their blog, with the numbers and charts. It's interesting that coverage in New Scientist talked about "quality", but none of the metrics PARC studied are actually quality. Wikipedia launched a strategic review which aims to tackle this and many other issues. (via ACM TechNews)
2. The Information Architecture of Social Experience Design: Five Principles, Five Anti-Patterns and 96 Patterns (in Three Buckets) -- teaser for upcoming O'Reilly book with some really good stuff. Balzac once wrote, “The secret of great wealth with no obvious source is some forgotten crime, forgotten because it was done neatly,” and many successful social sites today founded themselves on an original sin, perhaps a spammy viral invitation model or unapproved abuse of new users' address books. Some companies never lived down the taint and other seems to have passed some unspoken statute of limitations. (via BoingBoing)
3. Skulpt -- entirely in-browser implementation of Python. (via Andy Baio)
4. Why Can't Local Government and Open Source Be Friends? -- the Birmingham example is one of many. Government procurement and tendering processes are often fishing expeditions, which biases responses in favour of commercial software companies making mad margins such that they can respond to RFPs that are really RFIs, etc. It's an issue everywhere in the world because it happens at local, not just central, level.

Four short links: 9 July 2009

1. Ten Rules That Govern Groups -- valuable lessons for all who would create or use social software, each backed up with pointers to the social science study about that lesson. Groups breed competition: While co-operation within group members is generally not so much of a problem, co-operation between groups can be hellish. People may be individually co-operative, but once put in a 'them-and-us' situation, rapidly become remarkably adversarial. (via Mind Hacks)
2. Yahoo! TrafficServer Proposal -- Yahoo! want to open source their TrafficServer product, an HTTP/1.1 caching proxy server. Alpha geeks who worked with it are excited at the prospect. It has a plugin architecture that means it can cache NNTP, RTSP, and other non-HTTP protocols.
3. App Engine Conclusions -- I've reluctantly concluded that I don't like it. I want to like it, since it's a great poster child for Python. And there are some bright spots, like the dirt-simple integration with google accounts. But it's so very very primitive in so many ways. Not just the missing features, or the "you can use any web framework you like, as long as it's django" attitude, but primarily a lot of the existing API is just so very primitive.
4. Microsoft Hohm -- Sign up with Hohm and we'll provide you with a home energy report and energy-saving recommendations tailored to your home. Wesabe for power at the moment, with interesting possibilities ahead should Microsoft partner with smartmetering utility companies the way Google Powermeter does. This is notable because this is a web app launched by Microsoft, with no connection to Windows or other Microsoft properties beyond requiring a "Live ID" to login. For commentary, see Microsoft Hohm Gets Green Light for Launch and PC Mag. (via Freaklabs)

Four short links: 12 June 2009

1. New Media Challenges: Legal and Policy Considerations for Federal Use of Web 2.0 Technology (Center for American Progress) -- report on the issues around Web 2.0 use in Government, which include privacy, security, Public Records Act, advertising, etc. See also It's Not the Campaign Anymore: How the White House Is Using Web 2.0 Technology So Far from the same group.
2. Government Data and the Invisible Hand -- Ed Felten has written a fantastic piece on the relationship between data, presentations of the data, and the government departments that produce the data. It is full of powerful recommendations: The best way to ensure that the government allows private parties to compete on equal terms in the provision of government data is to require that federal websites themselves use the same open systems for accessing the underlying data as they make available to the public at large. (via timoreilly on Twitter)
3. Fast Modularity Community Structure Inference Algorithm -- This algorithm is being widely used in the community of complex network researchers, and was originally designed for the express purpose of analyzing the community structure of extremely large networks (i.e., hundreds of thousand or millions of vertices). The original version worked only with unweighted, undirected networks. I've recently posted a version that works on weighted, undirected networks. (via mattb on Delicious)
4. First Driver for USB 3.0 -- After a year-and-a-half's worth of work, Intel hacker Sarah Sharp announced that Linux will be the first operating system supporting USB 3.0. (via deusx on Delicious)

Loki's Net

The National Security Risks of Gov 2.0 and the Social Web

Every culture has its Trickster myths because Trickster lives on the edge of what the rest of us perceive as "real." He crosses boundaries so often and with such ease, not to mention panache, that our own boundaries expand because of him. Trickster is “the doorway leading out, the spirit of the road at dusk” (Lewis Hyde) that doesn't belong to any town but is in-between all towns; the province of thieves and spies.

Here's an updated version of an old Trickster tale that I think is particularly relevant to the topic of this post--the national security risks associated with a more open Government in general and social software in particular.

Loki, the Norse God of mischief and mayhem, had taken to the mountains for refuge after angering the other Gods with his latest antics. The first thing he did was build a house with four doors; one on every side so that he could see in all directions. With his Intrusion Detection System in place, Loki spent the rest of his time playing in the water as a salmon, leaping waterfalls and negotiating mountain streams.

One morning, Loki sat by a fire and considered how the gods might capture him. Since he spent much of his time as a fish, Loki grabbed some linen string and fashioned a fishing net of a size and weight sufficient to snare him. Unfortunately, just as he finished, the other Gods rushed in. Loki threw the net into the fire, transformed into a salmon, and swam away. Acting quickly, the Gods extracted the ashes of the net from the fire and, from the remnants, rebuilt Loki’s net, eventually ensnaring him in it.

Like Loki, we construct through our Twitter posts, Facebook Wall entries and LinkedIn profiles our own unique “net” that sets us up for a social engineering exploit, a financial crime, or an act of espionage.

The Trickster archetype aptly frames this discussion about the risks and benefits of bringing Government into a Web 2.0 world because the classic Trickster is neither good nor bad, but encompasses elements of both. Too often, the debate surrounding Gov 2.0 becomes polarizing. Critics are frequently grouped together as Gov 1.0 thinkers struggling against a 2.0 world, while advocates sometimes embrace Gov 2.0 as a holy quest, refusing to acknowledge any significant risks whatsoever.

I cannot emphasize enough that the surest way to slow our progress toward a more technologically open Government is to try to craft this debate in dualistic terms. Indigenous Trickster tales teach us that a more valuable approach is to substitute utility for morality. Loki and Coyote (a famous Trickster in Native American lore) both understand how to trap a fish because they have swum as fish. Hyde writes in his book Trickster Makes This World that “nothing counters cunning like more cunning. Coyote's wits are sharp precisely because he has met other wits.”

There are serious and significant risks associated with Government 2.0 and the use of Social Software from a national security perspective that need to be talked about and addressed. It is a topic that is both complex and far-ranging and deserves much more coverage than I can provide in this post, although I hope to at least start the conversation at a new and edgier level. To give some perspective to the problem, there are 22,000,000 employed by the U.S. government, not counting government contractors. That fact alone makes Gov 2.0 a very significant technological evolution.

There is ample evidence that state and non-state actors are engaged in finding ways to exploit vulnerabilities in the U.S.'s critical infrastructure as well as the Department of Defense's secure (SIPRNET) and non-secure (NIPRNET) networks. Many of these attacks have been well-documented by Inspectors General (IG) and Government Accountability Office (GAO) investigations as well as through Congressional committee testimony by experts. One of the easiest ways for an attacker to gain access to those protected networks is not through the firewall, but through the user. In any secure system, the human element is always the weakest link. As Tim Thomas wrote in his excellent "Cyber-Skepticism" article for IO Sphere, the mind has no firewall but skepticism. The attack vector that best takes advantage of that vulnerability is known as social engineering.

Do you recall how Matthew Broderick's character cracked the password for the DOD computer Joshua in the 1983 movie “War Games?" He studied details about the life of its creator. That's the same strategy that David Kernell used when he allegedly hacked into Governor Palin's Yahoo account, except he had the benefit of a Web 2.0 invention known as Wikipedia.

How did the individuals behind the GhostNet espionage ring manage to entice so many people (1300 computers in 103 countries) to open an infected document which loaded a Chinese trojan named ghostRAT onto their system? They crafted an enticing email and document that was tailor-made for their audience -- supporters and/or employees of the Office of His Holiness the Dalai Lama. It was such an effective social engineering campaign that 30% of the infected computers were in sensitive government offices. And to make matters worse, most anti-virus programs failed to identify the Trojan.

In Cyber Warfare terms, these types of hacks are a part of Computer Network Operations (CNO) known as Computer Network Exploitation (CNE). Today, over 130 countries are developing a cyber warfare capability with CNE as one component.

Social media like Twitter, Facebook, MySpace, LinkedIn, GovLoop, and many others are very attractive venues for CNE by our adversaries because they are easily accessible, target-rich environments that can be exploited with little to no risk under cover of anonymity.

According to a recent study conducted for one of the U.S. Armed Services, 60% of the service members involved in the study have posted enough information on MySpace to make themselves vulnerable to adversary targeting. And these weren’t only young recruits making bad Operations Security (OPSEC) decisions. The 60% group included officers and enlisted troops from Intelligence and Security postings as well as other sensitive positions posting such things as units they have deployed with, new duty stations, personal medical data, job duties, information about training, and pictures of themselves at deployed locations.

In their paper “Social Software and National Security," Mark Drapeau and Linton Wells discuss the use of Twitter by Colleen Graffy, formerly Deputy Assistant Secretary of State for Public Diplomacy, to “impress her personality and message on foreign media prior to arriving in their countries, and after leaving.” As the authors point out, there are positives and negatives to Graffy’s method of using Twitter. One of the negatives that they do not address is that Graffy’s Twitter usage can become a vector for a non-state hacker to exploit with a @colleen_graffy tweet containing a malicious link disguised as a tiny URL. All of a sudden, Graffy‘s public diplomacy 2.0 effort could result in a State Department computer becoming a zombie.

The Open APIs on Twitter and Facebook provide a virtually unlimited resource for building target profiles on employees of sensitive government agencies like the Departments of Defense, State, Justice, Energy, Transportation, and Homeland Security. The Twitter stream, for example, adds a timeline for tracking when you’re at work, where you’re going after work, and what you are doing right now.

Another risk category is disinformation. Twitter received a lot of coverage during the Mumbai terror attacks of November, 2008 for its role in covering the events in real time. Part of what emerged was the potential for terrorists to use Twitter to propagate disinformation about their whereabouts; i.e., to announce a new attack occurring at a wrong address, thus adding chaos and confusion to an already chaotic situation.

Finally, there is the phenomenon of online trust. If you work in a targeted industry, you will be approached, sooner or later, by someone who isn’t who she claims to be for the purpose of gaining and exploiting your trust to further her own nation’s intelligence mission. One of the quickest ways to establish trust online is by finding things you both hold in common. Both Twitter and Facebook postings excel at that discovery effort.

How do you mitigate the risks while enjoying the benefits of Gov 2.0 and the social web? You do it by thinking like your opponent; or like the Trickster. Read your post twice before you hit send; once as you and once as your adversary who is looking to exploit you. If you work for the DOD or a government contractor, start by re-reading your employer's OPSEC guidelines and edit your profile and your posts accordingly. If your office hasn’t created any OPSEC guidelines for social media yet, please let me know. My company GreyLogic is creating training for precisely that purpose. In the meantime, here are five things that you can do right now to reduce your risk profile:

1. Involve your family members. They should understand that by virtue of your employment with a department, agency, or service, their posts are prime fodder for CNE. You can start by having them read this article.

2. Make OPSEC fun by making a game of it. For example, trade Twitter or Facebook aliases among your coworkers and see how much information you can learn about each other by using publicly available search tools. Then draft two or three email topics that would entice that person to take your bait if you were an adversary running a Spear Phishing operation. I promise that you’ll be amazed at the results. In fact, you should do this same exercise with your family members.

3. Be more skeptical about anyone who contacts you as a result of your posting on a social network. See if you can find their Internet footprint by searching on their name and email address. An alias with no Internet history should immediately raise a red flag.

4. Anyone can start a DOJ, DHS, DOE or other government agency community on Ning, LiveJournal, Facebook, etc. Don’t affiliate yourself with any community that you don’t know for sure is an officially sponsored and sanctioned one. Talk about shooting fish in a barrel.

5. Facebook recently reported that 70% of its traffic comes from overseas. Become more cautious about who you friend and who is privy to reading your posts.

In myth, like in life, the Trickster relies on the instincts and appetites of his prey to spring his trap. For those of us in Government or affiliated with Government, we would do well to remember that as we engage with Gov 2.0 on the social web.