Computerization in Nilekani's Imagining India

by Andy Oram | @praxagora | comments: 0

Imagining India: The Idea of a Renewed Nation promises to occupy a central position in discussions about India as well as the world economy this year. The book was released last year in India, came out in the United States just this past March, and has racked up some prominent reviews recently. Particularly relevant to this blog are the book's observations on computers' role in the economy and society.

Author Nandan Nilekani can speak with quite a bit of authority on computers, having founded and led Infosys, an early success story in modern Indian commerce and a major player in the historic rise of outsourcing.

Imagining India is a huge book with many big agendas; it covers education, infrastructure, environmental challenges, government intervention, and the role of historical narrative, among other things. Biggest among its agenda--and the one that I wager will generate the most debate--is Nilekani's own version of a modern combination of neoliberalism and neoprogressivism that seems to be gaining ground. The general idea is that governments should take a leading role to promote social progress by creating an infrastructure that allows individuals to form their own destinies (good education, good health care, good physical infrastructure, a light-touch form of regulation that ensures quality, and occasional direct welfare payments) rather than preserving oases of protection and easily abused subsidies for particular interest groups, notably unions, small businesses, and disadvantaged castes.

But all that lies beyond the scope of the Radar blog and of my own powers of analysis. I'll just comment on the following points from the book, because they concern the role of computers and because they resonate with trends I see in the U.S. and elsewhere.

• Technophobia shouldn't be assumed
• Internet access goes along with transparency and egalitarianism
• Software leads innovation in other areas

Technophobia shouldn't be assumed

A lot of technologists glibly anticipate that computers and Internet access will be rejected by some group of people who are implicitly labeled ignorant or clueless: racial minorities, poor people, the elderly ("how can you get my grandmother to use this?"), etc. In every case, the key to adoption turns out to be access and sometimes the availability of useful applications. When presented with the opportunity, these populations always prove eager to take advantage.

Nilekani cites one instance after another of rural village dwellers, farmers, taxpayers, and others who quickly grasp what computers and Internet access can do for them. Whether it's the chance to learn English, check crop prices, or pay a utility bill, Indians at all levels come to depend on the computer once it's introduced. (The hard thing, as you might guess, is persuading agencies and local officials to install systems that undercut their power as gatekeepers.) And we've all heard of the Hole in the Wall Project, where Indian kids in slums come to enjoy and figure out how to use computers with little or no adult help.

Nilekani may be citing anecdotes selectively, but his observations echo other reports I've heard about disadvantaged or lagging communities. The problem is not the people, but other factors such as availability, cost, and usefulness.

Internet access goes along with transparency and egalitarianism

One reason the Indian population loves computers, according to Nilekani, is that it attacks favoritism and outright corruption. This advantage matches up with the promise of open government in the United States and other developed countries.

In some cases, Indians are burdened by extremely crude forms of corruption that crumble the instant computers are installed. One example in the book is the registration of changes in land records, which farmers are required to report to the government every year. Agency staff could easily steal land by deliberately filing wrong reports, or extract bribes by delaying the filing until the desperate farmer caves in. But a computerized system takes the staff person out of the process.

Bringing sunlight into government activities in most developed countries has somewhat subtler effects and becomes a more long-term project, but the essence is the same and depends on computerization to work. In the US, we have a lot more control over the stimulus package, thanks to Recovery.gov, than we have over expenditures in Iraq or the bail-out to the finance industry. Indians are similarly learning how to watch over their governments and raise their voices digitally, according to Nilekani.

The sunny role that people around the world are granting to the technologies of going online is not intrinsic to these technologies, because they also lie at the center of modern surveillance, warfare, and regimentation. The benign role is hard won, and represents a collective choice by the public that has adopted the technologies. As Nilekani puts it:

The idea of technology as something ominous and scary that is used by "Big Brother" to control our lives and eliminate jobs has given way to the idea that it empowers, liberates and gives us access to all the services that are due to us, as citizens and consumers.

Software leads innovation in other areas

The reason that the computer industry was the first to take off in 1990s India is that it required practically no infrastructure. Of course, it required a computer, which might require six to twelve months for an import license in those days. It also required electricity, which could be obtained in major cities and supplemented by private generators. (In areas of unauthorized urban growth, the slumlords strung the wires.) So in a regulatory environment that scrutinized and imposed conditions on every allocation of equipment, it was much easier for entrepreneurs to set up a computer firm than any business that had more physical manifestations.

As is well known, the relative independence of computing from physical infrastructure also made Indian companies lucrative in a world increasingly linked by the Internet. Nilekani says that this physical flexibility was also valuable internally, helping IT-savvy businesses cut across the logistical and political barriers that have always geographically segmented the Indian market.

Nilekani seems to believe that there's nothing about the computer industry that's uniquely suited to Indian talents and business acumen. Now that the computer industry set an example, the same advantages have been applied to many other industries. In the 1980s, economists doubted that India could succeed in any industry, and a few years ago they wondered whether India could succeed in any industry except computer services. The evidence is now strong that the country will become a leader in many areas.

Indian industry is just one example where computerization has shown light on a path that social change can take. A worldwide example is provided by the open source movement, which Nilekani mentions only in the most fleeting manner in his conclusion--unfortunately enough, because free software can be a compelling wild card in story of international development, especially as part of a trend I dubbed tech-splicing in another article.

The first open license was a software license (the GNU General Public License). When it was released, the phenomena of allowing unlimited changes and sharing these changes looked like a peculiar aspect of software. But many years later, these ideas seeped out into fields of innovation with a more physical basis, and research by Eric von Hippel showed they always had legs.

Software was also the inspiration for gene splicing and other aspects of synthetic biology, even to the extent that biologists share their innovations in repositories that look like software libraries (check out the BioBricks Foundation).

Finally, the popularity of scripting and other software hacking initiated--or revived, or perhaps just legitimized--a tradition of solving a problem through invention instead of settling for a standardized, commercial solution. The DIY movement found in many areas of the world--including the Indian practice of assembling local motor vehicles called jugaad--makes it more and more likely that products of many types will come out of small, even amateur workshops.

Products of creativity and pure thought embody a freedom that allows them to metamorphose and spread quickly. The added formality and clarity that software brings to these activities doubles the power of that freedom. So my guess is that software will often lead the way in social innovations by a decade or more.

Seeing the Future of Mapping in Crimespotting

by Brady Forrest | @brady | comments: 9

This week Stamen Design released San Francisco Crimespotting. It's a crime map and notification system that allows for time and crime trend analysis. SF Crimespotting has launched just over two years after the release Oakland Crimespotting (Radar post). Stamen had been waiting for crime data all this time and with the launch of DataSF they are able to use an official API for crime data.

SF Crimespotting is very similar to the initial release for Oakland. As I wrote in 2007:

Each type of crime is assigned a color-coded icon with an abbreviation. You can highlight all of a crime type's markers with a mouseover. You can also change the number of days for which crimes are shown. Each crime has a detail page and that crime can then be viewed in context with others. You can also slice the data by day, type and the intersection of the two. You can also subscribe to get email alerts and RSS feeds for a specific place in Oakland.

The latest releases of the Crimespotting platform reflect several important trends in online mapping:

1) Crowdsourced Maps - When Oakland Crimespotting launched it used Microsoft Live maps (which would now be called Bing). They have switched to Cloudmade maps which are based on Open Street Map data. The maps look amazing and at initial glance they appear to be the same as any other major provider's maps. Google's Mapmaker project (Radar posts) has also been seeing more attention and just this week expanded into Mexico (I wonder how long until they bring Mapmaker to the US). Waze (Radar post) is using user-generated traces to create their realtime maps.

2) Temporal Mapping - Time is being added to online maps and other visualizations. As data comes to use in realtime there are new conventions that need to be developed. Stamen, through this project and their work with Trulia Hindsight (Radar post) and MySociety (Radar post), are at the forefront of designing methods of dealing with varying scales and types of time data. In their post The Pie of Time Stamen details their thinking for how to represent hours, days and years in the project. The old Crimespotting did not allow you to navigate to archival data. With the new UI there are now permalinks to all crime reports The hours control is shown to the right. Only the crimes that occurred during the highlighted times will appear on the map. Stamen has included quick links to show specific times like "Commute" and "Nightlife".

The slider and dropdown used to navigate days, months and years are shown above. Each day of the slider shows the total amount of crime that day. The highlighted area dictates the crime shown on the map.

3) Government Data - The new federal administration has shown a renewed interest in releasing data (most of this will have some geocomponent). The 2010 Census is around the corner and that will add to the data flow. As more data is released you can expect an explosion of government mashups. You can also expect more civic minded companies (especially after this week's exit by Everyblock (Radar post)).

4) Geo-Analysis - GIS used within enterprises, governments and universities are designed to take massive geodata sets and simplify them so that decisions can be made. Crimespotting may look like a slick consumer app and that's because it is. However as you manipulate its many controls you'll realize that you can learn a lot about a city and how a time of day or section of the city impacts the likelihood of your being involved in a crime. You can determine if you're more likely to be mugged in the Castro on Thursdays vs. Tuesdays. The only problem is that you are limited to crime data. I'd love to have ability to add other layers like housing prices or average income. Crimespotting has a read API; I hope Stamen adds Write capabilities.

I'd ask the kind folks at Stamen (very nicely) to make a Crimespotting Seattle, but unfortunately we don't publicly release our crime data. Here's to hoping that we get a mayor in this Fall's election who will open the data coffers. Does your city share its data? If so include a link in the comments.

All of these trends are going to be big topics at this year's Where 2.0 (3/31-4/2 in San Jose). Submit your topic now!

Privacy and open government: conversations with EPIC and others about OpenID

by Andy Oram | @praxagora | comments: 2

A few days ago I proposed a way to offer more privacy to people visiting government web sites. This blog builds on that proposal, which was largely technical, by examining the policy and organizational issues that swirl around it.

My ideas are informed by a discussion I had with Lillie Coney, Associate Director of the Electronic Privacy Information Center. The blog is also inspired by two comments on the earlier blog and brief email I exchanged with one commenter, which intertwine with Coney's in intriguing ways.

As I said in the first blog, my proposal focused on a very narrow question driven by the Obama Administration's interest in revising a memorandum from 2000 concerning the use of cookies in web browsers. The proposal suggested a way to better approach anonymity, but didn't look at the related social and political issues:

• The kinds of privacy and the degree of privacy people want
• When it's appropriate to make visitors identify themselves, or at least to provide some persistent identity
• Whom people trust to maintain identity information

This blog offers a number of points about those issues. The sections are:

• Can the government be your friend?
• Anonymity, pseudonymity, and participation
• Who should run an OpenID server?
• Thought experiment: could federal agencies offer anonymous authentication for whistle-blowers?

(continue reading)

Four short links: 3 July 2009

Stats, Public Domain, Sewers, and Garbage

by Nat Torkington | @gnat | comments: 1

1. OECD Factbook -- Flash-built impressive data explorer from OECD. Go to Indicators > Load and, in the words of Ben Goldacre, "prepare for nerdgasm". (via bengoldacre on Twitter)
2. James Boyle is on Twitter -- author of the book The Public Domain.
3. Sewers and Startups (Pete Warden) -- designing to last, reminds me of Saul Griffith's heirloom design riff. When I joined Apple back in 2003, the central build farm for all projects had both PowerPC and x86 Darwin boxes, and our code had to compile on both. Steve was playing a long game, years before the Intel switch he was obviously planning for it, (though I only caught the significance in retrospect).
4. Open Data Makes Garbage Collection Sexier, Easier, and Cheaper -- pragmatic use for open government data. For more on the author of this post, see Hello World for Open Data by Tim Bray.

Naming an Emerging Movement

by John Geraci | @johngeraci | comments: 62

There's a movement going on around the world.

We don't have a name for it, though.

Gov2.0, e-gov, e-democracy, open gov--these are all names that get applied to what is happening. And they are great for describing a certain aspect of this movement, the aspect that actually deals with government.

What's really going on right now is much bigger than that. Open gov is a big part of the story, but not the whole story. On top of Open Gov, there are organizations like The Open Planning Project (TOPP), or Front Seat, or my own DIYcity, working alongside these open gov groups, trying to make the whole civic system work better. Or there is Robin Chase, CEO of GoLoco and founder of ZipCar, singlehandedly trying to reinvent the way transportation works in cities. Or there is the subway alert I just got in my inbox, courtesy of New York City's MTA, notifying me that the F train has delays due to mechanical problems. All of these entities--TOPP, Front Seat et al, plus the open gov groups--are interrelated, and together create a new, emerging ecosystem of information, user activity, and possibility. But that ecosystem doesn't fit neatly under the hood of "Gov 2.0" or any of the other "gov" labels.

Recalling my post last week about the four pillars of an open civic system, these "gov" names--e-gov, gov2.0, open gov--focus on the G2C aspect of what is going on, to the exclusion of the other aspects of this open civic system that is emerging.

And this new civic system should have a name, because it is a real ecosystem. It is also a movement, with more and more people focusing on it around the world every day. It is also increasingly becoming an industry.

So what do we call this new thing?

What do we say when we want to say to someone, "All of the stuff that is emerging right now in the civic space that helps communities operate better, both with and without direct or indirect involvement on the part of the government?"

I was talking with Micah Sifry, co-host of this week's Personal Democracy Forum, a while back, and he suggested the name "civic software" for the apps that come out of this space. Riffing off that, I have been talking about the "open civic system." Fred Wilson of Union Square Ventures, who recently posted on this movement, thinks that name is too long, but also thinks "civic software" doesn't quite do it justice.

So I thought I would open up a thread here on Radar for a discussion:

What should this new space be called?

Let me know what you think. All ideas are welcome...

Personal Democracy Forum ramp-up: adaptive legislation can respond to action in the agora

by Andy Oram | @praxagora | comments: 0

This article is the last in a series leading up to the Personal Democracy Forum. The first article was posted on June 16 and the second article on June 19.

---

Whole libraries could be filled with writings about the growth of executive power during United States history. The power of the executive branch is likely to increase with technology. But for open government, that growth may be a necessary transition to more public involvement.

As its name indicates, the executive branch is responsible for carrying out the law. The open government movement wants the public to have more say in its own governance, and envisions a more fine-grained implementation of government's role in everyday life. For instance, open government advocates want more citizen input into details such as the siting of physical facilities and the choice of projects for funding. Logically speaking, therefore, the public has more control over implementation if decision-making is shifted from the legislature to the agencies carrying out the law.

Congress should also crack open its hidden chambers; law-making itself could be much more open. It will be interesting to see what comes out of work on a collaborative law drafting project in health care, started by Congressman Anthony D. Weiner of New York. I don't harbor any fantasies, though, that much of his public input will survive the traditional Congressional horse-trading that will follow.

But even the most ideal legislative process ends up with a static document that tries cumbersomely to anticipate every use and abuse of its language. (That's why laws are filled with hedges such as "This passage shall not be construed to...") Legislation is like setting off over rough terrain in a tank. Although the tank can complete the journey, it does so only by flattening everything it encounters.

Some political scientists also think that the executive branch is inherently better suited to understanding and responding to public needs. Here is an intriguing quote from Jane E. Fountain's Building the Virtual State: Information Technology and Institutional Change, summarizing work by Alfred C. Stepan:

Intellectual activities and decisions of civil servants working for long periods on policy questions are arguably more powerful and influential than the sporadic attention of legislators to particular policies.

So I'll take a look at the future of the executive branch, and end this three-part series with speculation about how to build fewer legislative tanks and more Jeeps.

• The executive branch: power and potency
• The legislative branch: how to write laws for an engaged public
• Appendix: the top question asked on Change.gov

The executive branch: power and potency

There's little mystery concerning about why the power of the executive branch tends to grow. Of the three branches of government, it's the one that actually arrives on the scene. It makes decisions about real people and activities on a daily basis and takes responsibility for those decisions.

To act effectively, the executive branch tends to centralize. (Unfortunately, so have many legislative branches in recent decades. The Commonwealth of Massachusetts is run by three people, when they're not fighting indictments or running off to seek other positions: the Governor, the Speaker of the House, and the President of the Senate.)

Because knowledge is power, technology will cause the power of the executive branch to skyrocket over the next few decades. Civil liberties advocates already decry surveillance cameras, wiretapping, the subpoening of information collected by private firms, and the computer analysis that the government applies to all the resulting data. But the data currently available is miniscule compared to everything that will be collected by atmospheric sensors, electronic toll collectors, and various other technologies that are starting to be installed. If Microsoft can produce a game machine cheap enough for the consumer market with face recognition, voice recognition, and full-body motion sensing, what can the government do to track us?

So the power that the executive branch takes on in the political realm will be multipled by the potency it obtains from the data it collects and from ever more sophisticated tools for analyzing that data.

(Strangely, the strict constructionists and "original intent" scholars, who bar judges from interpreting the Constitution broadly, don't apply these restrictions to the ever-expanding executive branch.)

I don't know how to halt this expansion of power. We could open-source the Panopticon by demanding that the public have access to all data collected by public cameras and senors. That won't help, though, because the data will still prove useful mostly to large organizations with the time and expertise to analyze it. And do you want to encourage every budding computer hacker in the country to become a data-mining Nancy Drew?

We could call for strict laws to restrict the collection or sharing of data. You'll still suspect that somebody is collecting information on you. But you'll rest easier because the fear of prosecution will keep them from sharing the data with most of the people you are afraid to have know it.

Still, the reasoning in this article suggests that open government advocates should welcome the shift of initiative away from the legislative branch to the executive one. But only if that's a transitional stage to lodging decision-making more in public hands.

In fact, the other two branches of government and the public had better find ways to implement collective participation, because it may be the only alternative to a resurgent Government 1.0.

To make this shift a positive change, we'll need well-established government/public collaborations that run through the whole cycle I described in my first article. We'll need to make sure that everybody is online and has the training to participate in decisions at the level of their competence and interest. We'll also need to refine polls and discussions to give us confidence that the public's most important concerns and desires rise to the top of the forums.

And when all that's in place, we can start to experiment with adaptive legislation.

The legislative branch: how to write laws for an engaged public

I mentioned at the beginning of this article that legislatures could develop laws in a more transparent manner. But that's only a start. If they could rely on public participation during the implementation of the law, they could write laws that embrace such input.

Laws already include feedback mechanisms. Many call on an executive agency to collect information on the effects of the law, run hearings, and release a report after a fixed amount of time so that the legislature can evaluate whether the law is achieving their goals. This practice could be dramatically extended by involving the public in the implementation of the law at the start, though continuous forums. The feedback loop would be reduced from years to weeks.

Laws also include ways to delegate control. For instance. Community Block Grants are offered to municipalities to spend as they see fit. (My town manager spent several hundred thousand dollars of our block grant to improve a park next to Town Hall, which in my opinion showed dubious judgment during an affordable housing crisis.) The idea of delegation could also be extended to more and more facets of law. What if a virtual town hall debated the expenditure of the town's Community Block Grant?

Critics of government solutions to social problems--usually political conservatives--accuse the law of being too rigid. The legislative process has trouble evolving with the times and responding flexibly to new conditions. Well, with provisions for public comment and group decision-making, laws can be as flexible as we want.

Congress needs evidence, though, that public feedback reflects the diverse needs and values of the population. Public participation must be protected against the complementary evils of capture by special interests and tyranny of the majority, which I have termed the problem of stakeholders.

If the public can live with a law it debates and tweaks as well as it can live with a law designed by Congress, adaptive legislation is viable.

And we need this flexibility, because the really big problems we have to tackle are what computer scientists call "massively distributed." Problems of this type include climate change, health care cost control, a food crisis that leads to rampant obesity in some populations and rampant starvation in others, job creation in an era of reduced staffing needs, and more.

The presence of the term "Collaboration" in the Administration's open government initiative reflects their understanding that they cannot solve the problems by themselves. Nor can technology, the market, or educational efforts--they must all work together. The concept of Megacommunity perhaps reflects the size of the effort (I actually find the "mega" part of the term slightly redundant) but may not even be enough to capture the extent of cultural adaptation required. In any case, adaptive legislation could trigger related efforts and bolster their effectiveness.

Appendix: the top question asked on Change.gov

Although Obama's approach to data sharing is a welcome sea change from the previous administration, the most committed members of his constituency press him to show more transparency about things that particularly matter to them, such as the role the Administration is playing in the financial system and what it knows about torture.

When Change.gov opened a public forum for questions at the beginning of Obama's presidency, the first place was taken by a question about prosecuting US officials suspected of promoting torture. Progressives then cried foul when the Administration failed to answer.

But did Obama really fail to answer? On April 16 he released Bush Administration memos that showed irrefutably that highly placed officials had discarded legal safeguards to institute interrogation practices that were described by these memos in gory detail.

Yes, Obama has resisted investigations of torture before and after this moment. But I am convinced that by releasing the memos he launched a historical process that cannot be reversed. The memos were his answer to the question that the public forced on him in January.

He has made the kind of political calculation that is his hallmark, deciding not to confront Republicans directly with a torture investigation. But if decent citizens keep up the pressure, prosecution will ultimately reach any US officials responsible for human rights violations, just as it did Pinochet and Fujimori. Open government applications do not free activists from the responsibility to engage with every accessible locus of power.

What democracy advocates must remember is that open government is not just a discussion forum. It's a maelstrom of intersecting investigations and competing proposals just as complex as the current political process. In fact, open government can succeed only by integrating with a political process that has a twenty-five hundred year history, even though our goal over time is to transform that process.

Now that the Administration wants to dance, we must learn all the steps. Listen closely: the musicians have already struck up their first round.

Twenty-five hundred years of Government 2.0

by Andy Oram | @praxagora | comments: 7

This article is the second in a series leading up to the Personal Democracy Forum. The first article was posted on June 16.

---

There's been a lot of excitement lately about the term "Government 2.0." Strip away the RESTful interfaces and you see that the new practices in government transparency are just intensifications of things democracies have done for a long time: public comment periods, expert consultation, archiving deliberations, and so forth. So let's look back a bit at what democracy has brought to government so far.

Like any telescoped presentation of history, this one reduces the swirling forces that extend and retract their way through the centuries into a couple near-mythological categories. I do this in the service of evaluating the concepts we toss around when discussing government participation.

Government 1.0: empire

Last year, Boston residents and visitors got the chance to see an exhibit of sculptures preserved from the culture that earned a special role in history as the first major power to exert ruthless control over many peoples: the Assyrians. Other dynasties--Egyptian, Chinese, Babylonian, and Akkadian--were around before the Assyrian empire, but the Assyrians were the ones that set a new standard for cruelty. The fearful image assigned to them in biblical texts also assures them a special fascination for Westerners.

Most visitors to the Museum of Fine Arts were thrilled by the artistic quality of the wall reliefs, human figures, and everyday objects. Personally, I was depressed by the unrelenting scenes of war and cruelty.

Assyria refined a strategy of subjecting cities just outside their borders and using the resulting booty to raise soldiers and provisions to attack the next frontier. Any populations whose subjugation was in doubt would be uprooted and forced to move closer to the center of the empire, replaced in their old homelands by more compliant subjects.

When the court entertained local dignitaries or foreigners (the lobbyists of the day), they walked through "lobbies" adorned with the scenes of carnage that ended up last year at the center of the Boston exhibit. The depictions of chariots crushing helpless civilians and soldiers impaled on stakes gave visitors a clear message: submit or end up the same way. Thus the Assyrians promulgated a "shock and awe" doctrine four thousand years before US troops brought their own version to the same geography.

This went on, with interruptions, for 1,300 years, and established a practice that guided other empires for thousands of years to come.

Some empires were more humane, of course. Empires could provide their inhabitants with protection and stability through currencies, constables, and courts (remember Hammurabi's Code). But all these policies remained subject to the whim of the supreme ruler.

And that is the distinguishing trait of Government 1.0: unchecked power centered in one individual. The reason emperors could stay in power was that they exploited their hierarchies to delegate both power and wealth. As long as governors maintained loyalty to the emperor, they could exert broad powers in the regions under their control and use those powers to accumulate great amounts of money. They in turn delegated power to those beneath them, and so on down through the hierarchy.

What could be more successful than this carrot-and-stick methodology combining vast rewards with threats of terror?

Government 2.0: democracy

There must be something persnickety about the character of ancient Athens. They couldn't tolerate strong leaders. Almost anyone who ever pulled off a major military victory, proved to be a persuasive orator, or got a corner on political power eventually found himself executed or exiled. (The Athenians invented the idea of "ostracism"--a fiercely democratic institution in their implementation, ironically.) Socrates was just one of the later examples of the propensity Greeks showed for bringing down anyone who was widely admired.

So this seems to be a natural setting for a system that grants a voice to a wide range of citizens. The decisions they reach may not be the best, but they're decisions that the political body can follow through on, having been reached democratically. The losers (if they weren't powerful enough to scare the winners) can stick around and try again at the next gathering in the agora.

Greeks recognized from the beginning the problems of democracy with which we are so familiar today. They knew that many votes were bought outright, and that others could be pulled in by smooth-tongued sophists. They also knew their democracy rested precariously on the labor of the slaves and other disenfranchised residents. And that a democracy could become an oppressive empire, using behavior against people next door that it would never tolerate within the walls of its own city.

I like this disturbing contradiction. That's why my web site, identi.ca account, and Twitter account are named after Praxagora, a character in an ancient Greek play that shows both the flaws and the immense power of democratic systems. The name Praxagora combines "action" with "public forum."

Right or wrong, a democratically reached decision--which if properly done, comes into focus as an emergent property of the assembled masses rather than being imposed by one party or individual--has an irreproachable authority. Socrates didn't like democracy, but if we are to believe Plato (who also didn't like democracy), Socrates insisted on obeying the popular will, even at the cost of his life.

We shouldn't hang a halo around direct democracy. In fact, the trend in technology-driven government transparency is not Athenian direct democracy--despite its idealization by some activists--but a tighter agency/public partnership. Today's experiments in public participation go far beyond electing representatives. But even the traditional American political culture consists of more than bills and vote counts. For instance, the executive branch tends to consult regularly with the public, a topic I'll take up in the next article in this series.

As we don digital media and communications--those somewhat ungainly garments we try to mold to human forms--in order to improve on twenty-five hundred years of flawed Government 2.0, we can learn some lessons from those millennia:

• No individual can be allowed to gather too much power, but every individual needs to be heard and to be protected from arbitrary persecution.
• Those who are excluded from the benefits of society will eventually rise up to wreck it.
• The majority is often wrong, and any political system can be abused.
• Good decisions take time, and a willingness to subject the decisions to constant re-examination.
• We need to rise above rhetoric and pursue the ultimate (if ultimately elusive) truth.

Like any useful technology, digital media and communications can help us realize a vision. Government 2.0 is a very old vision. A recognition of what has been achieved and what still challenges us can guide the development of the proper technology.

For instance, we can learn from history to bring the technology of participation to every member of the population and give them the opportunity to learn it, to subject the results of electronic deliberation to review by authorities governed by outside checks and balances, to highlight experts' reputations so they can wield more influence, and to give participants on electronic forums a few cycles of decision-making to work out processes that make effective use of the technology.

Deploying Government 2.0 technology will teach us more about that technology, and about ourselves.

Next article (Wednesday, June 24): adaptive legislation can respond to action in the agora.

Personal Democracy Forum ramp-up: from vulnerability and overload to rage, mistrust, and fear

by Andy Oram | @praxagora | comments: 3

The Personal Democracy Forum will hold its sixth annual conference at the end of this month. The theme, "how technology and the Internet are changing politics, democracy, and society," has been central to O'Reilly's work over the past few years (and a theme on which we're holding a summit of our own in September). Over the next two weeks I'll write three blogs on the Radar site to get some of my current thoughts off of my chest, clearing some space so that when I get to the PDF conference itself, my blogs can focus on its events and statements made by its participants.

This blog covers:

• The government participation cycle: if you want to dance, sir
• Vulnerability: a reason to put brakes on outgoing information
• Overload: a reason to put brakes on incoming information
• Rage, mistrust, and fear: inhibitors of the government participation cycle

The government participation cycle: if you want to dance, sir

The grand vision for government/public collaboration is a set of feedback loops that intensify the influence of the collective will on government policy. A feedback loop might consist of a cycle like this:

1. An agency (or less likely, a legislature) posts data in a downloadable format through a flexible API and announces a call for applications.
2. Companies and public interest groups define goals and put programmers on the task.
3. The public uses the resulting applications to generate data and share it with the agency.
4. The agency sets policy or changes direction in response to the data.

At any step, a failure by any of the responsible actors to follow through will leave the process hanging and discourage future projects in open participation.

This doesn't mean every project needs to include all four steps. The public may benefit from government data without offering feedback, and programmers could put their work under an open source license or into the public domain for the benefit of the government or members of the public without asking them to share more data. Agencies can also use programs to improve internal coordination instead of working with the public. But the full four steps serve as a canonical model for government/public collaboration.

Successful examples already exist for each step. As I write this, Data.gov has 261 data sets and 30 tools; thousands more data sets are promised soon. Appeals for donations of code, such as Vivek Kundra's Apps for Democracy in Washington, DC and the Sunlight Foundation's Apps for America, show that coders will play their part, at least in the current atmosphere of enthusiasm for the new initiative. And the public has responded to requests for data.

But at the federal level, we need to dance a few rounds of the full cycle before feeling confident that open processes are fully entrenched. I'll return to this theme in the last section of this article. The cycles of public participation will teach lessons, of course, that feed into a still larger cycle of constant experimentation and improvement.

As public participation moves forward, it's worth remembering that resistance to the free flow of incoming and outgoing information is not irrational. The resistance spring from healthy coping mechanisms learned by individuals and organizations learned over their lifetimes. I have already published and solicited comments on a list of fundamental questions on government participation; in this article I describe two such issues that play a special role in resistance to information sharing.

Vulnerability: a reason to put brakes on outgoing information

A couple months ago, I read a stirring report from a federal agency manager trying to sound out the Administration concerning how much the agency ought to reveal. The manager was stunned and inspired by the response of Bev Godwin, a prominent director at the White House and General Services Administration, who advised talking about the bad things as well as the good and soliciting negative as well as positive feedback.

Vulnerability is the keystone of transparency and openness. Online forums, if they are run democratically and competently, encourage vulnerability through a combination of self-correcting mechanisms:

The right to respond

Anyone criticized in a forum has repeated chances to defend himself at length. If the forum includes a rating system, persuasive arguments and well-chosen facts will float above false accusations as well as flaccid excuses from the accused.

Support networks

Proponents of each side pile on to each debate, turning it into a community issue and diluting the personal biases brought by the people who began the debate. A bit of a mob scene can erupt at times, awakening the risk that the losing side will walk away in a huff while sensitive community members flee the fury. But as long as participants value the community over partisan agendas and prefer honesty to grandstanding, the community comes out stronger, more aware of its options, and ready to integrate what it has learned into further action.

Community memory

Forum members recognize when old debates are re-ignited, and can fill new members in on the history. They can also predict the way prominent participants will line up on an issue. Debates are thus tighter and more quickly resolved.

A propensity for truth

These traits all end up privileging accuracy and making it harder (although not impossible) for bad judgment to prevail through false claims, manipulative demagogy, appeals to group solidarity, and the other tricks used by insincere factionalists.

This list may present online forums in a bit too rosy a light. But they do permit social norms that protect vulnerable people, even if the norms don't function perfectly. The real problem comes when words leave these forums and end up in other environments not subject to the same rules.

Government staff have already witnessed too many negative experiences in traditional, non-virtual settings. They have seen what happens when a comment is taken out of context and bandied about in the broadcast or print media, introduced into court testimony, or used as ammunition in partisan debates. They know that comment posted on the Web can be fodder for the same opinion machine--and are in fact even more dangerous because the Web makes them more visible.

That's not fair. It's very hard for anyone outside an agency to judge why it came down on one side of a debate or what that decision's long-term effects will be. Most agency actions are a complex fermentation blending the data that was gathered, assessments of the data's accuracy, assessments of the possible trends indicated by the data, consultations with the public (yes, outsiders are routinely consulted), judgments about Congress's intent, judgments about the interests of the Administration, and more. But groups with a cause like to ascribe one-dimensional reasons for key agency decisions and mine public statements for corroborating evidence.

This doesn't mean that all agencies are honest and act in the public interest. Plenty of bad government decisions have been made under pressure from well-organized special interests or to pay off political donors. One role of civil society is to expose these influences--that's what open government and the Personal Democracy Forum are all about. So we want more of these online forums. But we also need to protect the agencies whom we expect to use the forums. To encourage the necessary vulnerability, we have to combat those who abuse the results.

Journalism is starting to incorporate its own feedback loops and open its pages. Elections and policy debates are also monitored by the blogosphere. So some forums are becoming friendlier to the cause of vulnerability (the court system is unlikely ever to change). But it will be a long time before it's safe to lay out one's thoughts in an open, self-policing community.

Overload: a reason to put brakes on incoming information

The previous section mentioned the possibility of a "mob scene," and if people putting out information must be able to tolerate being vulnerable, those requesting input from the public have to deal with a potentially low signal-to-noise ratio.

We need not look far for an example. Take last month's brainstorming session on open government, launched by the White House and the Office of Science and Technology. It drew over 1,000 submissions in a single week. (Even more are on the site now, but they arrived after the official close of the session.)

The thousand submissions offered quite a smorgasbord for a group led by the new Deputy Chief Technology Officer for Open Government, Beth Noveck, to spoon through. They ended up with many intriguing ideas. But the gathering of ideas was simply a suggestion box, not real crowdsourcing. The web site offered no tools for editing, combining, and culling entries (and there would be inadequate time to use such tools anyway). The only aspect reminiscent of group behavior was a casual and anonymous rating system, which played little role in the results.

And that's a relief. After all, how many Americans would be able to assess the Office of Open Government created by Florida Governor Charlie Crist, or the potential for Cooperative Research and Development Agreements to help convert government data and applications to open source? Both of these projects earned a place in the results, even though the Florida model got only 24 votes and the Cooperative Research and Development Agreements only 46. (Although they might have conceivably been mentioned in an earlier brainstorming session conducted among government workers, I couldn't find them in the publicly posted comments.)

In response to a question about the voting, Noveck wrote me, "We wanted to encourage the National Academy of Public Administration to try different voting techniques. They started out by allowing voting by unregistered users, and later restricted it to registered users. Given the change, we didn't want to disadvantage anyone who participated. Consequently, we viewed the voting as informative but not determinative. On our weblog, only registered users can vote on comments."

As her statement indicates, the second phase of this transparency project has already sprouted more of the checks and balances found in mature discussion forums. We can expect the Administration to wend its way toward systems that gather useful opinions from self-organized groups of qualified commentators, the model pioneered by Noveck in her Peer to Patent project.

But will the White House have the time and resources to establish a foothold for a solid and lasting open government program? That depends on public tolerance for the Administration as a whole.

Rage, mistrust, and fear: inhibitors of the government participation cycle

Everyone knows that productive collaboration can't take place under conditions of rage, mistrust, or fear. Americans unfortunately are suffering from all these feelings right now.

Their rage has been directed at the heads of the financial industry. No peasant at the time of the French Revolution felt more hatred for Marie Antoinette than some of the comments I've seen about AIG. In addition, the current conditions of recession and financial uncertainty breed mistrust toward all three branches of government, and fear toward anyone who could seem to wangle an extra advantage over other Americans.

I'm not going to factor in the recent murders of law enforcement officers, Dr. George Tiller, and others because I'm sure the hate crimes were caused by lots of diverse factors, and it's unclear whether they represent a widespread cultural movement. We have plenty to worry about just by considering problems that will undeniably have a broad impact on Americans.

Over the coming year, lots of homes will continue to be foreclosed (because Congress failed to put a system in place to stop them), a blight that hits many neighborhoods like a dry Katrina. This ongoing crisis will be joined by credit card crisis (because Congress's bill didn't do much to stop that either) and perhaps already a student debt crisis. The Administration has its own challenges, waging two untraditional wars that nobody knows how to win and tinkering with a global financial system that always cracks its casings.

Open government doesn't deserve to be at the mercy of current political controversies. It did not originate with the Obama administration, and it doesn't require a Democratic Party philosophy. The George W. Bush administration took some steps toward open government (often forgotten amongst all the complaints over their unsavory maneuvers and information withholding). The Bill Clinton administration took steps too. But Obama is making it a centerpiece.

This gives us more hope than ever for openness, but ties its fortunes to the larger sphere of activities by the Administration and federal government.

To establish a foothold, openness needs some early, impressive success stories. Federal CTO Vivek Kundra has said his initiatives will prove themselves by saving money, although that certainly isn't his sole aspiration. If the Administration can land a few universally recognized successes--budgetary or otherwise--and especially if it can run through the whole cycle I laid out at the beginning of this article, such efforts will be continued by future Administrations.

Next article (Friday, June 19): twenty-five hundred years of Government 2.0.