Law.Gov: America's Operating System, Open Source

by Carl Malamud | @CarlMalamud | comments: 13

Public.Resource.Org is very pleased to announce that we're going to be working with a distinguished group of colleagues from across the country to create a solid business plan, technical specs, and enabling legislation for the federal government to create Law.Gov. We envision Law.Gov as a distributed, open source, authenticated registry and repository of all primary legal materials in the United States. More details on the effort are available on our Law.Gov page.

The process we're going through to create the case for Law.Gov is a series of workshops hosted by our co-conveners. At the end of the process, we're submitting a report to policy makers in Washington. The process will be an open one, so that in addition to the main report which I'll be authoring, anybody who wishes to submit their own materials may do so. There is no one answer as to how the raw materials of our democracy should be provided on the Internet, but we're hopeful we're going to be able to bring together a group from both the legal and the open source worlds to help crack this nut.

The idea for Law.Gov seems to be getting a good reception in Washington, D.C. Senator Lieberman, writing on behalf of the Senate Committee on Homeland Security and Governmental Affairs, the committee responsible for the E-Government Act, has already accepted our request to submit our report to the Committee. Additional formal requests to submit the completed report are outstanding.

Law.Gov is a big challenge for the legal world, and some of the best thinkers in that world have joined us as co-conveners. But, this is also a challenge for the open source world. We'd like to submit such a convincing set of technical specs that there is no doubt in anybody's mind that it is possible to do this. There are some technical challenges and missing pieces as well, such as the pressing need for an open source redaction toolkit to sit on top of OCR packages such as Tesseract. There are challenges for librarians as well, such as compiling a full listing of all materials that should be in the repository.

Law.Gov is an outgrowth of 3 years of work we've done at Public.Resource.Org along with our numerous colleagues in the open law movement across the country. There have been a series of piecemeal successes which have demonstrated that there is a demand and a need for more legal information to be more broadly available. I'm hopeful now that a truly national movement may have coalesced and that there is at least a chance we can bring this across the finish line and create a new function inside of government, the publication of America's operating system on an open source platform.

The factor that made this coalesce was the recent Government 2.0 Summit put on by Tim O'Reilly. I gave a talk at that summit about the need to put primary legal materials on-line, and it was gratifying to hear the Deputy CTO of the United States, in his closing keynote, highlight that as one of the issues which he thought the White House should help make real through their "moral authority and convening power." The Government 2.0 Summit was also an example of convening power, and I was very pleased that it was more than yet another conference about open government, it was a forum that brought together people interested in creating real change. Tim O'Reilly, as the Convener-in-Chief, should be congratulated, and I'm hoping that future Summits lead to even more concrete results.

Larry Lessig and Naked Transparency

by Carl Malamud | @CarlMalamud | comments: 5

Larry Lessig had a dream. In this dream, he was standing on K Street, preaching in the dark. Suddenly, a naked posse on Segways went whizzing by, shining their flashlights in people's faces. Bystanders were all blinded by these random lights and lost their night vision. When Larry turned around, the naked posse was racing towards the White House for an open government rally, trailed by a screaming mob of marijuana-smoking birthers.

Larry Lessig wrote up his dream in a cover article for the New Republic entitled “Against Transparency: The perils of openness in government.” I suspect that this article will cause some angst inside the Beltway, where you're either with us or against us. But, before the posse turns into a lynch mob, it is important to give the article a careful read.

(continue reading)

Questions (and Answers!) About the Federal Register

by Carl Malamud | @CarlMalamud | comments: 3

When the White House retweets Cory Doctorow, you know something unusual has happened. As many of you saw, the Office of the Federal Register announced that source code for the Federal Register is now available in bulk—for free—and has been converted to XML. Ed Felten's shop at Princeton created a site called fedthread.org to see what you can do with the data and Public.Resource.Org helped the Government Printing Office in testing early stages of the XML work.

All-in-all, a nice piece of public-private cooperation and an important step towards open source America's operating system, and I figured that was the end of that. So, imagine my surprise when I got a call from the White House saying they were making Raymond Mosley, Director of the Office of the Federal Register (OFR) and Michael L. Wash, the Chief Information Officer of the Government Printing Office (GPO) available just in case there were any technical questions from the net.

I gathered questions from a variety of sources, including on-line discussion groups and twitter, and have been doing email back and forth with both Ray and Mike. Hope this is useful (it certainly has been fun to do)!

(continue reading)

ICANN without restraints: the difficulties of coordinating stakeholders

by Andy Oram | @praxagora | comments: 2

People interested in coalitions and policy-making on a global scale--topics that are increasingly relevant in a world whose borders are irrelevant to carbon dioxide, flu viruses, and other critical entities--need to learn from other organizations that are dealing with these issues. This week brings particularly important news about the Internet Corporation for Assigned Names and Numbers (ICANN), which has been making policy for eleven years under a number of difficult premises:

• It was created hastily and arbitrarily without roots in the communities most interested in its mandate.
• Its concept of stakeholders is boundless, potentially involving anyone who uses the Internet or gets information that has passed at some point over the Internet.
• Its reach is global, and its decisions are affected by issues of language and culture.

(continue reading)

Computerization in Nilekani's Imagining India

by Andy Oram | @praxagora | comments: 0

Imagining India: The Idea of a Renewed Nation promises to occupy a central position in discussions about India as well as the world economy this year. The book was released last year in India, came out in the United States just this past March, and has racked up some prominent reviews recently. Particularly relevant to this blog are the book's observations on computers' role in the economy and society.

Author Nandan Nilekani can speak with quite a bit of authority on computers, having founded and led Infosys, an early success story in modern Indian commerce and a major player in the historic rise of outsourcing.

Imagining India is a huge book with many big agendas; it covers education, infrastructure, environmental challenges, government intervention, and the role of historical narrative, among other things. Biggest among its agenda--and the one that I wager will generate the most debate--is Nilekani's own version of a modern combination of neoliberalism and neoprogressivism that seems to be gaining ground. The general idea is that governments should take a leading role to promote social progress by creating an infrastructure that allows individuals to form their own destinies (good education, good health care, good physical infrastructure, a light-touch form of regulation that ensures quality, and occasional direct welfare payments) rather than preserving oases of protection and easily abused subsidies for particular interest groups, notably unions, small businesses, and disadvantaged castes.

But all that lies beyond the scope of the Radar blog and of my own powers of analysis. I'll just comment on the following points from the book, because they concern the role of computers and because they resonate with trends I see in the U.S. and elsewhere.

• Technophobia shouldn't be assumed
• Internet access goes along with transparency and egalitarianism
• Software leads innovation in other areas

Technophobia shouldn't be assumed

A lot of technologists glibly anticipate that computers and Internet access will be rejected by some group of people who are implicitly labeled ignorant or clueless: racial minorities, poor people, the elderly ("how can you get my grandmother to use this?"), etc. In every case, the key to adoption turns out to be access and sometimes the availability of useful applications. When presented with the opportunity, these populations always prove eager to take advantage.

Nilekani cites one instance after another of rural village dwellers, farmers, taxpayers, and others who quickly grasp what computers and Internet access can do for them. Whether it's the chance to learn English, check crop prices, or pay a utility bill, Indians at all levels come to depend on the computer once it's introduced. (The hard thing, as you might guess, is persuading agencies and local officials to install systems that undercut their power as gatekeepers.) And we've all heard of the Hole in the Wall Project, where Indian kids in slums come to enjoy and figure out how to use computers with little or no adult help.

Nilekani may be citing anecdotes selectively, but his observations echo other reports I've heard about disadvantaged or lagging communities. The problem is not the people, but other factors such as availability, cost, and usefulness.

Internet access goes along with transparency and egalitarianism

One reason the Indian population loves computers, according to Nilekani, is that it attacks favoritism and outright corruption. This advantage matches up with the promise of open government in the United States and other developed countries.

In some cases, Indians are burdened by extremely crude forms of corruption that crumble the instant computers are installed. One example in the book is the registration of changes in land records, which farmers are required to report to the government every year. Agency staff could easily steal land by deliberately filing wrong reports, or extract bribes by delaying the filing until the desperate farmer caves in. But a computerized system takes the staff person out of the process.

Bringing sunlight into government activities in most developed countries has somewhat subtler effects and becomes a more long-term project, but the essence is the same and depends on computerization to work. In the US, we have a lot more control over the stimulus package, thanks to Recovery.gov, than we have over expenditures in Iraq or the bail-out to the finance industry. Indians are similarly learning how to watch over their governments and raise their voices digitally, according to Nilekani.

The sunny role that people around the world are granting to the technologies of going online is not intrinsic to these technologies, because they also lie at the center of modern surveillance, warfare, and regimentation. The benign role is hard won, and represents a collective choice by the public that has adopted the technologies. As Nilekani puts it:

The idea of technology as something ominous and scary that is used by "Big Brother" to control our lives and eliminate jobs has given way to the idea that it empowers, liberates and gives us access to all the services that are due to us, as citizens and consumers.

Software leads innovation in other areas

The reason that the computer industry was the first to take off in 1990s India is that it required practically no infrastructure. Of course, it required a computer, which might require six to twelve months for an import license in those days. It also required electricity, which could be obtained in major cities and supplemented by private generators. (In areas of unauthorized urban growth, the slumlords strung the wires.) So in a regulatory environment that scrutinized and imposed conditions on every allocation of equipment, it was much easier for entrepreneurs to set up a computer firm than any business that had more physical manifestations.

As is well known, the relative independence of computing from physical infrastructure also made Indian companies lucrative in a world increasingly linked by the Internet. Nilekani says that this physical flexibility was also valuable internally, helping IT-savvy businesses cut across the logistical and political barriers that have always geographically segmented the Indian market.

Nilekani seems to believe that there's nothing about the computer industry that's uniquely suited to Indian talents and business acumen. Now that the computer industry set an example, the same advantages have been applied to many other industries. In the 1980s, economists doubted that India could succeed in any industry, and a few years ago they wondered whether India could succeed in any industry except computer services. The evidence is now strong that the country will become a leader in many areas.

Indian industry is just one example where computerization has shown light on a path that social change can take. A worldwide example is provided by the open source movement, which Nilekani mentions only in the most fleeting manner in his conclusion--unfortunately enough, because free software can be a compelling wild card in story of international development, especially as part of a trend I dubbed tech-splicing in another article.

The first open license was a software license (the GNU General Public License). When it was released, the phenomena of allowing unlimited changes and sharing these changes looked like a peculiar aspect of software. But many years later, these ideas seeped out into fields of innovation with a more physical basis, and research by Eric von Hippel showed they always had legs.

Software was also the inspiration for gene splicing and other aspects of synthetic biology, even to the extent that biologists share their innovations in repositories that look like software libraries (check out the BioBricks Foundation).

Finally, the popularity of scripting and other software hacking initiated--or revived, or perhaps just legitimized--a tradition of solving a problem through invention instead of settling for a standardized, commercial solution. The DIY movement found in many areas of the world--including the Indian practice of assembling local motor vehicles called jugaad--makes it more and more likely that products of many types will come out of small, even amateur workshops.

Products of creativity and pure thought embody a freedom that allows them to metamorphose and spread quickly. The added formality and clarity that software brings to these activities doubles the power of that freedom. So my guess is that software will often lead the way in social innovations by a decade or more.

Privacy and open government: conversations with EPIC and others about OpenID

by Andy Oram | @praxagora | comments: 2

A few days ago I proposed a way to offer more privacy to people visiting government web sites. This blog builds on that proposal, which was largely technical, by examining the policy and organizational issues that swirl around it.

My ideas are informed by a discussion I had with Lillie Coney, Associate Director of the Electronic Privacy Information Center. The blog is also inspired by two comments on the earlier blog and brief email I exchanged with one commenter, which intertwine with Coney's in intriguing ways.

As I said in the first blog, my proposal focused on a very narrow question driven by the Obama Administration's interest in revising a memorandum from 2000 concerning the use of cookies in web browsers. The proposal suggested a way to better approach anonymity, but didn't look at the related social and political issues:

• The kinds of privacy and the degree of privacy people want
• When it's appropriate to make visitors identify themselves, or at least to provide some persistent identity
• Whom people trust to maintain identity information

This blog offers a number of points about those issues. The sections are:

• Can the government be your friend?
• Anonymity, pseudonymity, and participation
• Who should run an OpenID server?
• Thought experiment: could federal agencies offer anonymous authentication for whistle-blowers?

(continue reading)

Making Government Transparent Using R

Danese Cooper thinks it will be an important tool in Open Gov

by James Turner | comments: 7

With Open Source now considered an accepted part of the software industry, some people are starting to wonder if we can't bring the same degree of openness and innovation into government. Danese Cooper, who is actively involved in the open source community through her work with the Open Source Initiative and Apache, as well as working as an R wonk for Revolution Computing, would love to see the government become more open. Part of that openness is being able to access and interpret the mass of data that the government collects, something Cooper thinks R would be a great tool for. She'll be talking about R and Open Government at OSCON, the O'Reilly Open Source Convention.

James Turner: Why don't you start by describing where you came from, and you're involved in, and what your interests are?

Danese Cooper: Okay. I'm Danese Cooper. I serve on the board of the Open Source Initiative. I have been serving for the last eight years. And I'm also currently employed by Revolution Computing, which is a start-up focusing on an open source language called R, as in the letter R, that is very useful for analytics and statistical analysis. I'm also an Apache member. And I also serve on an advisory board for Mozilla.

James Turner: One of the two panels you're going to be speaking on at OSCON is on open source and open government. If you could talk a little bit about what interests you about open government and also what open government means to you.

Danese Cooper: Sure. Well, along with a lot of open source people, I got interested in the Obama campaign and in helping President Obama get elected. And part of why he was so compelling was that the vision of how Washington needed to change is pretty close to the way that we think about working collaboratively in open source. The night that he was elected, there was a great little clip on CNET of a Republican commentator actually explaining open source as exactly what I just said. It was a really brilliant little two-minute clip. He pointed at The Cathedral and the Bazaar, that canonical document about how open source works. And he said, "Microsoft is the cathedral. It's their way or the highway. And the bazaar is a bunch of people working together grassroots to collaboratively build the things that they need. And so Obama's basically asking for the government to become open source, and the problem is Washington isn't really like that right now."

So anyway, that's the transformation that has to happen in order for government to really be transparent. To me, open source government is transparent government. There's been an awful lot of shenanigans in recent political history, like the last decade has been pretty crazy in terms of things happening that couldn't be traced back to any source. Even just the way we vote and the way that voting is managed, and the fact that the software that runs the machines that we vote on is not open source so it can't be inspected. And nobody knows quite what it does. There are all of these stories of weird updates to the software that happened right before major elections in states where there are strange results. Transparency, in the same way that it helped the software industry transform, could really help the government transform. So that's what I'm talking about. There's a bunch of other people on that panel. My good friend, Brian Behlendorf, and I co-proposed it. And he's actually taken the next step. He helped found Apache. And he's run off to Washington to work on projects that are interesting to the Obama government to try to figure out how to help them to more open source solutions. And he'll be talking about his progress on that panel. So I think it's a pretty exciting panel.

(continue reading)

Open Gov Is a Dialogue, Not a Monologue

by John Geraci | @johngeraci | comments: 9

At last week's Personal Democracy Forum I had a conversation with someone working for a city (I won't say which city), who was tasked with opening up that city's data. We were talking about the Apps for Democracy contests held recently in Washington D.C., and he explained his feeling about them:

"There were some interesting apps in there, but overall they didn't meet with the mayor's agenda for the city."

Being the non-confrontational person I generally am when in conversation with total strangers, I said "Oh yeah?" and the discussion continued without incident. Inwardly though I was thinking, did he really just say that? My god, this guy is missing the point ENTIRELY.

A city that opens up its data but expects that people building on that data should follow the mayor's agenda is going to fail miserably in its attempt at creating an open system.

Open government is about government as platform. And being a platform means letting people do whatever they like with your tools, letting them build in ways that meet their own agendas, not yours. It's about coming to see your users' agendas as your own agenda. If your users win, you win.

On the other hand, if you force your own agenda on your users, then they don't build anything, and everyone loses.

Open gov is a dialogue between governments and constituents, not a monologue. Everyone gets to decide what gets talked about and what gets built, not just the people with the data.

Everyone who works in the web understands this, of course. I know many of the people who are working on opening up government from the inside get this as well. But this conversation, with a senior-level employee at a government agency, made me wonder how many in government don't understand what open gov means, and what the real value and opportunity is to them. How many think of opening up APIs and such as a way to extend their own reach and increase their office's power?

This is one problem with grafting new ideas about platforms and APIs onto an age-old system rooted in a culture of contracts and RFPs. Can this graft produce a living, thriving hybrid of the two? Or will one necessarily become the subordinate of the other? If the latter is the case, which notion wins out? Open platform or fixed agenda?

Personal Democracy Forum ramp-up: adaptive legislation can respond to action in the agora

by Andy Oram | @praxagora | comments: 0

This article is the last in a series leading up to the Personal Democracy Forum. The first article was posted on June 16 and the second article on June 19.

---

Whole libraries could be filled with writings about the growth of executive power during United States history. The power of the executive branch is likely to increase with technology. But for open government, that growth may be a necessary transition to more public involvement.

As its name indicates, the executive branch is responsible for carrying out the law. The open government movement wants the public to have more say in its own governance, and envisions a more fine-grained implementation of government's role in everyday life. For instance, open government advocates want more citizen input into details such as the siting of physical facilities and the choice of projects for funding. Logically speaking, therefore, the public has more control over implementation if decision-making is shifted from the legislature to the agencies carrying out the law.

Congress should also crack open its hidden chambers; law-making itself could be much more open. It will be interesting to see what comes out of work on a collaborative law drafting project in health care, started by Congressman Anthony D. Weiner of New York. I don't harbor any fantasies, though, that much of his public input will survive the traditional Congressional horse-trading that will follow.

But even the most ideal legislative process ends up with a static document that tries cumbersomely to anticipate every use and abuse of its language. (That's why laws are filled with hedges such as "This passage shall not be construed to...") Legislation is like setting off over rough terrain in a tank. Although the tank can complete the journey, it does so only by flattening everything it encounters.

Some political scientists also think that the executive branch is inherently better suited to understanding and responding to public needs. Here is an intriguing quote from Jane E. Fountain's Building the Virtual State: Information Technology and Institutional Change, summarizing work by Alfred C. Stepan:

Intellectual activities and decisions of civil servants working for long periods on policy questions are arguably more powerful and influential than the sporadic attention of legislators to particular policies.

So I'll take a look at the future of the executive branch, and end this three-part series with speculation about how to build fewer legislative tanks and more Jeeps.

• The executive branch: power and potency
• The legislative branch: how to write laws for an engaged public
• Appendix: the top question asked on Change.gov

The executive branch: power and potency

There's little mystery concerning about why the power of the executive branch tends to grow. Of the three branches of government, it's the one that actually arrives on the scene. It makes decisions about real people and activities on a daily basis and takes responsibility for those decisions.

To act effectively, the executive branch tends to centralize. (Unfortunately, so have many legislative branches in recent decades. The Commonwealth of Massachusetts is run by three people, when they're not fighting indictments or running off to seek other positions: the Governor, the Speaker of the House, and the President of the Senate.)

Because knowledge is power, technology will cause the power of the executive branch to skyrocket over the next few decades. Civil liberties advocates already decry surveillance cameras, wiretapping, the subpoening of information collected by private firms, and the computer analysis that the government applies to all the resulting data. But the data currently available is miniscule compared to everything that will be collected by atmospheric sensors, electronic toll collectors, and various other technologies that are starting to be installed. If Microsoft can produce a game machine cheap enough for the consumer market with face recognition, voice recognition, and full-body motion sensing, what can the government do to track us?

So the power that the executive branch takes on in the political realm will be multipled by the potency it obtains from the data it collects and from ever more sophisticated tools for analyzing that data.

(Strangely, the strict constructionists and "original intent" scholars, who bar judges from interpreting the Constitution broadly, don't apply these restrictions to the ever-expanding executive branch.)

I don't know how to halt this expansion of power. We could open-source the Panopticon by demanding that the public have access to all data collected by public cameras and senors. That won't help, though, because the data will still prove useful mostly to large organizations with the time and expertise to analyze it. And do you want to encourage every budding computer hacker in the country to become a data-mining Nancy Drew?

We could call for strict laws to restrict the collection or sharing of data. You'll still suspect that somebody is collecting information on you. But you'll rest easier because the fear of prosecution will keep them from sharing the data with most of the people you are afraid to have know it.

Still, the reasoning in this article suggests that open government advocates should welcome the shift of initiative away from the legislative branch to the executive one. But only if that's a transitional stage to lodging decision-making more in public hands.

In fact, the other two branches of government and the public had better find ways to implement collective participation, because it may be the only alternative to a resurgent Government 1.0.

To make this shift a positive change, we'll need well-established government/public collaborations that run through the whole cycle I described in my first article. We'll need to make sure that everybody is online and has the training to participate in decisions at the level of their competence and interest. We'll also need to refine polls and discussions to give us confidence that the public's most important concerns and desires rise to the top of the forums.

And when all that's in place, we can start to experiment with adaptive legislation.

The legislative branch: how to write laws for an engaged public

I mentioned at the beginning of this article that legislatures could develop laws in a more transparent manner. But that's only a start. If they could rely on public participation during the implementation of the law, they could write laws that embrace such input.

Laws already include feedback mechanisms. Many call on an executive agency to collect information on the effects of the law, run hearings, and release a report after a fixed amount of time so that the legislature can evaluate whether the law is achieving their goals. This practice could be dramatically extended by involving the public in the implementation of the law at the start, though continuous forums. The feedback loop would be reduced from years to weeks.

Laws also include ways to delegate control. For instance. Community Block Grants are offered to municipalities to spend as they see fit. (My town manager spent several hundred thousand dollars of our block grant to improve a park next to Town Hall, which in my opinion showed dubious judgment during an affordable housing crisis.) The idea of delegation could also be extended to more and more facets of law. What if a virtual town hall debated the expenditure of the town's Community Block Grant?

Critics of government solutions to social problems--usually political conservatives--accuse the law of being too rigid. The legislative process has trouble evolving with the times and responding flexibly to new conditions. Well, with provisions for public comment and group decision-making, laws can be as flexible as we want.

Congress needs evidence, though, that public feedback reflects the diverse needs and values of the population. Public participation must be protected against the complementary evils of capture by special interests and tyranny of the majority, which I have termed the problem of stakeholders.

If the public can live with a law it debates and tweaks as well as it can live with a law designed by Congress, adaptive legislation is viable.

And we need this flexibility, because the really big problems we have to tackle are what computer scientists call "massively distributed." Problems of this type include climate change, health care cost control, a food crisis that leads to rampant obesity in some populations and rampant starvation in others, job creation in an era of reduced staffing needs, and more.

The presence of the term "Collaboration" in the Administration's open government initiative reflects their understanding that they cannot solve the problems by themselves. Nor can technology, the market, or educational efforts--they must all work together. The concept of Megacommunity perhaps reflects the size of the effort (I actually find the "mega" part of the term slightly redundant) but may not even be enough to capture the extent of cultural adaptation required. In any case, adaptive legislation could trigger related efforts and bolster their effectiveness.

Appendix: the top question asked on Change.gov

Although Obama's approach to data sharing is a welcome sea change from the previous administration, the most committed members of his constituency press him to show more transparency about things that particularly matter to them, such as the role the Administration is playing in the financial system and what it knows about torture.

When Change.gov opened a public forum for questions at the beginning of Obama's presidency, the first place was taken by a question about prosecuting US officials suspected of promoting torture. Progressives then cried foul when the Administration failed to answer.

But did Obama really fail to answer? On April 16 he released Bush Administration memos that showed irrefutably that highly placed officials had discarded legal safeguards to institute interrogation practices that were described by these memos in gory detail.

Yes, Obama has resisted investigations of torture before and after this moment. But I am convinced that by releasing the memos he launched a historical process that cannot be reversed. The memos were his answer to the question that the public forced on him in January.

He has made the kind of political calculation that is his hallmark, deciding not to confront Republicans directly with a torture investigation. But if decent citizens keep up the pressure, prosecution will ultimately reach any US officials responsible for human rights violations, just as it did Pinochet and Fujimori. Open government applications do not free activists from the responsibility to engage with every accessible locus of power.

What democracy advocates must remember is that open government is not just a discussion forum. It's a maelstrom of intersecting investigations and competing proposals just as complex as the current political process. In fact, open government can succeed only by integrating with a political process that has a twenty-five hundred year history, even though our goal over time is to transform that process.

Now that the Administration wants to dance, we must learn all the steps. Listen closely: the musicians have already struck up their first round.

Twenty-five hundred years of Government 2.0

by Andy Oram | @praxagora | comments: 7

This article is the second in a series leading up to the Personal Democracy Forum. The first article was posted on June 16.

---

There's been a lot of excitement lately about the term "Government 2.0." Strip away the RESTful interfaces and you see that the new practices in government transparency are just intensifications of things democracies have done for a long time: public comment periods, expert consultation, archiving deliberations, and so forth. So let's look back a bit at what democracy has brought to government so far.

Like any telescoped presentation of history, this one reduces the swirling forces that extend and retract their way through the centuries into a couple near-mythological categories. I do this in the service of evaluating the concepts we toss around when discussing government participation.

Government 1.0: empire

Last year, Boston residents and visitors got the chance to see an exhibit of sculptures preserved from the culture that earned a special role in history as the first major power to exert ruthless control over many peoples: the Assyrians. Other dynasties--Egyptian, Chinese, Babylonian, and Akkadian--were around before the Assyrian empire, but the Assyrians were the ones that set a new standard for cruelty. The fearful image assigned to them in biblical texts also assures them a special fascination for Westerners.

Most visitors to the Museum of Fine Arts were thrilled by the artistic quality of the wall reliefs, human figures, and everyday objects. Personally, I was depressed by the unrelenting scenes of war and cruelty.

Assyria refined a strategy of subjecting cities just outside their borders and using the resulting booty to raise soldiers and provisions to attack the next frontier. Any populations whose subjugation was in doubt would be uprooted and forced to move closer to the center of the empire, replaced in their old homelands by more compliant subjects.

When the court entertained local dignitaries or foreigners (the lobbyists of the day), they walked through "lobbies" adorned with the scenes of carnage that ended up last year at the center of the Boston exhibit. The depictions of chariots crushing helpless civilians and soldiers impaled on stakes gave visitors a clear message: submit or end up the same way. Thus the Assyrians promulgated a "shock and awe" doctrine four thousand years before US troops brought their own version to the same geography.

This went on, with interruptions, for 1,300 years, and established a practice that guided other empires for thousands of years to come.

Some empires were more humane, of course. Empires could provide their inhabitants with protection and stability through currencies, constables, and courts (remember Hammurabi's Code). But all these policies remained subject to the whim of the supreme ruler.

And that is the distinguishing trait of Government 1.0: unchecked power centered in one individual. The reason emperors could stay in power was that they exploited their hierarchies to delegate both power and wealth. As long as governors maintained loyalty to the emperor, they could exert broad powers in the regions under their control and use those powers to accumulate great amounts of money. They in turn delegated power to those beneath them, and so on down through the hierarchy.

What could be more successful than this carrot-and-stick methodology combining vast rewards with threats of terror?

Government 2.0: democracy

There must be something persnickety about the character of ancient Athens. They couldn't tolerate strong leaders. Almost anyone who ever pulled off a major military victory, proved to be a persuasive orator, or got a corner on political power eventually found himself executed or exiled. (The Athenians invented the idea of "ostracism"--a fiercely democratic institution in their implementation, ironically.) Socrates was just one of the later examples of the propensity Greeks showed for bringing down anyone who was widely admired.

So this seems to be a natural setting for a system that grants a voice to a wide range of citizens. The decisions they reach may not be the best, but they're decisions that the political body can follow through on, having been reached democratically. The losers (if they weren't powerful enough to scare the winners) can stick around and try again at the next gathering in the agora.

Greeks recognized from the beginning the problems of democracy with which we are so familiar today. They knew that many votes were bought outright, and that others could be pulled in by smooth-tongued sophists. They also knew their democracy rested precariously on the labor of the slaves and other disenfranchised residents. And that a democracy could become an oppressive empire, using behavior against people next door that it would never tolerate within the walls of its own city.

I like this disturbing contradiction. That's why my web site, identi.ca account, and Twitter account are named after Praxagora, a character in an ancient Greek play that shows both the flaws and the immense power of democratic systems. The name Praxagora combines "action" with "public forum."

Right or wrong, a democratically reached decision--which if properly done, comes into focus as an emergent property of the assembled masses rather than being imposed by one party or individual--has an irreproachable authority. Socrates didn't like democracy, but if we are to believe Plato (who also didn't like democracy), Socrates insisted on obeying the popular will, even at the cost of his life.

We shouldn't hang a halo around direct democracy. In fact, the trend in technology-driven government transparency is not Athenian direct democracy--despite its idealization by some activists--but a tighter agency/public partnership. Today's experiments in public participation go far beyond electing representatives. But even the traditional American political culture consists of more than bills and vote counts. For instance, the executive branch tends to consult regularly with the public, a topic I'll take up in the next article in this series.

As we don digital media and communications--those somewhat ungainly garments we try to mold to human forms--in order to improve on twenty-five hundred years of flawed Government 2.0, we can learn some lessons from those millennia:

• No individual can be allowed to gather too much power, but every individual needs to be heard and to be protected from arbitrary persecution.
• Those who are excluded from the benefits of society will eventually rise up to wreck it.
• The majority is often wrong, and any political system can be abused.
• Good decisions take time, and a willingness to subject the decisions to constant re-examination.
• We need to rise above rhetoric and pursue the ultimate (if ultimately elusive) truth.

Like any useful technology, digital media and communications can help us realize a vision. Government 2.0 is a very old vision. A recognition of what has been achieved and what still challenges us can guide the development of the proper technology.

For instance, we can learn from history to bring the technology of participation to every member of the population and give them the opportunity to learn it, to subject the results of electronic deliberation to review by authorities governed by outside checks and balances, to highlight experts' reputations so they can wield more influence, and to give participants on electronic forums a few cycles of decision-making to work out processes that make effective use of the technology.

Deploying Government 2.0 technology will teach us more about that technology, and about ourselves.

Next article (Wednesday, June 24): adaptive legislation can respond to action in the agora.

Personal Democracy Forum ramp-up: from vulnerability and overload to rage, mistrust, and fear

by Andy Oram | @praxagora | comments: 3

The Personal Democracy Forum will hold its sixth annual conference at the end of this month. The theme, "how technology and the Internet are changing politics, democracy, and society," has been central to O'Reilly's work over the past few years (and a theme on which we're holding a summit of our own in September). Over the next two weeks I'll write three blogs on the Radar site to get some of my current thoughts off of my chest, clearing some space so that when I get to the PDF conference itself, my blogs can focus on its events and statements made by its participants.

This blog covers:

• The government participation cycle: if you want to dance, sir
• Vulnerability: a reason to put brakes on outgoing information
• Overload: a reason to put brakes on incoming information
• Rage, mistrust, and fear: inhibitors of the government participation cycle

The government participation cycle: if you want to dance, sir

The grand vision for government/public collaboration is a set of feedback loops that intensify the influence of the collective will on government policy. A feedback loop might consist of a cycle like this:

1. An agency (or less likely, a legislature) posts data in a downloadable format through a flexible API and announces a call for applications.
2. Companies and public interest groups define goals and put programmers on the task.
3. The public uses the resulting applications to generate data and share it with the agency.
4. The agency sets policy or changes direction in response to the data.

At any step, a failure by any of the responsible actors to follow through will leave the process hanging and discourage future projects in open participation.

This doesn't mean every project needs to include all four steps. The public may benefit from government data without offering feedback, and programmers could put their work under an open source license or into the public domain for the benefit of the government or members of the public without asking them to share more data. Agencies can also use programs to improve internal coordination instead of working with the public. But the full four steps serve as a canonical model for government/public collaboration.

Successful examples already exist for each step. As I write this, Data.gov has 261 data sets and 30 tools; thousands more data sets are promised soon. Appeals for donations of code, such as Vivek Kundra's Apps for Democracy in Washington, DC and the Sunlight Foundation's Apps for America, show that coders will play their part, at least in the current atmosphere of enthusiasm for the new initiative. And the public has responded to requests for data.

But at the federal level, we need to dance a few rounds of the full cycle before feeling confident that open processes are fully entrenched. I'll return to this theme in the last section of this article. The cycles of public participation will teach lessons, of course, that feed into a still larger cycle of constant experimentation and improvement.

As public participation moves forward, it's worth remembering that resistance to the free flow of incoming and outgoing information is not irrational. The resistance spring from healthy coping mechanisms learned by individuals and organizations learned over their lifetimes. I have already published and solicited comments on a list of fundamental questions on government participation; in this article I describe two such issues that play a special role in resistance to information sharing.

Vulnerability: a reason to put brakes on outgoing information

A couple months ago, I read a stirring report from a federal agency manager trying to sound out the Administration concerning how much the agency ought to reveal. The manager was stunned and inspired by the response of Bev Godwin, a prominent director at the White House and General Services Administration, who advised talking about the bad things as well as the good and soliciting negative as well as positive feedback.

Vulnerability is the keystone of transparency and openness. Online forums, if they are run democratically and competently, encourage vulnerability through a combination of self-correcting mechanisms:

The right to respond

Anyone criticized in a forum has repeated chances to defend himself at length. If the forum includes a rating system, persuasive arguments and well-chosen facts will float above false accusations as well as flaccid excuses from the accused.

Support networks

Proponents of each side pile on to each debate, turning it into a community issue and diluting the personal biases brought by the people who began the debate. A bit of a mob scene can erupt at times, awakening the risk that the losing side will walk away in a huff while sensitive community members flee the fury. But as long as participants value the community over partisan agendas and prefer honesty to grandstanding, the community comes out stronger, more aware of its options, and ready to integrate what it has learned into further action.

Community memory

Forum members recognize when old debates are re-ignited, and can fill new members in on the history. They can also predict the way prominent participants will line up on an issue. Debates are thus tighter and more quickly resolved.

A propensity for truth

These traits all end up privileging accuracy and making it harder (although not impossible) for bad judgment to prevail through false claims, manipulative demagogy, appeals to group solidarity, and the other tricks used by insincere factionalists.

This list may present online forums in a bit too rosy a light. But they do permit social norms that protect vulnerable people, even if the norms don't function perfectly. The real problem comes when words leave these forums and end up in other environments not subject to the same rules.

Government staff have already witnessed too many negative experiences in traditional, non-virtual settings. They have seen what happens when a comment is taken out of context and bandied about in the broadcast or print media, introduced into court testimony, or used as ammunition in partisan debates. They know that comment posted on the Web can be fodder for the same opinion machine--and are in fact even more dangerous because the Web makes them more visible.

That's not fair. It's very hard for anyone outside an agency to judge why it came down on one side of a debate or what that decision's long-term effects will be. Most agency actions are a complex fermentation blending the data that was gathered, assessments of the data's accuracy, assessments of the possible trends indicated by the data, consultations with the public (yes, outsiders are routinely consulted), judgments about Congress's intent, judgments about the interests of the Administration, and more. But groups with a cause like to ascribe one-dimensional reasons for key agency decisions and mine public statements for corroborating evidence.

This doesn't mean that all agencies are honest and act in the public interest. Plenty of bad government decisions have been made under pressure from well-organized special interests or to pay off political donors. One role of civil society is to expose these influences--that's what open government and the Personal Democracy Forum are all about. So we want more of these online forums. But we also need to protect the agencies whom we expect to use the forums. To encourage the necessary vulnerability, we have to combat those who abuse the results.

Journalism is starting to incorporate its own feedback loops and open its pages. Elections and policy debates are also monitored by the blogosphere. So some forums are becoming friendlier to the cause of vulnerability (the court system is unlikely ever to change). But it will be a long time before it's safe to lay out one's thoughts in an open, self-policing community.

Overload: a reason to put brakes on incoming information

The previous section mentioned the possibility of a "mob scene," and if people putting out information must be able to tolerate being vulnerable, those requesting input from the public have to deal with a potentially low signal-to-noise ratio.

We need not look far for an example. Take last month's brainstorming session on open government, launched by the White House and the Office of Science and Technology. It drew over 1,000 submissions in a single week. (Even more are on the site now, but they arrived after the official close of the session.)

The thousand submissions offered quite a smorgasbord for a group led by the new Deputy Chief Technology Officer for Open Government, Beth Noveck, to spoon through. They ended up with many intriguing ideas. But the gathering of ideas was simply a suggestion box, not real crowdsourcing. The web site offered no tools for editing, combining, and culling entries (and there would be inadequate time to use such tools anyway). The only aspect reminiscent of group behavior was a casual and anonymous rating system, which played little role in the results.

And that's a relief. After all, how many Americans would be able to assess the Office of Open Government created by Florida Governor Charlie Crist, or the potential for Cooperative Research and Development Agreements to help convert government data and applications to open source? Both of these projects earned a place in the results, even though the Florida model got only 24 votes and the Cooperative Research and Development Agreements only 46. (Although they might have conceivably been mentioned in an earlier brainstorming session conducted among government workers, I couldn't find them in the publicly posted comments.)

In response to a question about the voting, Noveck wrote me, "We wanted to encourage the National Academy of Public Administration to try different voting techniques. They started out by allowing voting by unregistered users, and later restricted it to registered users. Given the change, we didn't want to disadvantage anyone who participated. Consequently, we viewed the voting as informative but not determinative. On our weblog, only registered users can vote on comments."

As her statement indicates, the second phase of this transparency project has already sprouted more of the checks and balances found in mature discussion forums. We can expect the Administration to wend its way toward systems that gather useful opinions from self-organized groups of qualified commentators, the model pioneered by Noveck in her Peer to Patent project.

But will the White House have the time and resources to establish a foothold for a solid and lasting open government program? That depends on public tolerance for the Administration as a whole.

Rage, mistrust, and fear: inhibitors of the government participation cycle

Everyone knows that productive collaboration can't take place under conditions of rage, mistrust, or fear. Americans unfortunately are suffering from all these feelings right now.

Their rage has been directed at the heads of the financial industry. No peasant at the time of the French Revolution felt more hatred for Marie Antoinette than some of the comments I've seen about AIG. In addition, the current conditions of recession and financial uncertainty breed mistrust toward all three branches of government, and fear toward anyone who could seem to wangle an extra advantage over other Americans.

I'm not going to factor in the recent murders of law enforcement officers, Dr. George Tiller, and others because I'm sure the hate crimes were caused by lots of diverse factors, and it's unclear whether they represent a widespread cultural movement. We have plenty to worry about just by considering problems that will undeniably have a broad impact on Americans.

Over the coming year, lots of homes will continue to be foreclosed (because Congress failed to put a system in place to stop them), a blight that hits many neighborhoods like a dry Katrina. This ongoing crisis will be joined by credit card crisis (because Congress's bill didn't do much to stop that either) and perhaps already a student debt crisis. The Administration has its own challenges, waging two untraditional wars that nobody knows how to win and tinkering with a global financial system that always cracks its casings.

Open government doesn't deserve to be at the mercy of current political controversies. It did not originate with the Obama administration, and it doesn't require a Democratic Party philosophy. The George W. Bush administration took some steps toward open government (often forgotten amongst all the complaints over their unsavory maneuvers and information withholding). The Bill Clinton administration took steps too. But Obama is making it a centerpiece.

This gives us more hope than ever for openness, but ties its fortunes to the larger sphere of activities by the Administration and federal government.

To establish a foothold, openness needs some early, impressive success stories. Federal CTO Vivek Kundra has said his initiatives will prove themselves by saving money, although that certainly isn't his sole aspiration. If the Administration can land a few universally recognized successes--budgetary or otherwise--and especially if it can run through the whole cycle I laid out at the beginning of this article, such efforts will be continued by future Administrations.

Next article (Friday, June 19): twenty-five hundred years of Government 2.0.

The Four Pillars of an Open Civic System

by John Geraci | @johngeraci | comments: 15

Everyone is talking a lot about open government and transparency these days. It's exhilarating stuff, and it's even more exciting to see governments get behind it, creating sites like data.gov in the U.S. for the public to access government information via APIs. But every time I hear someone say something like "our organization is really into transparency" (which is often) it sounds odd to me. It's only talking about a part of the puzzle, not the whole puzzle. What we really want (or what I really want anyway) is not simply government transparency, but an open civic system - a civic system that operates, and flourishes, as a fully open system, for whatever level we happen to be talking about - federal, state, city, neighborhood, whatever. And transparency is a big part of that open civic system, but it is still only one part.

In fact there are four parts to a functioning open civic system. These are:

Government to Citizen (G2C). This is what people speak of when they talk about transparency and open government data. It's the idea of creating open pipelines for data directly from government and gov't agencies to whoever is interested in receiving it. G2C gets you accountability - watchdog groups suddenly have easy access to the paper trails for everything that is going on, etc. It also gets you things like transit schedules, minutes from meetings, and zoning data - things that can be built on by third parties to make the civic system work better. G2C is critical stuff, but without the other three components in place, we can't make the most of this open government data. What we need is not simply a pipe of open data, we need an ecosystem of open civic data, all interconnected, all flowing every which way. That's what the other three "pillars" of an open civic system gets us.

Citizen to Government (C2G). The counterpart to G2C. This is the idea of creating open pipelines from the people directly to the government - hopefully with someone listening on the other end. Adding C2G to G2C completes the circuit and makes open government APIs and such that much stronger - it takes what was a uni-directional data flow and turns it into a feedback loop of information, input and output. At the city level, C2G is taking shape right now in the form of Open 311 - a open API that anyone can build on that allows residents to create "problem tickets" for their city to address one way or another. Washington D.C. is currently launching an open 311 API, and I expect more cities will follow suit soon. Other examples of C2G include UK's FixMyStreet and SeeClickFix.com from New Haven, Connecticut, both sites with a huge amount of potential. There are a million different, nuanced ways C2G could be played out, at the local, state and federal levels.

Citizen to Citizen (C2C). Okay so now we have both open G2C and C2G data flows going, and that's great - huge amplification of civic activity, great realization of efficiency with regards to interaction between government and people. But there are all sorts of ways to improve civic life that don't really need to involve the government at all - what about those things? That's where Citizen to Citizen, or C2C, data flows come in. C2C is the citizens' brigade of data flow - it's the people doing it for themselves, whatever "it" happens to be. Clever Commute, in New Jersey, is one example of a great C2C data flow. Everyone who commutes by train into NYC subscribes to the Clever Commute feed, and then notifies each other of what the current delays are, and where, each morning. The system works better than anything New Jersey Transit has been able to pull together, and at a cost of essentially zero. This is the great thing about C2C - it is added value to the civic system at no additional cost to the system itself. The cost to operate C2C is passed on to those who are using it, and spread out amongst individuals, to the point where the costs become negligible. Instead of New Jersey Transit coming up with a system that knows how late each of its trains are at a cost of millions of tax dollars, the users of Clever Commute bear the cost of the system, and it costs pennies for each user to operate (the cost of sending a text message). C2C is a huge value-add on top of G2C and C2G, and as governments consider how to get increased services in these recessionary times, I expect C2C to be huge - once governments get used to the idea.

Government to Government (G2G). Lastly, the square is not complete without open Government-to-Government data flows. Entities within governments should have easy, open data exchange with each other, without having to issue a request, parse something out of a PDF, and so forth. The ability for, say, the NYC Department of Health to get data from the Los Angeles DoH in realtime, without having to talk to anyone or issue a request could be a huge asset. Or think of the efficiencies that could be gained if the NYC DOT were able to exchange realtime data with the NYPD. If these examples sound vague, it's because G2G is the "pillar" I know the least about, having never worked in a government agency. From what I've learned though, it seems to me that there could be a huge increase to civic utility with a little bit of thought about an open G2G system.

And of course you can blend these data flows and come up with hybrids all you like. DIYcity's SickCity, for example, is basically a C2C tool in its present, basic 1.0 incarnation - it detects instances of residents in your city saying they're sick, and passes that news on to other residents. But a more sophisticated version of the tool would also pass that information on directly to the Department of Health when relevant, and would also, optimally, accept data from the DoH to pass that back to residents. Suddenly it has gone from a simple C2C tool to a tool that is C2C, C2G and G2C. Now we're talking about interesting stuff. Each additional channel of data makes the system exponentially more valuable.

With all of these systems properly developed and engaged, our civic systems - local, regional, federal - should bloom and transform into the properly modern, Internet-age things they ought to be. This will translate to increases in efficiency, greater innovation and rate of change, better adaptability, and greater resilience, in addition to other advantages. To get there though, we've got to get beyond thinking simply in terms of transparency and government APIs.

Local forums to implement high-speed networks (broadband): proposal open for votes

by Andy Oram | @praxagora | comments: 10

I've posted a proposal titled Local forums to implement high-speed networks (broadband) to a forum on open government put up by the White House. I ask this blog's readers to tell other people who might be interested, and vote up the proposal if you like it.

The Open Government Dialog site where this proposal appears is part of the White House's implementation of the Memorandum on Transparency and Open Government that Obama signed on his first day in office. Hundreds of ideas have already been posted. Many are very specific and some look quite worthy, but I think mine stands out for the reasons listed in my justification:

First, one of the Administration's major goals is to bring high-speed networking to every resident of the country.

Second, this goal is fundamental to the other goals in the Memorandum on Transparency and Open Government. Members of the public need continuous access to the Internet and the ability to handle video and sophisticated graphical displays in order to make full use of the resources provided in open government efforts.

The local community aspect is also crucial, for reasons I list in my justification.

Many readers will note that the people who need my proposal the most the ones who have the most trouble participating in the forums--people who can't afford computers, who have access only to intermittent dial-up Internet access, etc. I deal with this ironic problem in the proposal in several ways (public terminals, face-to-face meetings, partnering with newspapers and television).

Because the formatting came out a mess, I'm reprinting the proposal below.

---

Local forums to implement high-speed networks (broadband)

Municipalities and regions undertaking projects in high-speed networking be encouraged to create online forums that:

• Post regional maps showing the demographic features, geographical features, patterns of network use, and technological facilities relevant to the project

• Accept proposals, provide comment and rating systems, and run polls

• Provide public terminals and low-bandwidth versions of data, so that people who are currently on the disadvantaged side of the digital divide can offer input to help cross that divide

• Are supplemented by face-to-face gatherings

• Collaborate with newspapers and with television and radio news programs to publicize proposals, meetings, and opportunities for public comment

• Create visitor accounts, perhaps with validation procedures for determining local residence, and allow visitors to identify their expertise and credentials

• Provide tools for mapping proposed facilities and for calculating the reach, bandwidth, and costs of proposed facilities

• Provide data about ongoing deployments in standardized, open formats on maps and in downloadable form

The federal-level initiative can support these efforts by:

• Mandating the types of information that participating municipalities and companies should provide, such as the capabilities of current facilities, statistics on current usage, demographic information such as income and connectivity on a neighborhood basis, and detailed implementation plans with measurable milestones

• Funding the development of software tools, such as programs that can estimate the quality of wireless coverage for different terrains, or the time period required to recoup the costs of building out networks

• Providing formats and quality standards for the data provided

• Publicizing successful initiatives, the tools they used, and their best practices

Why Is This Idea Important

High-speed digital networking (also known as "broadband") should concern open government advocates in two ways.

First, one of the Administration's major goals is to bring high-speed networking to every resident of the country.

Second, this goal is fundamental to the other goals in the Memorandum on Transparency and Open Government. Members of the public need continuous access to the Internet and the ability to handle video and sophisticated graphical displays in order to make full use of the resources provided in open government efforts.

Why do I stress the local nature of these forums?

All networking is (on one level) local. Given the limited resources available for any network deployment, and the trade-offs that must be made during plans, decision-makers need to take into account local demographics, geography, topology, social and economic priorities, and existing facilities. Here are just a few examples the many local issues typically considered:

• Which neighborhoods are already relatively well-served or poorly served

• Where it's cost-effective to string fiber, versus serving a neighborhood through a high-bandwidth wireless solution

• Whether there are existing facilities and lines that could be repurposed or upgraded for high-speed networking

• How many public funds to invest and which private firms to contract with to provide infrastructure or Internet service

• Whether a non-essential service, such as wireless for spots where tourists or businesspeople congregate, could generate enough new jobs or revenue to be worth an investment

• What public and private partners and sources of investment are available

• Whether people in potential new markets have the desire and education to use new network services, and how to create the conditions under which the populations would use the services

Innumerable issues like these require local knowledge and judgment. That is why many innovative and successful initiatives to provide digital networks have been launched by local governments or local private service providers.

Local collaboration to promote network penetration can also build bonds that support local communities in other ways. The global reach of the Internet has long been stressed, but the role of digital networks in connecting people within geographical communities and improving their way of life may be even more important and is beginning to be recognized.

Although complex, the issues are no more complex than many other issues being considered for implementation of the open government directive. With proper organization and support, community members could make these decisions and monitor their implementation.

Local community forums also attract participants more easily than geographically distributed "communities of interest." People are likely to respond to the invitations of friends and neighbors, and to be more loyal to the forums when they know the participants personally. So local forums are good ways to initiate the general public to the notion of transparent and participatory governance.

A note on current federal broadband initiatives

The American Recovery and Reinvestment Act (ARRA) includes a Broadband Technology Opportunities Program (BTOP), operated through the NTIA, that creates a 4.7-billion-dollar program to promote broadband, particularly for unserved areas and populations.

The implementation does not involve any of the innovative aspects of the open government directive, such as collaborative online forums and data exposed through open formats and APIs. Like other programs in ARRA, the focus on providing a fast economic stimulus led to a schedule that does not accommodate time to set up and accept comments in this manner. A public comment period was held from March 12 to April 13, 2009, and proposals must be submitted by September 2009.

The FCC adopted the goal of expanding broadband access many years ago, and cites this goal in many opinions concerning competition. The FCC also continues to offer funds for broadband under the Universal Service Fund (USF), which was expanded by the 1996 Telecommunications Act to include Internet access. The USF does not involve public online forums or open data access.

The FCC also plans to publish a national broadband plan by February 2010. Because the funds from BTOP will probably be disbursed by then, this plan could be a locus for the kinds of forums describes in this proposal.

Quick disclaimer: broadband adoption is hard to measure--it depends on such fuzz factors as the minimum speed defined as "broadband," the difference between potential and usual speeds, and uncertainties about actual availability versus official penetration rates--but recent estimates suggest that half of the United States population has always-on, high-speed network access. Although this reflects a substantial increase in recent years, it still leaves the US behind many other developed nations. Further improvements will require more intensive planning and careful resource allocation, as we try to extend adoption to populations with fewer resources or geographical challenges.

Summary of benefits

• When the public can evaluate the options available to their community and the trade-offs required, they can reach agreement on a digital networking policy that reflects the values of many constituents and communities.

• Tools for measuring the impacts of different proposals can help everyone in the community agree on what trade-offs exist, and provide a factual basis for decision-making.

• Technically trained members of the community can use the measurement and visualization tools on the forum to educate those who are less technically sophisticated and ensure that everyone has the opportunity to make valid and appropriate input.

• Progress in implementation can be followed by the public, who can demand accountability in spending and results.

• Collaboration in building local networks can lead to continued collaboration in using those networks to improve economic, educational, and policy initiatives in the communities. They can also give visitors the skills and interests to join larger, national efforts in fulfillment of the Memorandum on Transparency and Open Government,

• Standardization and information sharing between communities can help later communities reach successful conclusions more quickly and with less wasted effort.

• Finally, the public participation fostered by local forums can educate the public about telecommunications issues that have a national or even international scope, such as expanding major access points, fostering technological innovation, and changing national policies and laws.

Update: FCC discusses broadband

May 27: About the time I submitted this proposal, the FCC released a report titled Bringing Broadband to Rural America: Report on a Rural Broadband Strategy. See my blog on its relation to my proposal.

Four short links: 21 May 2009

by Nat Torkington | @gnat | comments: 0

1. Us Now -- UK documentary, available streaming or on DVD, about how open government and digital democracy makes sense. It's good to watch if you've not thought about how government could be positively changed by technology, but I don't think it's radical enough in the future it describes.
2. It's Gonna Be The Future Soon -- great video for the Jonathan Coulton song that's the Radar theme song, my theme song, and probably works well as an anthem for most of us goofy future-loving freaks. Taken from the DVD of a live show. (via BoingBoing)
3. Jetpack -- Mozilla Labs' new extension system. Mozilla Labs is building quite the assemblage of interesting hack tools, and it's interesting how significantly they're aimed at the developer and encouraging lots of add-ons and after-market extensions for the browser. I wonder whether this is a deliberate strategy ("community will beat off Chrome!") or whether it's a simple consequence of the fact that Mozilla is a developer organisation.
4. Sci Bar Camp -- Science topics, Palo Alto, 7 July 2009.

The Future of Our Cities: Open, Crowdsourced, and Participatory

by John Geraci | @johngeraci | comments: 15

Guest blogger John Geraci has spent the last six years making life in cities better with the use of web technologies. His latest project, DIYcity.org, has web developers and urban planners all over the world teaming up to create open source tools for residents of cities everywhere. Prior to DIYcity, Geraci co-founded the hyperlocal news network Outside.in.

Back in January, the city of Los Angeles announced a gap of $433 million for their 2009 budget. Instead of just cutting services however, LA Mayor Antonio Villaraigosa took the unusual step of posting a survey online for residents of the city to fill out. For each category of city service, the survey asked residents, "what program would you reduce to help balance the budget?", followed by an itemized list of services they could choose from.

It was in one sense a remarkable sign of the new openness and desire for participation sweeping government all over the U.S.

In another sense though it begged a larger question: if you're going to involve city residents in these issues, why stop at asking people which services they would like to cut? Why not go a bit further and ask them for input on how to keep these services, while making them leaner, more efficient, and smarter? And why not then ask for their help in making those changes happen?

These are questions cities everywhere should be asking today, as they find themselves faced with the challenge of gigantic budget shortfalls brought on by the recession. The conversation about the future of our cities should involve the people living in those cities. But it should not be about which services to eliminate, it should be about how to reinvent these services as modern, efficient things, how to make them work at a fraction of their current cost, and, while we're at it, how to make them better than they are now.

Why? Because cities don't have the money to improve, or even sustain these services on their own. Because people have good ideas, often more innovative than the ones coming from the cities themselves. And because increasingly, people have the means to actually build and implement these services - not as centralized, closed, top-down systems we think of as public services today, but as distributed, participatory web-based systems built using data open to all.

(continue reading)

Transforming the Relationship Between Citizens and Government: Making Content Findable Online

by Vanessa Fox | comments: 9

Thursday on this blog, Congressman Honda asked, "how can congress take advantage of web 2.0 technologies to transform the relationship between citizens and government?" He noted that "A dramatic shift in perspective is needed before that need can be met. Instead of databases becoming available as a result of Freedom Of Information Act requests, government officials should be required to justify why any public data should not be freely available to the taxpayers who paid for its creation." He asked for input on what web 2.0 features he should add to his website to take advantage of today's online world.

The most important feature government web sites can add isn't really feature at all. But it would absolutely transform the relationship between citizens and government and make an amazing array of public data available. What's this magic feature?

Make government web sites search engine friendly.

How we look for information
Search is the primary navigation point for the web. Often when citizens look for government information, they start at a major search engine. They don't think to themselves, I need some information on vitamins, so I'll just go on over to the Office of Dietary Supplements at http://dietary-supplements.info.nih.gov. And then I need to make sure I'm eating a balanced diet, so I'll just check out http://www.nutrition.gov from the National Agricultural Library. And before I head to the grocery store, I'll make sure I understand how nutrition labels work from the information provided by the Center For Food Safety and Applied Nutrition at http://www.cfsan.fda.gov. Mostly, they go to Google and type in [food labels]. And in some cases, this works perfectly and the information appears.

But when information from government web sites doesn't show up on the first page of results for those searches, the information may as well not exist at all. For instance, an amazing amount of data exists from the U.S. Census Bureau, but it's inaccessible from search engines because it's locked behind JavaScript forms and the content itself doesn't use language that searchers would use. If I search for [98116 census data], results from census.gov are nowhere to be found.

Obstacles to being found in search engines
One problem is that the U.S. Census Bureau pages don't use zip codes to denote regions. They use tract numbers. Even if the pages were written in plain language searchers might use, search engine crawlers couldn't get past the JavaScript forms to access the pages.

Try doing a search using the same terminology as the U.S. Census Bureau, and you start to see the problem with the site's findability. Take [census track 97.02]:

None of those results lead to these handy details:

In addition to being buried behind JavaScript and containing little language people would actually search for, it's hidden in a popup with a URL like this: http://factfinder.census.gov/servlet/IdentifyResultServlet?_mapX=281&_mapY=216&_latitude=&_longitude=&_pageX=442&_pageY=554&_dBy=100&_jsessionId=0001cv7n8rWxjslrmI9aRw5nr-V:134a7lbrs">http://factfinder.census.gov/servlet/IdentifyResultServlet?_mapX=281&_mapY=216&_latitude=&_longitude=&_pageX=442&_pageY=554&_dBy=100&_jsessionId=0001cv7n8rWxjslrmI9aRw5nr-V:134a7lbrs

The server appends a session ID to the end of the URL (the portion beginning with "jessionsId"), which is tied to an individual visitor session and times out after 60 minutes. If I share that URL on a social media site, email, or in this blog post, anyone who tries to visit it just gets a "session as expired" message. It goes without saying that this kind of URL can't be indexed by search engines no matter how sophisticated they become.

(continue reading)

Request for ideas: Crowdsourcing the Evolution of Congressional Websites

by Mike Honda | comments: 21

Guest blogger Rep. Mike Honda, D-San Jose, serves in the U.S. Congress on the House Appropriations subcommittee on the Legislative Branch.

Tim recently asked readers of this blog to help provide me with guidance on the best way to make official legislative databases available to the developer community. The question, which also made its way onto Slashdot, led to a wealth of proposals, some of which I am considering developing into new legislation. Following on the success of that initial conversation, I’d like to ask for your guidance once again.

How can Congress take advantage of web 2.0 technologies to transform the relationship between citizens and government? Instead of viewing the public as a customer for services, I believe that we should empower citizens to become our partners in shaping the future of our nation.

Sites like stimuluswatch.org, for example, have shown how the public can advise officials on which elements of the economic recovery program are most effective in creating jobs and resuscitating our struggling economy. Together we can identify and cut ineffective government programs and simultaneously support cost effective initiatives that maximize Return On Investment.

Websites like these only become possible when government data (in this case a list of project requests from the US Council of Mayors) is repurposed to enable public participation. Until more government databases become available, however, the full potential of web 2.0 technologies will remain unfulfilled. A dramatic shift in perspective is needed before that need can be met. Instead of databases becoming available as a result of Freedom Of Information Act requests, government officials should be required to justify why any public data should not be freely available to the taxpayers who paid for its creation.

As one leading e-government expert recently advised:

Free your data, especially maps and other geographic information, plus the non-personal data that drives the police, health and social services, for starters. Introduce a ‘presumption of innovation’ – if someone has asked for something … give them what they want: it’s probably a sign that they understand the value of your data when you don’t.

My constituents in Silicon Valley understand how opening up data can catalyze dramatic innovation, and I recently enacted legislation to provide free public access to legislative databases with that goal in mind. It is my hope that this information can foster the development of initiatives to empower the public to collaborate with and provide advice to Members of Congress. No longer will individuals simply petition their representatives – instead you should be our most valued advisors.

Government 2.0 is an achievable goal, and together we can make it a reality. In fact, I recently began a comprehensive redesign of my website with the goal of developing new and unprecedented ways of collaborating with my constituents.

To solicit ideas for the new website, I sent my Online Communications Director to a conference to lead a website brainstorming session. That conversation resulted in several intriguing proposals, including the suggestion that I post my hearing schedule for the week so that my constituents could propose questions for me to ask of witnesses.

The success of that session, and the quality of your answers to the last question I posted here, gives just a hint of the possibilities that can result from greater partnership between elected officials and the public. While I may not be able to implement every idea that is suggested, I do plan on providing a list of the most innovative ideas to my fellow Members of Congress.

What features could I implement on my website to tap into the wisdom of the crowds?

• Propose an idea
• View submitted ideas
• Join our conversation on twitter using the #honda2.0 tag

With your help we can empower the public to partner with Representatives in improving the policies of our nation. Let’s begin making Gov 2.0 a reality.

- Mike

Four short links: 2 Mar 2009

by Nat Torkington | @gnat | comments: 2

You open the letterbox. Inside are four interesting links covering politics, mobile business, Javascript, and MySQL:

1. The Minimal Compact (Adam Greenfield) -- a manifesto on "open source constitutions for post-national entities". Sample: "Of interest are alternatives that are designed from the beginning to: Ensure the greatest freedom for the greatest number, without simultaneously abridging the freedoms of others; Permit individuals with common goals and beliefs to act in their own interest at the global level and with all the privileges afforded nation states, even when those individuals are separated by distance; Provide robust resistance to attempts to concentrate power, and other abuses of same."
2. Wireless carrier financial results (Matt Gross) -- Matt extracted the data from GigaOm's article on wireless carrier finances and presented them in simple tables for comparison.
3. jQuery Sparklines -- elegant micro-charting library.
4. How Friendfeed Uses MySQL to Store Schemaless Data -- another entry in the post-normalized database stakes. "We like MySQL for storage, just not RDBMS usage patterns."

What Does It Mean To Be An Internet President?

by Joshua-Michéle Ross | @jmichele | comments: 11

FDR was our radio president, JFK was our television president and Barack Obama will be our Internet President.

Quietly at noon yesterday, as the world was fixated on the televised inauguration of Barack Obama, some obscure IT managers flipped a switch (metaphorically) and transferred Change.gov to Whitehouse.gov... While the inauguration spectacle was inspiring and the speech lived up to its promise, Whitehouse.gov is the herald of bigger changes. Government is entering the Internet age and Barack Obama is our first Internet president.

What does that mean?

Each medium has a unique signature (McCluhan would say it’s “message”); a set of characteristics that have a more profound influence on society than the content that flows through it. Television, for example is a capital-intensive broadcast medium requiring a passive viewer. These “pacifying” characteristics are one reason why Al Gore spent time during the Web 2.0 Summit to decry television’s corrosive effect on the democratic process.

Our democracy was constructed well before television (much less the Internet) in an era when the dominant technologies were the printing press and the horse-drawn carriage (Placement of district courts was based on a half-day’s horse and buggy ride to provide each citizen access to court services and the interregnum between presidential transitions took months in order to allow distant presidents to prepare and make the journey to Washington). These technologies invested themselves into every construct of our government.

So how do we re-imagine democracy in the age of networks, where the dominant metaphor is the hyperlink, and the printing press has yielded to the blog; where productivity (open source and crowdsourcing) and decision-making (idea exchanges, prediction markets, online voting etc.) has marked a shift in power from the core to the edge? We are at least a decade away from the answers. Here are a few general principles for democratic government to better serve us in the age of networks.

Listening beats Talking In the network - listening is a prerequisite to learning. It is the critical precursor of everything we do - the beginning of joining conversations, building trust, learning and developing relationships. In a networked democracy, good government (at every level) will need to find avenues available to listen and respond to its citizens. We saw some of this evidenced at Change.gov (where prosecution of torture was the foremost concern on peoples' mind) and in Tim Kaine’s video response to questions on the future of the Democratic party.

Open beats Closed There is more untapped talent outside any organization than inside (government included). Open beats closed points towards two fundamentals: (1) getting beyond a paternal sense of government (what government does for me) and towards a participatory model of government embodied by Mybarackobama.com and subsequent incarnations, and (2) open, standardized data that enable citizens to remix and add their creative energies. Washington D.C. is doing a great job in this arena. The other side of the coin are operations like MySociety and Frontseat.org that are looking to work with data that is already available to improve civic life.

Leadership Counts More Than Ever Although power has shifted from core to edge, vision and leadership counts more than ever before. Our generation’s notion of leadership will differ from the past (“Chainsaw” Al Dunlop anyone?). Consistent with the medium, leadership does not emanate from one highly leveraged point. It is a call to leadership at all levels of society. It is an open call to participation. In this regard, Obama has been a powerful model for a new generation of leaders ….

What do you think it means to be an Internet President? What do you think are other implications of the Internet and technology on Government and democracy?

Put change.gov Under Revision Control!

by Tim O'Reilly | @timoreilly | comments: 66

Last week, the New York Times wrote about Changes at change.gov:

The policy section of the transition site was removed without notice just days after Change.gov went live shortly after the election. At the time a spokesman for the Obama-Biden transition effort said they were “re-tooling” it.

There was an almost instantaneous outcry from bloggers and other advocates of transparency in government who noticed disappearance. At least one site posted a complete archive of the old Agenda pages. (Increasing transparency, by the way, is a key feature of Mr. Obama’s government reform agenda, according to the site’s “Ethics” page.)

The changes, as it turned out, were mostly to tone down the partisan politics of the policy documents as published during the campaign. But the lesson remains: when public documents can be changed without notice, it's essential for the public to be able to see what changed, and why.

There's a profound and simple tool that the Obama administration can use to improve government transparency. It's something that's enabled worldwide collaboration among software developers, and whose relevance for content development has been definitively demonstrated by wikipedia: Revision control. Not only does revision control allow a community to work independently on a common project, it makes it possible to review the changes.

There's a primitive form of revision control in word processing products like Microsoft Word, but we need more than that, especially for documents that bring together the work of multiple independent authors. For change.gov, the wikipedia model might work: logging of every change, with only authorized participants allowed to make changes, but everyone (the public) able to review and comment on associated discussion pages.

The real holy grail, of course, would be to provide revision control on all government regulations, and eventually, on legislation. This would no doubt be fought tooth and nail by lobbyists who don't want their fingerprints on the final result, but that's precisely why it would be such a breakthrough. And that's also why I suggest that the Obama team start with change.gov: demonstrate that the system works, that it has enormous benefits in transparency, and work from there.

Of course, there are major technical and workflow obstacles. Many of the documents in question are probably worked on independently as Microsoft Office files, with bulk merges that obscure the history. What we really need are distributed revision control tools. There's a lot of good work happening in this area in the software development community; it would be fascinating to see it extended to collaborative document development. (Of course, shared editing a la Google Docs is coming to Microsoft Office as well, so perhaps the point of leverage is for Google to improve the revision control capabilities in Google Docs, starting an arms race with Microsoft. Once the tools are in place, the social pressure to use them has a point of leverage.)

Like so many things that go under the rubric of "change," I'm sure that there would be many complications to this proposal, many problems I haven't thought of. Many current assumptions and processes would need to be challenged, and some of the challenges would take us down dead ends. But that's what change is all about. If it was just like the present, it wouldn't be change.

I'd love your thoughts both on the general proposal and specific ideas for implementation.

P.S. I wrote on this same subject about a year and a half ago, in a post entitled Why Congress Needs a Version Control System. It was Karl Fogel who first put this idea into my head, and that post explains his thinking. There are also some comments there from 2007 that may provide more grist for the discussion.