Four short links: 5 November 2009

Heat Maps in R, EC2 Blackhat Tricks, Snickersome Unicode, and Decoding Statistics

by Nat Torkington | @gnat | comments: 0

1. Heat Maps in R -- We used financial data here because it's easier to access than the airline data, but it's actually a pretty interesting way of looking at a financial time series. Weekend and holiday effects are a bit more obvious, and it's a bit like being able to see the daily, weekly, monthly and yearly closes all at once (by scanning your eye over the calendar in different directions). Includes source code. (via migurski on Delicious)
2. BlackHat and EC2 -- Theft of resources is the red-headed step-child of attack classes and doesn't get much attention, but on cloud platforms where resources are shared amongst many users these attacks can have a very real impact. With this in mind, we wanted to show how EC2 was vulnerable to a number of resource theft attacks and the videos below demonstrate three separate attacks against EC2 that permit an attacker to boot up massive numbers of machines, steal computing time/bandwidth from other users and steal paid-for AMIs. (via straup on Delicious)
3. Funny Characters in Unicode -- I never get tired of the wacky stuff in Unicode. I love the thought of a Unicode committee somewhere arguing passionately about the number of buttons on the snowman .... (via Hacker News)
4. Statistics to English Translation -- The terms sensitivity and specificity generally refer to diagnostic or screening procedures, such as an HIV or allergy tests. The sensitivity of a test is its true positive rate; the specificity is its true negative rate, although it can be more intuitive to think of specificity as the complement of the false positive rate. This matters. Bandying around numbers with misleading labels, or misinterpreting numbers that have a precise and defined meaning, does not further understanding. (Said 78.4% of statisticians, with a 20% confidence factor probability of false positives)

 

Safari Books Online 6.0: A Cloud Library as an alternate model for ebooks

by Tim O'Reilly | @timoreilly | comments: 134

There has been a lot of attention paid to ebooks lately, and for good reason. Electronic books are portable, searchable, and more affordable than print books. The web has accustomed readers to having the latest information at their fingertips; we all ask why books should be any less available "on demand."

Amazon’s Kindle has received the most mainstream attention (with new entries like Barnes & Noble's Nook making dedicated ebook readers into the latest competitive horse-race), but ebooks are taking off even faster on the iPhone and other smart phones. Ebooks are one of the most popular classes of iPhone application. Recent releases of O'Reilly ebooks as iPhone applications have even outsold the same books in print. Direct sales of the ebook bundles we offer from oreilly.com (PDF, epub, or mobi files) also exceed our direct sales of print books from the site.

Yet our most popular ebook offering by far is often not even thought of as an ebook. Safari Books Online is an online book and video subscription service, launched in partnership with the Pearson Technology Group in 2001. It contains more than 10,000 technical and business books and videos from more than 40 publishers. It has more than 15 million users (including the number of concurrent seats available through libraries and universities); it is now the second largest reseller of O’Reilly books, exceeded only by Amazon.com, and its revenue dwarfs our sales of downloadable ebooks. It's also the most affordable of our ebook offerings for those who are regular consumers of technical content. The average Safari Books Online subscriber uses at least seven books a month, and many use dozens (or even more), yet the monthly price (depending on the subscription plan) ranges from little more than the price of a single downloadable ebook to no greater than that of two or three.

Here’s the rub: most people thinking about ebooks are focused on creating an electronic recreation of print books, complete with downloadable files and devices that look and feel like books. This is a bit like pointing a camera at a stage play and concluding that was the essence of filmmaking!

At O’Reilly, we’ve tried to focus not on the form of the book but on the job that it does for our customers. It teaches, it informs, it entertains. How might electronic publishing help us to advance those aims? How might we create a more effective tool that would help our customers get their job done?

It was by asking ourselves those questions that we realized the advantages of an online library available by subscription. One of the best things about online technical books is the ability to search the full text of a book. How much better would it be to be able to search across thousands of books? Safari Books Online was our answer.

And it just got better. Safari Books Online 6.0, released yesterday, brings a new level of ease of use. It’s a complete, bottom-to-top revamping of the original service. The old UI was, to say the least, getting long in the tooth.

The new UI is slicker and faster, with the kind of drag-and-drop goodness that people expect from a modern web application. In addition, we’ve added some long-requested features, including:

Improved Interactivity -- With 6.0 you can make inline notes, in the actual text you are reading. You can dog-ear or bookmark specific pages. You can highlight text and associate it with notes. When you are done you can print those pages with both your highlights and notes on them. You can scroll non-stop through the pages of a book without any page refresh, or scan a block of pages in thumbnail view to spot the page you are looking for.

Personalized Folders - Rather than having thousands of books and videos organized by us in a single technology topic taxonomy, you can now put together your own organization, grouping books in the categories most useful to you. You can restrict searches to only the books you’ve chosen, and can search within the results of a saved search.

Collaboration - Even better, if you’re a corporate subscriber, you can share your categorization with other members of your company or workgroup. Not only can team members share folders, they can share book reviews, notes and highlights.

Smart Folders - New books, videos and articles are being added to Safari Books Online all the time. Searches saved as "smart folders" make it easy to keep up with the latest content in your area of interest. We have also improved our search user interface to allow you to search inside the book or in other books without leaving the page you are reading. Switch pages only when you find what you want.

As you can see, many of these features take advantage of the online medium in ways that aren’t possible with standalone ebooks. To be sure, there are times you want your own offline copy, and in Safari Books Online, you can indeed download books or chapters for offline use. But especially given the rise of the smartphone as an access device, the times when we are truly "offline" are becoming few and far between. The vision with which we started Safari, that of always-on access to a library of technical content, not just to individual ebooks, is now within reach. Safari Books Online can be used on a desktop or laptop computer or in the browser on a mobile phone. Everything is always in sync because your library is in the cloud.

An ebook cloud works the same way the web itself works. It provides ubiquitous access and shared experience.

Lessons Learned from the development of Safari Books Online

As I outlined above, Safari adopted a "cloud library" model rather than downloadable ebooks as its fundamental design metaphor. I thought it might be worthwhile to understand how we arrived at that decision, as well as some of the other lessons we’ve learned over what is now 22 years of ebook publishing experience. (O’Reilly published its first ebook, Unix in a Nutshell for Hypercard, back in 1987!) With that, a few reflections on lessons learned:

(continue reading)

iPhone Killers, Blackberries and Chicken Parts

by Mark Sigal | @netgarden | comments: 5

There is an unfortunate tendency to confuse delivering a bunch of 'chicken parts' with producing an actual living, breathing chicken.

MG Siegler, over at TechCrunch, has written an excellent article that shines a light on the cycle from hype to disappointment that goes with being dubbed an 'iPhone Killer.'

BlackBerry Storm, Palm Pre, the G2, and now Droid have all been touted as contenders to the mobile computing crown, yet the iPhone continues to kick butt.

No less, Apple has levered its market leadership position with iPhone (and the iPod Touch) to create a halo effect on the rest of its business, generating bottom line results that are industry-defining (see analysis of Apple's Q4 results HERE).

Meanwhile, conventional wisdom, shaped by the history of Apple vs Microsoft during the PC Wars, tells us that Android is 'destined' to be bigger than the iPhone worldwide.

And to be clear, would-be iPhone slayers are indeed establishing strategic positions that have the potential to become compelling and differentiated within the mobile market. Examples include:

• Android: We are more open than Apple;


• RIM: We are more enterprise-ready;


• Palm Pre: We are more web-native;


• Android, RIM, Nokia, et al: We are a heterogeneous device platform.

But, alas, there is a fly in the ointment. Many of the above solutions are at a functional stage where they still fail to deliver a 'more than the sum of the parts' experience - at a time when Apple is clicking on all cylinders from a product innovation and new product pipeline perspective.

(continue reading)

My 140conf Talk: Twitter as Publishing

by Tim O'Reilly | @timoreilly | comments: 6

I spoke at Jeff Pulver's 140conf a few weeks ago. My subject was the continuity of what I do, from publishing through conferences through my presence on twitter. I tried to draw the connections, and to explain how "social media" means drawing from, curating, and amplifying the voices of a community. I suggest that the role of an editor and publisher is analogous to the role of a point guard in basketball, handing out "assists" and improving the performance of his or her teammates. After all, I point out, I couldn't possibly tweet enough to cover all the topics I am interested in. But by using my retweets to build the visibility of others, I can create and foster a community that cares about the ideas, trends, and people that I care about.

My talk starts about 1:40 into the video, after a few comments from Jeff Pulver, the conference organizer. I've provided a lightly edited and linkified transcript below, for those of you who don't have time to watch the entire 15 minute video. If you do have the time, you can watch the video from the entire two-day conference at http://www.140conf.com/watchit.

What I learned from Twitter

Hi. I want to talk to you a little bit about Twitter and media. I'm a publisher. I'm a publisher in print. And it turns out I'm also a publisher on Twitter. I want to explain the roots of media and how that connects with what we're doing in this newest form of media.

When you think about the original use case of Twitter, which @Leisa described so wonderfully as “ambient intimacy,” it's really news from your close friends. But it's news nonetheless. And sometimes the news from individuals becomes news that matters to a whole lot more people. When someone in Tehran today is reporting their personal news, it's news that matters to all of us. And so you can see the continuum between the personal and the international in those moments.

But that continuum exists all the time, and it's existed always in media.

(continue reading)

Personal Democracy Forum: Politics in the Web 2.0 Era

by Tim O'Reilly | @timoreilly | comments: 2

In the past year or so, I've been urging people to work on stuff that matters. The world is faced with serious problems, and we in the technology community have a unique contribution to make, as the tools we've created help us to collaborate and organize at an unprecedented scale outside of industrial-era top-down organizations.

One area where technology and real world concerns meet is in the challenge of remaking democracy in a Web 2.0 world. With the support of the President of the United States himself, the US government is committed to exploring how to use technology to make government more transparent, accountable, and collaborative. How cool is that?

And how important is it that we, the technology community, rise to the challenge? If a couple of years go by, and nothing changes, the opportunity will have been lost; new media and new technology will be relegated to the dustbin of fads that have come and gone in Washington. It's up to us to make it not so, to prove to ourselves that we can indeed use technology to make a difference - in governing, but also in the critical tasks that face us as citizens: creating a more robust economy, improving our educational system, reducing the cost and increasing the effectiveness of health care, achieving energy independence and halting climate change.

I've been organizing two events in Washington to create new bridges between the technology community and the political community, the Government 2.0 Expo Showcase and the Government 2.0 Summit, to be held in Washington D.C. in early September. You'll be hearing more from me about these events in the coming months - they are consuming a lot of my time and energy as I try to understand how to bring the best of what we've learned about the age of networks to the problems of government.

But in the meantime, I want to let you know about an event that is happening in New York at the end of June.

Personal Democracy Forum is a two-day tech + politics brainfest that brings together a thousand political activists, organizers, hackers and hacks, along with many leading elected and government officials, NGO leaders, academic observers and journalists. It's now in its sixth year, taking place June 29-30 in New York City at Jazz at Lincoln Center.

This year's conference is focused on the theme of "We.gov" and all the ways that campaigns, elections, media, advocacy, and governance are becoming more open, participatory and collaborative. I've known the organizers, Micah Sifry and Andrew Rasiej, for several years now, and in addition to their work as technology advisers for the Sunlight Foundation, they are part of a vanguard of individuals who are leading that change.

Come hear keynotes from speakers including: White House CIO Vivek Kundra; Deputy CTO for Open Government Beth Noveck; State Department Senior Adviser for Innovation Alec Ross; New York Times columnist Frank Rich; Craigslist founder Craig Newmark; Fivethirtyeight.com blogger Nate Silver; Twitter co-founder Jack Dorsey; Obama '08 new media director Joe Rospars; Edwards '08 campaign strategist Joe Trippi, writers Clay Shirky, danah boyd and Doug Rushkoff, and anthropologists of the future Mike Wesch and Mark Pesce, among many others.

In addition to covering lots of the brass tacks of doing politics in a networked age (online targeting, using mobile platforms, spreading viral video, raising money, harnessing volunteers effectively), the agenda also tackles a lot of cutting-edge topics, including:

• Twitter as a platform for organizing and fundraising (with speakers like Amanda Rose of Twestival and Abby Kirigin of the startup TipJoy)
• Imagining White House 2.0 (with Jim Gilliam of WhiteHouse2.org, Ellen Miller of Sunlight, Fabrice Florin of Newstrust and Mark Elliott of Collabforge)
• The Rise of Health Care 2.0: Participatory Medicine (with Esther Dyson and James Heywood of PatientsLikeMe)
• Building the Social Economy (with Doug Rushkoff and Tara Hunt)
• Redesigning .Gov for Transparency and Participation (with Clay Johnson and Ali Felski of Sunlight among others)

You can see the full conference details here.

O'Reilly Radar readers can save $100 off the conference registration by using this coupon code: "Oreimedi100"

Four short links: 1 June 2009

Spymaster, Arsenic, Maps, and Happiness

by Nat Torkington | @gnat | comments: 1

1. Spymaster -- a faux-spy game on Twitter: Each player becomes a master of a spy ring based upon their Twitter followers list. The more people that follow you and are playing characters in Spymaster, the more powerful your network will be. As a spymaster, you can perform tasks or attack other spymasters on Twitter. With each successful attempt, you will gain virtual currency and points that allow you to grow even stronger. I'm nervous that it's a project of a classified ads company, but intelligent friends appear to be enjoying it, but that may just be be the jaded eye of a world-weary veteran of pyramid schemes and spamalots.
2. Getting Arsenic Out Of Water -- MIT Technology Review piece about the IBM discovery that a chemical used to pattern chips also acts as a membrane to remove arsenic. More stuff that matters. (via roterhund on Twitter)
3. Mapumental -- MySociety folks making maps useful. It's the continuation of time travel maps, where bus, train, tram, tube, and ferry timetables are mashed with real estate prices to show you where you can live for what you can afford and how long a commute you want. A new twist is crowdsourced "how scenic is this area?" data, so you can choose other dimensions for where you might want to live. New dimensions on transportation data and travel planning.
4. What Makes Us Happy? (The Atlantic) -- the real world is a lot more complex than trivial "get happy fast!" self-help books would have you believe. This longitudinal study shows how complex happiness and misery are. Vaillant’s other main interest is the power of relationships. “It is social aptitude,” he writes, “not intellectual brilliance or parental social class, that leads to successful aging.” Warm connections are necessary—and if not found in a mother or father, they can come from siblings, uncles, friends, mentors. The men’s relationships at age 47, he found, predicted late-life adjustment better than any other variable, except defenses. Good sibling relationships seem especially powerful: 93 percent of the men who were thriving at age 65 had been close to a brother or sister when younger. In an interview in the March 2008 newsletter to the Grant Study subjects, Vaillant was asked, “What have you learned from the Grant Study men?” Vaillant’s response: “That the only thing that really matters in life are your relationships to other people.” (via timoreilly on Twitter)

Geeks Invade Government With Audacious Goals

by Mark Drapeau | @cheeky_geeky | comments: 16

Guest blogger Mark Drapeau is the Co-Chair of the Gov 2.0 Expo Showcase in Sept 2009 and the Gov 2.0 Expo in May 2010, both in Washington, DC. He holds the title of Associate Research Fellow at the Center for Technology and National Security Policy at the National Defense University, a professional military educational school run by the Joint Chiefs of Staff. Mark is also co-founder of Government 2.0 Club, an international platform for sharing knowledge about the intersection between technology and governance.

When one thinks about important problems facing the United States, and indeed people all over the world, it is difficult to not come up with the laundry list that every talking head seemingly has on the tip of their tongue: jobs, education, health care, national security, poverty. There are so many problems to solve, with so many constraints on spending money, and such a short supply of manhours to get the job done. Many government employees spend a lot of time working on the issue or crisis of the day (or the hour) rather than thinking about long range planning and strategy.

This might be Alexander Hamilton's fault. One of the first things I was indoctrinated with after moving to Washington, D.C. was that the U.S. system of federal government was not designed to make good decisions; rather, it was designed to not make horrible ones. This is counterintuitive, perhaps, but mainly true. And this flies in the face of ideas about using technology to make government more efficient, mainly because the purpose and organization of government is quite different from that of business.

Nevertheless, more and more people from the private sector are interested in playing a role in government, thanks in no small part to the excitement surrounding the Obama election and inauguration, in which social media technologies and information sharing were showcased at their best - massive fundraising from many small donors, empowering people to self-organize locally, and direct public relations that circumvented a mainstream media lens. Now, people enamoured with emergent social technologies want to know how they themselves can revolutionize not only politics, but also governance.

For those who don't follow fashion trends in Washington, D.C., allow me to present the new and increasingly popular species of talking head - The Geek. (The Geek is distinguished from The Wonk, studious, preppy, bespectacled types that run Washington policy, know exactly what intersection Brooks Brothers is on, and enjoy cocktail parties for "networking," and The Nerd, the type of scientist or other fastidious pointy-head rarely seen outside a laboratory or professorial tower, with nary an interest outside their own peculiar and narrow slice of life.)

The prototypical Geek is a different breed of talking head, one that usually lacks media training, one that often hails from Silicon Valley, San Francisco, Seattle, New York, Austin, Boston, St. Paul, or Boulder, one who likely knows more about the inside of a computer than the average person does about the inside of their fridge, a well-read introvert shy in real life but outgoing on Twitter and in the blogosphere, who is erudite enough to have always felt there was a better way to run the government but feeling entirely disconnected from the apparatus.

No longer. When I speak about Government 2.0 to audiences around D.C. I am fond of telling them about the very smart and motivated outsiders (i.e., The Geeks) who think that they can run the government better than the government can. I enjoying dropping the line, "The government can no longer afford to work at the pace of government," because people never really know what to say in response as they mull it over. That statement is somewhat tongue-in-cheek and not entirely fair to hundreds of thousands of hard-working government employees; but of course, my role as a speaker is usually to provoke thought and get a point across, not to be fair. And gradually, through my efforts and those of many other Government 2.0 enthusiasts, people inside the Beltway are understanding that new ideas and new technologies can bridge gaps between government and the citizens (and that outsiders are starting to utilize such technologies whether the government gives permission or not).

(continue reading)

The Myth of Macroinnovation

by Nat Torkington | @gnat | comments: 13

An idea is making the rounds and appearing in articles like this New York Times piece, and it goes roughly thus: the age of the small inventor is over because to work on stuff that matters requires the largescale coordination of people and materiel that only governments and large corporations can provide. This notion that we're entering a Golden Age of Macroinnovation is bunkum, I'm happy to report.

Scale matters, scale has always mattered, but scaling is not innovating. It's true that there are many opportunities for businesses and governments to do big things. That's always true—all my friends who worked at Yahoo! and Microsoft said one of the attractions was the ability to write code that would be used by hundreds of millions of people. However, the article basically says, "large institutions are tackling large problems." That's wonderful news, much better than large institutions ignoring large problems, but has nothing to do with innovation.

Perhaps I'm wrong, perhaps scaling is a form of innovation. Innovation is characterised by disruption and the unknown. Think of those governments and large corporations and ask yourself: are these the birthplaces of radical thinking, new ways of getting things done, and risk-taking leaps into the unknown? Of course not. Governments are the most risk-averse institutions in the world, more so than medicine where lives hang in the balance—doctors at least listen to evidence, whereas the definition of bureaucracy is "we follow the rules regardless of reality". Governments exist to preserve the status quo that elected them, not disrupt it.

Don't go hoping for a change in government's risk aversion any time soon. Every penny is spent knowing that to fail means to be vilified for "squandering public money". Doing something new risks failure. Like a puppy that has been harshly house-broken, Government associates failure with pain and so responds with fear, hostility, and concealment. With that mindset you can never learn from failure, and so unless you luckily get it right first time you'll find your Government road to delivering something new to be harsh, difficult, and largely untrodden.

Do you think things are different now? Consider Obama's billions on health record rollout. KP have a model EHR system, "KP HealthConnect". It cost $4B and took five years to buy and roll it out. This is KP's second go at it: in 2002 they wrote off $770M they'd spent with IBM trying to build their own EHR system. Do you think that the Obama Administration will get a second chance if his first attempt at EHRs loses $770M?

Big businesses aren't much different: a large company is a small Government that has more flexibility on HR. The profit focus of a business is a help and a hindrance as Innovator's Dilemma so clearly showed. The NY Times piece quotes Clayton Christensen saying, "The good news is that, once they recognize the benefits of disruptive thinking, the big companies have all the resources necessary to induce change.”

My experience with large companies and governments shows me that it is not a simple or trivial matter to recognize the benefits or marshal the resources. A common failure mode is where the leadership say they want disruption and innovation, the grass roots want it, but the middle management tiers aren't incentivised to deliver it because their bonuses are tied to metrics on existing product lines. Disruption eats into existing businesses. "Maximizing Shareholder Value" is a wonderful focusing device but, without an explicit timeframe for that value, innovation risks shareholder lawsuits for sabotaging profitability.

In his delicious.com comment on the NY Times piece, Michal Migurski observed, "New New Deal is at heart a massive, all-fronts realignment—where's the role for the small and the nimble in this universe?". It is premature to declare Mission Accomplished for reinvention of Government (see the Government 1.5 meme). At its heart, this is an attempt to get Government to use the Web 2.0 tools we built in the last decade ... tools that were largely the product of one or two people. I don't see bureaucrats using decade-old tools as an "innovation" that the small and nimble have to worry about.

I love that governments, NGOs, businesses, and citizens are going to be tackling large and meaningful problems with the aid of the tools and techniques developed by researchers, entrepreneurs, and hackers around the world. But to mistake using those techniques for inventing them is to ignore that great lesson of Margaret Mead: "Never doubt that a small group of thoughtful committed people can change the world; indeed, it is the only thing that ever has."

Welcoming Eric Ries to the Radar Team

by Tim O'Reilly | @timoreilly | comments: 4

The Radar blog is a community of thinkers organized around the O’Reilly mission to change the world by spreading the knowledge of innovators. Some of the folks with posting privileges on Radar are O'Reilly employees: Brady Forrest organizes the ETech, Where 2.0 and Web 2.0 Expo events, Mike Loukides, Andy Oram, Brett McLaughlin, and Mike Hendrickson are editors of many of the books you know and love, Ben Lorica does data analysis in our research group, Andrew Savikas heads up our digital publishing efforts, Dale Dougherty is the publisher of Make:, Sara Winge runs the Radar group and organizes our annual Foo Camp.

Others work part-time with us, such as our open source maven Alison Randal, who co-chairs the Open Source Convention, and “Master of Disaster” Jesse Robbins, who co-chairs the Velocity conference on large scale web operations. Some are alumni such as Nat Torkington and Marc Hedlund, who have gone on to other jobs but remain very much part of the O'Reilly family.

But others are interesting people we have met along the journey like Artur Bergman, Jim Stogdill, and Nick Bilton. These are people who've stimulated our thinking and helped us reflect on areas we want to learn about. In each case the goal is the same - talk about "Stuff That Matters" and generate meaningful conversation. With that in mind, I wanted to welcome Eric Ries to the Radar community.

I met Eric a few months ago, and immediately realized that he was someone I could learn a lot from, and whose ideas I wanted to spread as widely as possible. Eric has been championing the concept of The Lean Startup; a methodology that helps startups learn and adapt faster than the competition. Startups get lean through a mixture of agile development, leveraged product development and implementing direct, tight customer feedback loops. The result is a new type of company - one that uses operational excellence to drive down costs and accelerate learning.

Eric’s methodology has been honed by running successful startups (and learning from running unsuccessful ones) along with experience gathered through consulting, mentoring, and advising entrepreneurs. The Lean Startup is deeply prescriptive and practical; it is a vision for a new way to start, build and grow your company—starting on day one.

One of the things that excites me about the Lean Startup is that it doesn’t just apply to the traditional “two guys in a garage.” The questions that I have seen technology startups face time and again are increasingly relevant to institutions of all kinds: Who exactly is my customer? What exactly do they want? How do I deliver my product quickly and effectively at lower cost? Lessons learned in the crucible of entrepreneurship are applicable to enterprise and to government as both struggle to do more with less, to grow to reach new markets, and to innovate.

You will find Eric here occasionally on Radar as well as on his blog. Additionally, Eric has partnered with O’Reilly to produce a series of upcoming workshops intended to help people master the concepts of The Lean Startup.

Here is a video that Radar’s Joshua-Michéle Ross shot with Eric recently.

Social Science Moves from Academia to the Corporation

by Joshua-Michéle Ross | @jmichele | comments: 4

This is the latest of a series of posts addressing questions regarding social technologies. Previous posts: The Evangelist Fallacy, Captivity of the Commons and The Digital Panopticon. These topics will be opened to live discussion in an upcoming webcast on May 27 with a special guest to be announced.

In order to control a thing you must first classify a thing -- and we are seeing a massive classification of social behavior. While that classification falls under the guise of making life easier (targeted ads, locating a nearby pizza joint using your mobile), history tells us that we should be leery of motives and masters of our social data (see Captivity of the Commons).

Social sciences (behavioral psychology, sociology, organizational development), whose historical lack of data and scientific method left them open to ridicule from the “hard” sciences, finally have enough volume of data and analytics and processing power (see Big Data) to make “social” much more scientific. But this time social science is going to be coming to you not courtesy of Princeton, but courtesy of Google. Not through small studies on willing subjects, but through massive multivariate testing and optimization upon (largely) unknowing test subjects. The corporation, in other words, will hold the keys to social science at a level of precision only dreamed of by the academic and state institutions of yore.

This recent New York Times article highlights just how much social science, psychology, and personal data converge when a credit card company wants its debts repaid (via Andy Oram’s Radar post).

Should we be concerned about this shift from academia to the corporation?

I hold the current structure of government and corporations in equal regard in terms of how well they adhere to Google’s maxim, “Don’t be Evil.” So in some regard, I shouldn’t really be troubled that social science has moved from academia (which has often been a handmaiden of government) to the corporation (which really just wants to understand what moves you to click that “buy” button, or bump up your average order size by $10, etc.). Except…

Except if you believe that consumer culture is wreaking havoc upon the systems that support life and that the application of social science on behalf of the corporation is intended to simply turbo charge the status quo...

We find ourselves in 2009 facing deep, structural challenges -- peak oil, environmental degradation, climate change, and financial meltdown.

That's why the notion of social science in service of accelerating the existing system troubles me. Tim has spoken about the need to “Work on Stuff that Matters.” How might we apply social science toward "stuff that matters" instead of toward "buying more stuff that doesn't matter?"

Four short links: 1 May 2009

Smart Grids, Open Source, Stuff That Matters, and Global Culture

by Nat Torkington | @gnat | comments: 2

1. A Little Give and Take On Electricity (NY Times) -- Dennis L. Arfmann, a lawyer at the Boulder office of Hogan & Hartson who specializes in environmental law, said he had no idea how much electricity he and his wife, Dr. Julie Brown, had used before he filled his roof with solar panels producing 4.5 kilowatts of power. During the day he sells power to Xcel and at night he buys it back; his goal is to cut his use so his net sales rise. All hardware networked, everywhere!
2. Open Source World Map (Red Hat) -- very nice map showing the intensity of open source use in countries around the world. (via Flowing Data)
3. Imagine Cup -- Microsoft's contest to get students working on stuff that matters. The winners of the New Zealand leg, Team Think, tackled literacy: they devised a program for tablets that provides both handwriting recognition and audio output, eliminating the need for basic literacy to understand lessons or instructions. They hope to take this prototype to developing countries that have underutilised computers due to literacy issues. (via Idealog newsletter and Scoop)
4. UGT -- It is always morning when person comes into a channel, and it is always late night when person leaves. [...] The idea behind establishing this convention was to eliminate noise generated almost every time someone comes in and greets using some form of day-time based greeting, and then channel members on the other side of the globe start pointing out that it's different time of the day for them. Now, instead of spending time figuring out what time of day is it for every member of the channel, we spend time explaining newcomers benefits of UGT. (via migurski on delicious).

Four short links: 30 Apr 2009

Youth, Government, Tween Arduino Hackers, and Table Slurpage

by Nat Torkington | @gnat | comments: 0

1. Ypulse Conference -- conference on marketing to youth with technology, from the very savvy Anastasia Goodstein who runs the interesting Ypulse blog on youth culture that I've raved about before. Register with the code RADAR for a 10% discount (thanks, Anastasia!).
2. Government in the Global Village -- departing post by the NZ CIO (and Kiwi Foo Camper) Laurence Millar. The principles here are applicable to almost every nation. We need to recognise the network effects of opening up government data in a form that means others can access it. Economic value is created by businesses building innovative new services using government data. Public value is created by enabling a richer and deeper understanding and dialogue among interested individuals about what the data tells us about our lives.[...] The legal, policy, and moral position is clear - New Zealanders own the data, having paid for its collection through taxes. These “problems” will all be solved by the community, and our role as government is to give priority to this. These efforts are stuff that matters. See also Google adds search to public data.
3. Children's Arduino Workshop (Makezine) -- video of three eleven-year old girls working on an Arduino project, and should be inspiration to anyone who has ever wanted to work on hardware projects with kids. Whoever did it succeeded in making it fun! (via followr on Twitter)
4. With YQL Execute, The Internet Becomes Your Database -- YQL is a query language for Yahoo! data sources, and now they've added a server-side Javascript way to import your own web page's tables into YQL. YQL and Pipes are turning into very interesting pieces of infrastructure (e.g., Museum Pipes blog). (via Simon Willison and straup on delicious)

Importance of Innovation in Finance & BarCampBank

by Jesse Robbins | @jesserobbins | comments: 2

“Progress is not the mere correction of evils. Progress is the constant replacing of the best there is with something still better.” -Edward Filene

Two years ago, when we were organizing the first BarCampBank in the US, many people found it hard to believe that banks & credit unions could a place for meaningful grassroots innovation. Even crazier was the idea of organizing an unconference to begin bringing open source, transparency, identity, and community into the very closed world of banking & finance.

Since then the BarCampBank idea has turned into a movement. There have been over 14 events all over the world, and many of the ideas generated are beginning to turn into action.

To me, the global financial system is a platform that exists to “create more value than it captures”. Tim explained this in his Work on Stuff that Matters post, saying:

“A bank that loans money to a small business sees that business grow, perhaps borrow more money, hire employees who make deposits and take out loans, and so on. The power of this cycle to lift people out of poverty has been demonstrated by microfinance institutions like the Grameen Bank. Grameen is clearly focused on creating more value than they capture; not so the like of Fannie Mae and Freddy Mac, or WaMu, or many of the other failed financial institutions involved in the current financial meltdown.”

There has never been a more important time to bring meaningful innovation into the financial system, and there has never been more opportunity for our community to make it happen.

The next event is occurring this weekend (April 25-26, 2009) on Treasure Island in San Francisco.

After that, the following events are planned:

• BarCampBankVegas is set for May 2, 2009.
• BarCampBankCharleston2 is set for June 13, 2009
• BarCampBankGermany is set for October 23-25, 2009

Four short links: 14 Apr 2009

by Nat Torkington | @gnat | comments: 0

Open data, lean startups, RSS-as-newspaper, and a design call to arms:

1. OpenSecrets Goes Open Data -- The following data sets, along with a user guide, resource tables and other documentation, are now available in CSV format (comma-separated values, for easy importing) through OpenSecrets.org's Action Center [...] : CAMPAIGN FINANCE: 195 million records dating to the 1989-1990 election cycle, tracking campaign fundraising and spending by candidates for federal office, as well as political parties and political action committees; LOBBYING: 3.5 million records on federal lobbyists, their clients, their fees and the issues they reported working on, dating to 1998; PERSONAL FINANCES: Reports from members of Congress and the executive branch that detail their personal assets, liabilities and transactions in 2004 through 2007; 527 ORGANIZATIONS: Electronically filed financial records beginning in the 2004 election cycle for the shadowy issue-advocacy groups known as 527s.
2. The Lean Startup Presentation at Web 2.0 -- with audio. I've raved about Eric Ries blog before.
3. Times -- an RSS feedreader with a newspaper's layout. News reading can be improved and newspapers are in the middle of dying, so it makes sense that someone would try a face transplant. I'm not convinced that the newspaper's front page is the model for perfect news delivery, although I do love the ultimate in dense news layouts: Arts & Letters Daily. (via joshua's delicious feed)
4. Designing Through a Depression (NY Times blog) -- exhortation to work on stuff that matters. This rethinking needs to come not just from designers but from the manufacturers, companies and other clients who decide what products and projects will be produced. There’s no excuse not to examine and re-examine what’s made, how it’s manufactured, what materials are used (and which are recyclable), what benefit it’s giving the consumer (or lack thereof) and what contribution, if any, it’s making to anything other than landfill. I believe recessions are when good things flourish.

It's Always Ada Lovelace Day at O'Reilly

by Tim O'Reilly | @timoreilly | comments: 10

I had a hard time choosing just one of the many marvelous women in tech that I might write about for Ada Lovelace Day, because, frankly, I'm surrounded by those women! Where so many think of the tech world as male-dominated, women have always played a major role at O'Reilly. A large part of our management team has always consisted of women, and women are the creators of some of our best known products and brands. I want to acknowledge their contributions, highlighting the fact that they have been so central to the success of my company. I also want to emphasize that there are many ways to contribute to a tech community, and that being a coder is not the only way to have an impact on the world of computing.

My first hat tip has to go to my wife, Christina O'Reilly. She's a playwright and choreographer, not a techie. But if you've been influenced by me, you've also been influenced by her. The company, its values, and much of its unusual nature have been profoundly shaped by our relationship. (You can see her influence in some of the early company documents you'll find here,.) In more ways than I can count, I've built the company to be one that she would be proud of. We met when I was nineteen, and she's been part of everything I've ever done, in the same way that Elizabeth Barrett Browning said of her husband, Robert Browning:

What I do and what I dream include thee,
As the wine must taste of its own grapes

From the earliest years of the company, most of my key managers have been women. From 1985 till about 2000, there was a troika--Linda Walsh, Linda Lamb, and Cathy Brennan--who helped me shape the corporate culture, and for many years were touchstones for the values I still espouse.

Linda Walsh was my first employee. She helped me build my original documentation consulting business, and helped me come into my own as a leader when I broke up with my original business partner. She was also the key business leader for our digital books initiative in the late 1990s and one of the founders of Safari Books Online.

Linda Lamb was a key member of the team (along with me and Dale Dougherty) that developed our original publishing program back in 1985. Linda also served as our director of marketing for many years, with a delicious, quirky sense of humor. (I still remember one of her earliest trade show pieces, a wonderful riff on the National Enquirer, in which she reported on abductions by strange aliens with big eyes, programmers forced to participate in camel races, and exorcisms performed after errors in programming with curses.) She was also the original author of one of our first books, Learning the Vi Editor, and later creator (with Nancy Keene) of our series of Patient-Centered Guides.

I hired Cathy Brennan (now Strider) as a receptionist in 1985 or 1986, but her common sense soon made her one of my most trusted advisers and her ability to get things done made her one of my senior executives. When I decided to move to California from Boston in 1987, Cathy was the one who persuaded me to move our then-fledgling publishing business along with me, and agreed to move out herself to set up customer service and operations for this new line of business. She built and ran our operations as we grew from a tiny startup to a publishing powerhouse. She also managed the design and construction of our office complex in Sebastopol.

Laura Baldwin joined the company as CFO in 2001, just as we were crashing into the wall of the dot com bust. I can with confidence say that we wouldn't have survived without her. I had always held up Harold, the last of the Saxon kings of England, as one of my management heroes. He went down to defeat by William the Conqueror rather than abandon his people to fight another day. Laura convinced me that I needed to do layoffs--and when I did, I found that the people we laid off moved on to what were often better jobs for them, and the company itself became leaner, more creative, and more effective. Laura brought financial discipline to the company. She helped bring us back from the brink, and built a bigger, more profitable, and more successful company. Living to fight another day helped us to birth the Web 2.0 movement, Foo Camp, Make: magazine, the Missing Manuals, and many of the other post-2001 products we are known for today.

Laura is now our COO as well as CFO, and is the day-to-day manager of the business. Those of you who wonder how I find so much time to spend on twitter while running a business, look no further!

I have learned more about the nuts and bolts of business from Laura than from any other person. Harold Geneen once said "The skill of management is to achieve your objectives through the efforts of others." Laura knows how to do that in spades - she's a great manager. But she's also the most amazingly productive person I've ever met. Usually, you have to choose between an effective manager and an effective individual contributor. Laura somehow manages to be both.

The list goes on. CJ Rayhill was our CIO for seven years, working with Laura to build the information systems that turned O'Reilly into a "real company." (She was also one of the first women to graduate from the Naval Academy - here's an interview about her history in the tech industry.) She's now the COO for Safari Books Online, where she's managing the extremely cool features that will be appearing in Safari 6.0 later this year.

Gina Blaber was the original managing editor for GNN, the world's first commercial website. She then ran our software group (remember Website Pro, the first PC-based web server?) and now runs the O'Reilly Conferences group. If you've ever been to an O'Reilly event, it's easy to think that the speakers and program chairs do all the work, forgetting that it's Gina and her team (mostly women) who make it all look so easy.

Laurie Petrycki ran several of our publishing divisions (notably Head First and Missing Manuals) before taking on the challenge last year of launching our new education division.

And that's leaving out the many women who've worked as editors, copyeditors, designers, and production staff at O'Reilly over the years. If you've ever read and enjoyed an O'Reilly book, take a moment to appreciate how many hands, how many eyes, read it before you did, to shape it into its final form.

Sara Winge is the creator of Foo Camp, one of O'Reilly's quirkiest and most influential initiatives. While Wikipedia claims me as a co-creator of the event, I have to say that it has always really been Sara's brainchild. I had wanted to do some kind of events at O'Reilly after the dot com bust left us with a lot of unused space, and I might have even proposed residential events, but Sara is the one who picked up this idea from the heap where we tend to leave good ideas that don't have anyone to make them real.

Sara conceived and developed the format (inspired in part by Open Space, she says, but to my mind, all the best parts were original.) I've just been the front-man and impresario. So if you've been to Foo Camp, or to Bar Camp, or any of the many other "Camp" spinoffs, you owe a big round of thanks to Sara. Foo Camp also demonstrates a uniquely feminine sensibility. Sara didn't charge to the front; she created a context where other people can shine, quietly facilitating. As Lao Tzu said, "When the best leader leads, the people say, 'We did it ourselves.'"

Edie Freedman is the creator of the distinctive O'Reilly animal brand. Many people know a bit about the story of how strange animals came to be the symbols of so many technologies, but what they probably don't know that behind this brand, so central to the company's heritage, was an act of generosity by a complete stranger.

Our first books, published late in 1985, all had the same simple cover, featuring the image of a nutshell. The idea was that these small books had everything you needed, in a nutshell. In 1987, with seven books in print, we realized that people at trade shows weren't recognizing that we had more than one book, so we hired a designer to produce some new covers. She developed a treatment that was colorful, geometric and high-tech. We had an all-hands meeting on a Friday afternoon to review the new cover treatment. I just couldn't go for it. It was too expected. I said we'd have to try again.

Linda Lamb shared our plight with her housemate, Edie, who at the time was a designer at Digital Equipment Corporation. Edie thought that Unix program names sounded like weird animals. She also realized that 19th century woodcuts provided a unique, low-cost design option. Fair enough. But she went further than that. She produced and laid out seven mechanicals of possible covers over the weekend. Linda brought them in on Monday, free of charge. Here's one of the original designs, for Sed & Awk, a book that didn't even exist yet.)

How cool is that? One of the great brands in tech was a gift from a stranger! (Edie did later come to work for us, and served as our Creative Director for many years. She is still with the company.) That is one of the many experiences that made me receptive to the idea of open source as a gift culture when Eric Raymond wrote about that in 1997.

(While I'm on the subject of the kindness of strangers, I have to call out the contributions of Danese Cooper and Linda Stone, two people who've never worked for O'Reilly but who might as well have done, for all the tireless work they do on our behalf. I met Danese through our mutual work on open source when she was the "open source diva" at Sun, and later at Intel. She's the person I always call when anyone asks me for advice on open source licensing, community, or corporate adoption. Linda Stone, former maven at Apple and Microsoft, is a mentor and inspiration on the value of making connections between people who ought to know each other. Linda is also the one who had the brilliant idea of Science Foo Camp when Timo Hannay of the Nature Publishing Group and I were scratching our heads trying to find a project to do together. She's also the one who suggested we ask Google to host the party. Never mind random acts of kindness; there's real power in random acts of connection. When I said a few years back that our purpose at Foo Camp is to create new synapses in the global brain, I was channeling what I learned from Linda.)

I could go on and on. There's Allison Randal, who Nat Torkington already wrote about in his own Ada Lovelace Day post earlier today. There's Kathy Sierra, the creator of the amazing Head First series of books, and also the subject of many another Ada Lovelace Day post. There's Carla Bayha, who for many years was the computer book buyer at Borders, and whose penetrating judgment helped books on many an obscure topic find space on shelves across America. Carla is truly an unsung hero of the industry.

And of course, I celebrate the many other women authors, conference speakers, and coders who've been a part of O'Reilly's story over the years. Truly, we could never have done it without you. As far as I'm concerned, it's always Ada Lovelace Day at O'Reilly.

How to build companies that matter

by Eric Ries | @ericries | comments: 23

Eric Ries, a serial entrepreneur, most recently was co-founder and Chief Technology Officer of IMVU, his third startup. He's the author of the blog Lessons Learned, co-author of several books including The Black Art of Java Game Programming (Waite Group Press, 1996), and a Venture Advisor at Kleiner Perkins Caufield & Byers. He. In 2007, BusinessWeek named Ries one of the Best Young Entrepreneurs of Tech. He'll be presenting on "The Lean Startup: a Disciplined Approach to Imagining, Designing, and Building New Products" at Web 2.0 Expo.

We're living in a time of renewed possibility for startups. Major trends - from the pain of the economic crisis to the disruption of web 2.0 - are breaking the old models and paving the way for a new breed of company. I call it the Lean Startup.

The Lean Startup is a disciplined approach to building companies that matter. It's designed to dramatically reduce the risk associated with bringing a new product to market by building the company from the ground up for rapid iteration and learning. It requires dramatically less capital than older models, and can find profitability sooner. Most importantly, it breaks down the artificial dichotomy between pursuing the company’s vision and creating profitable value. Instead, it harnesses the power of the market in support of the company’s long-term mission.

Tim O'Reilly has recently been advocating that as an industry we focus on building stuff that matters. In response, I want to try and present a way of building startups that can realize that dream. In particular, he as articulated three principles:

(1) Work on something that matters to you more than money, (2) Create more value than you capture, and (3) Take the long view.

Given the hype and easy credit that has been the hallmark of technology startups the past few years, it's been too easy for them to be unclear about whether they are really creating more value or just spending money to create the appearance of success. The lean startup approach tackles this problem from the very beginning of a startup's life. My experience is that startups need to be built from the ground up for learning about customers and what they will pay for. That means an obsessive focus on finding out "is our company vision really the path to a brave new world, or just a delusion?"

(continue reading)

Four short links: 18 Mar 2009

by Nat Torkington | @gnat | comments: 1

Open data sites, Kenyan mobile phone startup, open cheminformatics, and Uncle Sam Can't Surf:

1. Timetric -- time series data, charted. Takes earlier Pro-Am data analysis tools Many Eyes and Swivel a step further with formulaes, moving averages, triggers, and updated data. Oh, and it plays well with others: OpenID, OAuth, OpenSearch, and a useful API. I met one of the founders at Science Foo Camp a year or two ago, where he was introducing computational chemists to the semantic web.
2. txtEagle -- ETech 2009 video, Nathan Eagle on mobile startups in Kenya. The first five minutes alone redlined my awe-ometer: he's been teaching CS students and then professors, in sub-Saharan Africa, how to develop for mobile phones. Now there are hundreds, perhaps thousands, of mobile phone developers where before there were few, and plenty of startups. Work on Stuff That Matters.
3. Open Cheminformatics -- in this post about a new open cheminformatics journal, Peter Murray Rust summarizes the current state of cheminformatics It suffers from closed data, closed source and closed standards, and thereby generally poor experimental design, poor metrics and almost always irreproducible results and conclusions which are based on subjective opinions and then goes on to list the rays of hope, the open data, open source, open standards (ODOSOS) projects in cheminformatics.
4. Government 2.0 Meets Catch-22 (NY Times) -- “We have a Facebook page,” said one official of the Department of Homeland Security. “But we don’t allow people to look at Facebook in the office. So we have to go home to use it. I find this bizarre.” This is the perfect antidote to those who would say that the web isn't the transformative break with the status quo: bureaucrats represent the ultimate in status quo, and if they're grappling with the web then that means the web is as different as we've always said it is.

Sunlight Hackathon at Web 2.0 Expo

by Jennifer Pahlka | @pahlkadot | comments: 9

Guest blogger Jennifer Pahlka is the general manager and co-chair of the Web 2.0 events at TechWeb.Earlier, Jennifer ran the games group at CMP, where she oversaw the growth of the Game Developers Conference and launched the Independent Games Festival and Gamasutra.com. Pahlka Dot is her personal blog.

I am fond of reminding people who recall the last economic crash that this time, it's not the web industry's fault. In fact, this time around, tech is the way out of the mess we're in. There are many ways to see this, including the efficiency gains of adopting 2.0 tools and practices still latent within our businesses, and many folks have rightly pointed out that innovative startups will be needed if we are to reinvent our economy. In the broadest sense, however, we're talking about a way of thinking that centers around participation, transparency, and openness. In retrospect, these were the assets that have been in the shortest supply in recent years.

But if you're not keen on launching a banking 2.0 start-up at the moment, what's one easy way a developer can start building the new economy and society we sorely need? Participate in the Open Government movement. A growing number of talented coders and designers are taking advantage of the data the government is beginning to make available and mashing it up in ways that make it accessible, useful, and meaningful to citizens, lawmakers, and any constituent you can imagine. Tim wrote about the significance of this trend here. Vivek Kundra, the new US Federal CIO (currently on leave) who previously served as the CTO of the District of Columbia, famously sponsored an innovation contest called Apps for Democracy, which invited hackers to mash up DC's data. Sunlight Labs (the development arm of the Sunlight Foundation, which "uses the power of the Internet to shine a light on the interplay of money, lobbying, influence and government in Washington") has extended this concept with Apps for America, which is taking submissions now through March 31st. Expect more opportunities to build utility out of government data as this movement builds.

But Sunlight is also coming to Web 2.0 Expo. We believe that the more smart people we can get involved in this movement, the better, so we've invited the great folks at Sunlight to come out and run a Hackathon at the Expo this year. There are a dozen-odd projects currently on listed on their website; we invite you to go vote for the one you'd most like to contribute to, or suggest a new one. The one that gets the most interest will be chosen as the project for Web 2.0 Expo, and attendees can come by the hackroom and help build the app and meet some great people during the event.

You know you want to. Work on stuff that matters!

Four short links: 11 Mar 2009

by Nat Torkington | @gnat | comments: 0

Four ETech-related links, from your humble author who is following the action from afar:

1. Criminals Are "Targeting Basic Blocks of the Internet" (Guardian) -- writeup of Alex Stamos's talk. "Basic infrastructure failure is what we're going to see over the next few years," he said. "The most interesting research is either taking things that we thought were unexploitable and exploiting them, and also the breaking of the basic building blocks of the internet from the 1970s and 1980s." For another "we're all so boned" moment on Internet security, read Peter Gutmann's overview of the commercial malware industry.
2. Phil Gyford's ETech 09 Posts -- Phil takes notes and attends a lot of the sessions I'd have wanted to be in, like Tim's "Work on Stuff that Matters".
3. Mary Lou Jepsen's Talk (Guardian) -- interesting bit for me was a low powered television set that can display high definition video but can run without being plugged in. "We've had a lot of pull," she said. "People want TV even if they don't have power… an HDTV that's under 10W and can be human-powered. We've figured out a way to do that." Not that I'm in love with television, but the technology that gets mass-produced for cud-chewing couch-butts gets cheaper for the likes of you and me. See her new company, Pixel Qi.
4. ETech on Hashtags -- see the latest tweets tagged with "#etech". E.g., @fortunebird's Rebecca Allegar: Don't predict the future, design it., and @Technomadia's We just controlled a chocolate lab live via iPhone. Now.. we eat more chocolate! I like this presentation lots!

ETech Preview: Science Commons Wants Data to Be Free

by James Turner | comments: 4

John Wilbanks has a passion for lowering the barrier between scientists who want to share information. A graduate of Tulane University, Mr. Wilbanks started his career working as a legislative aide, before moving on to pursue work in bioinformatics, which included the founding of Incellico, a company which built semantic graph networks for use in pharmaceutical research and development. Mr. Wilbanks now serves as the Vice President of Science at Creative Commons, and runs the Science Commons project. He will be speaking at The O'Reilly Emerging Technology Conference in March, on the challenges and accomplishments of Science Commons, and he's joining us today to talk a bit about it. Good day.

John Wilbanks: Hi, James.

JT: So science is supposed to be a discipline where knowledge is shared openly, so that ideas can be tested and confirmed or rejected. What gets in the way of that process?

JW: Well, most of the systems that scientists have evolved to do that: sharing, confirmation and rejecting, evolved before we had the network. And they're very stable systems, unlike a lot of the systems that we have online now, like Facebook. For science to get on the Internet, it has to really disrupt a lot of existing systems. Facebook didn't have to disrupt an existing physical Facebook model. And the scientific and scholarly communication model is locked up by a lot of interlocking controls. One of them is the law. The copyright systems that we have tend to lock up the facts inside scientific papers and databases, which prevents a lot of the movement of scientific information that we take for granted with cultural information.

Frequently, contracts get layered on top of those copyright licenses, which prevent things like indexing and hyperlinking of scholarly articles. There's also a lot of incentive problems. Scientists and scholars tend to have an incentive to write very formally. And the Internet, blogging, email, these are all very informal modalities of communication.

(continue reading)