Four short links: 17 June 2009

Word Mining, Open Ideas, Power Meter BotNet, and Realtime Web Traffic Tracking

by Nat Torkington | @gnat | comments: 0

1. NY Times Mines Its Data To Identify Words That Readers Find Abstruse -- the feature that lets you highlight a word on a NY Times web page and get more information about it is something that irritates me. I'm fascinated by the analysis of their data: boggling that sumptuary is less perplexing than solipsistic. Louche (#3 on the list) has been my favourite word for two years, by the way, since I heard Dylan Moran toss it out in that uniquely facile way the Irish have with words. I think Irish citizens get this incredible competence with the English language for free, along with staggering house prices and beer you can walk on.
2. Open Ideas -- Alex Payne's blog of Concepts in the public domain, awaiting collaboration and appropriation.
3. Buggy 'smart meters' open door to power-grid botnet (The Register) -- Paul Graham said that we've found what we get when we cross a television with a computer: a computer. Similarly, intelligent power meters are computers, computers that apparently haven't been well-secured. To prove his point, Davis and his IOActive colleagues designed a worm that self-propagates across a large number of one manufacturer's smart meter. Once infected, the device is under the control of the malware developers in much the way infected PCs are under the spell of bot herders. Attackers can then send instructions that cause its software to turn power on or off and reveal power usage or sensitive system configuration settings.
4. Chartbeat -- the sexiest web analytics ever. It gives realtime count of users, whether they're reading or writing (based on whether focus is in a form element), where they're from, mentions on Twitter, and more and more and more. This is a different form of analytics than Google Analytics, which tells you trends and historical access. Love this for the pure sex appeal of a heads-up dashboard that can tell you exactly how many people are on your site and exactly what they're doing. (via Artur)

Twitter Aphorisms, Epigrams and Repartee

by Tim O'Reilly | @timoreilly | comments: 40

I recently encountered the following zinger on twitter:

@jayrosen_nyu: Scoble is like a guest at a hotel for one, where a huge staff is trying to anticipate his every need. And he's angry.

Shades of noted wits from the past! As when Dorothy Parker, asked to use the word horticulture in a sentence, said "You can lead a whore to culture, but you can't make her think" or when Nancy Astor said to Winston Churchill, "Winston, if I were your wife, I'd put poison in your coffee," and he replied, "Nancy, if I were your husband, I'd drink it," or less meanly, Oscar Wilde on his deathbed: "Either that wallpaper goes or I do." (My all time favorite, from the collection Viva la repartee, tells how the Earl of Sandwich braced reformist politician John Wilkes with the insult, "Upon my soul, Wilkes, I don't know whether you'll die upon the gallows, or of the pox," to which Wilkes cuttingly replied, "That will depend, my lord, on whether I embrace your principles, or your mistress." Ouch!)

It occurred to me that twitter, with its 140 character limit, its dialogue between people who may be rivals as well as friends, is a breeding ground for the rebirth of repartee and of the aphorism and epigram. So I started keeping track of some of these sparks of wit. Some of these are actual rejoinders; others are simply clever insights. Here are a few I've captured recently:

@sacca: You can't really appreciate the vapidity of most people's taste in music until you live directly above a traffic signal.

@tempo: Evolution is a sorting process that is the very antithesis of random. (in response to http://is.gd/1Do1 )

@davorg: Conference Driven Development - submitting talks to conferences so that it will galvanise you into doing the work you're talking about.

@sourcegirl: procrastination is our brain's way of saying that something is not as important as we may think it is...

@gstein: OH: isn't a smoking area in a restaurant like a peeing area in a pool?

@amandachapel:@lisahoffmann "I've had conversations with people I never would have met otherwise." Like hanging out at the bus terminal.

I only follow a few hundred people out of millions of twitter users, so I'm thinking that there must be tens of thousands of great lines waiting out there to be captured into a book of twitter one-liners. If you know of any, and want to share them, either tweet them to the attention of @timoreilly or leave them in the comments here. A book of twitter wit and wisdom would make a fun conference giveaway, don't you think?

Keep in mind that, as Aristotle said, "Wit is educated insolence." I'm not looking for abuse, per se, but cleverness and concise expression of insight. When I asked for suggestions on twitter, several people pointed me to feeds of people who routinely insulted others, often crudely. A great insult may be appropriate, but it's far from the soul of wit. Consider the examples above, and give me more like that!

P.S. And be sure to give me the link to the individual status message if you have it. I can hopefully find it via twitter search without, but that's more work than you might think, especially if the quote isn't exact. As I discovered finding the links above, one wrong word that you're sure you remembered correctly can get you a seemingly mysterious "no results."

Social Network Signature for Entity Resolution

by Tim O'Reilly | @timoreilly | comments: 17

In the lazyweb department, I had an idea the other day that I thought I'd put out more broadly (lest someone else have the same thought, plus the thought to patent it.) And that is the idea that one side-effect of the "social graph" is to create a unique identity signature. Who my friends are can be used for entity resolution. (Background: one problem in identity is figuring out whether two people who have the same name are in fact the same person. This is complicated by variations on the name. So there's a whole set of questions: is T. O'Reilly the same as Tim O'Reilly the same as Timothy F. O'Reilly? Are you referring to my brother Sean O'Reilly or my father Sean O'Reilly? And when you find a reference to Tim O'Reilly, is it the Tim O'Reilly who blogs here or the Sydney musician who also has a wikipedia entry, or one of the hundreds or thousands of others who have the same name.)

Typically, you resolve identity conflicts by adding additional information: a phone number, an address, a social security number.

Now, clearly, there are far more cases where you might have easier access to this kind of real-world information than you would have access to someone's friends list from Facebook. But it's also true that a site like Facebook could offer an identity service by which they present a unique hash of someone's friends list at a particular point in time as a unique credential that doesn't actually require disclosure of any confidential information.

Of course, this is just a special case of a much broader situation, namely that our "identity" is in fact a function of everything we show to the world. Mechanisms might generate credentials by hashing our purchase history at Amazon, our search history at Google, or our surfing patterns in Firefox as easily as they could hash our social network. But the point is that it is possible to generate credentials that are as unique as fingerprints.

It would be kind of cool not to have to enter passwords, but for a site to "recognize" me because I was able to present a hash of my past interactions with the site, automatically recorded by both the site and my browser.

You could think of this as a kind of public key cryptography. Your private key would be the timestamp at which the hash was created, or of the start time and end time that were used to create it. Your public key would be the hash itself.

Food for thought.

German and Japanese Wikipedia scanner

by Artur Bergman | comments: 7

Time for more excitement on the Wikipedia front. Last week Virgil released the English Wikiscanner, as I wrote about previously. This week, it is time for German and Japanese edits to be exposed, with the newly-released German and Japanese Wikiscanner. Sadly, I don't read either German nor Japanese, but I am sure some of you readers do. If you find any nice ones, please post them in the comments. Thanks to a tipster, I know of this Scientology ping pong

Web performance book could use examples of slowwwwww Ajax

by Tim O'Reilly | @timoreilly | comments: 13

Andy Oram writes on the O'Reilly editors' list: "Steve Souders at Yahoo! is writing a book for us on Web performance. The focus of the book is on speeding up downloads and browser behavior. He doesn't concentrate on Ajax, but some of what he writes is definitely relevant, and he is asking us for get examples of sites where Ajax updates seem to be slow." That seemed like a good job for Radar readers! Please nominate your candidates, your exercises in frustration, in the comments.

Presidential Primary 2.0?

by Tim O'Reilly | @timoreilly | comments: 7

If Web 2.0 isn't really about any particular suite of technologies, but rather about understanding how to harness the internet more effectively, whatever your field of endeavor, have we really searched out how to apply it to politics. Peter Meyers had a great take on this question on an internal O'Reilly mailing list. He wrote:

So far the Web's been used as a political tool mainly in a partisan way -- for candidates' sites, single-issue organizations, activists, and so on. But what if somebody created a Web-based tool that served a much broader group of people?

Here's my LazyWeb idea: On November 4, 2007 (1 year before the actual presidential election), some neutral organization should stage an Internet primary for both Republican and Democratic candidates. It obviously couldn't be totally secure and/or accurate, but you could implement a few small requirements (e.g. one vote per email address) to prevent massive abuse. It'd be different from the zillion different polls currently being done -- assuming sufficient publicity could be generated -- since you could stage it as a tide-turning one-time-only event.

With all the jockeying among states vying to upstage Iowa and New Hampshire's disproportionate early primary influence, wouldn't it be kind of neat if someone harnessed the Web's "collective intelligence" to gauge the early preferences of voters across the country?

Sounds like a job for Carl Malamud!

Social Blog Aggregators Revisited

by Nikolaj Nyholm | comments: 7

Recently, there's been a number of interesting posts on building personal blog aggregators. This really does seem like the next thing that is going to make some blog company succesful since just there's just too many feeds to read. Global aggregation and popularity engines like Digg and Techmeme only adds material to read.
I'm sure hundreds of people have had ideas along these lines long since, but this particular strand of meme starts with a request from Ross Mayfield for myMeme, not a personalized version of TechMeme, but a socialized one - "what are people I read, reading". Sam Ruby promptly put together a prototype implementation based on his feed subscriptions which seems to work -- but Gabe Rivera of Techmeme is not convinced by the example. His contention is that Ruby's example only picks out news that TechMeme was already picking up.

Rivera does have a point -- indeed Ruby's output looks like just the tip of the Digg iceberg, but depending on your goals this might not be a problem. The problem with unread news is not that it is unwritten, but that it is drowning in all the other news. I'd settle for a meMeme that worked as a reliable garbage detector.

Workware Instead of Shareware?

by Tim O'Reilly | @timoreilly | comments: 11

There's an interesting suggestion over on bitporters media, about how Amazon's Mechanical Turk could be used to create a new form of shareware:

Think AMT + Adsense or “AMTSense” as I will call it through out the rest of this post.... AMT at the core is a WEB SERVICE, so why not add features to the API that allow users to submit HIT’s directly from desktop programs. You’ve heard of SHAREWARE, what about the concept of WORKWARE (people perform a few micro tasks in your name to pay for the right to use it). I see this concept changing the business model of anything from file download sites to the independent game industry / open source community.

Since I picked up this link from Amazon's Web Services blog, they are paying attention, and might well do this if there is sufficient demand. So if you like the idea, let them know.

Flickr and Interestingness

by Tim O'Reilly | @timoreilly | comments: 17

Recently, I was looking for a photo to illustrate a talk I was giving at a company meeting. I was telling the old story about three men working. A passerby asks them what they're doing. The first man grunts "working" and goes back to his stonework. The second man says, "I'm building a wall." The third man stops, gazes off into the distance for a moment, and says, "I'm building the most beautiful cathedral in Ireland."

I wanted to remind people what we're really doing each day. Not just working, not just building a wall, but focused on what Jim Collins calls a Big Hairy Audacious Goal. (O'Reilly's is changing the world by spreading the knowledge of innovators.)

So I needed some good pictures as visual bullets. I first went to Google image search, but while the photos were good, they were all somewhat "expected." After paging through the first four sets of results, still hadn't found anything suitable. So I went over to flickr, set my search to "most interesting" rather than most recent or most relevant, and soon had a fabulous selection of unusual photos of cathedrals.

Google made a breakthrough in web search with its original idea of links as citations (i.e. PageRank), and they are still the undisputed leader in general web search, but they haven't done as well in searching rich media. I think they have some things to learn from Flickr. More specifically, web search innovators all need to think through what makes results "interesting" for a given domain. I like what flickr has done in calling out "interestingness" as a quality worth searching for, and leaving it as a playground for exploration.

Lazyweb: My GMail Wishlist

by Tim O'Reilly | @timoreilly | comments: 25

I dropped something heavy on my laptop the other day just as the system was rebooting, and managed to trash the disk. While it's in the shop, I've had to rely on GMail as my main mail application. (I normally read mail with mail.app on the Mac, but have everything autoforwarded to GMail for backup and access from other computers.) Using GMail full time reminds me of all the things that I think it ought to do as a mail client from Google, which in theory is one of the internet-savviest companies around (see Levels of the Game), as well as things that I expect as normal good behavior, plus some things that I miss from old Unix mail clients like mh and mush/zmail.

Right now, GMail is operating at the lowest level of Web 2.0. It uses Ajax to give an online experience, but it actually doesn't do anything that can't be done in a local mail client. (And actually, since all mail is networked, even if the client itself is local, these same features could be implemented by anyone, from Microsoft to Apple. But it's GMail that's on my mind right now....) If anyone at Google is reading, please put these features on your punch list:

(continue reading)

IMDb API??

by Tim O'Reilly | @timoreilly | comments: 27

Interesting back-channel email conversation with Phil Torrone: "i talked to hb from imdb. here's what i told him i'd to do with an imdb api / access to some of the data...

1. Fastr-like game for movies. If you've ever played Fastr, you're addicted - it pulls in Flickr photos and you guess what they were tagged with - you compete with others, and it's way too much fun. For IMDB, I'd do the same, you get photos of actors, directors, stills from movies, etc. You can guess a variety of things, it's timed, you can have teams, etc - you could do ads on and between rounds, a quick "buy the DVDs/etc you saw here". This could also make a good mobile game for phones.

(continue reading)

Lazyweb Request: theyworkforyou.com for the US

by Tim O'Reilly | @timoreilly | comments: 13

I'm a big fan of transparency, and one of the areas we need it most is in the workings of government. Is there anything like the UK's theyworkforyou.com for the US? There really ought to be. This site lets you search the speeches made by Members of Parliament, so you can see what they have been up to. Here in the US, you can search the congressional record, but it's the easy linkage of people to positions that makes theyworkforyou.com so interesting.

(While I'm at it, I'm wondering if fundrace.org, Eyebeam's site that provides transparency into campaign finance, is going to be brought live again for the 2006 elections, or whether it was a one-time stunt.)

Directions Microformat -> Google for directions?

by Tim O'Reilly | @timoreilly | comments: 12

At Mix06, I had pressed Bill Gates a bit about the importance of microformats. In his reply, Gates mentioned that they could be useful for directions--and Tantek Celik, who was in the audience, picked up on this idea and made a call-to action requesting a microformat for directions.

Don Marti picked up on the significance of this idea: "When people put geographical directions up using microformats, someone will crawl them and string the route decisions together to get a directions search engine with common sense (because it borrowed the common sense of millions of users) that doesn't tell people to make an illegal left into oncoming traffic, the way a certain map site used to tell me to leave my old house every day. (70mph combined speed motor vehicle slalom! Yaaaaahooooo!)"

This is exactly the train of thought that I'm sure Bill G was having, because in addition to our backstage discussion on the importance of microformats (which led to the exchange Tantek reported on stage), we also talked about how Bill thought that Navteq's lock on directions data would come under attack as Web 2.0-style collaborative data gathering gained steam.

Tantek's lazyweb call for a directions microformat as a way to start this bottom-up process seems really imporant to me. It would be really cool to see some progress on this idea by our Where 2.0 conference in June.

Web 2.0-or-Not meter

by Tim O'Reilly | @timoreilly | comments: 4

Don Marti, of the original sucks-rules-o-meter, wrote a very interesting blog entry entitled On Web 2.0, application uses you! He's riffing on the idea that Web 2.0 is about what I've called architectures of participation, systems whose value is built "from many small information contributions that users don't mind making. Every user whitewashes a little bit of the fence."

In the course of the posting, he makes a couple of very interesting points. One was a simple heuristic for whether or not something is Web 2.0-ish or not, and that is whether the API is truly open, or requires some kind of partner signup process. He says:

You could probably do a pretty reliable Web-2.0-or-not-o-meter based on dates in the RSS feed for API announcements vs. dates in press releases matching /partner/i.

Here's the full context:

(continue reading)