Four short links: 13 November 2009

Open Source Design, Interesting NoSQL Use, Copyright Documentary, Location Intelligence

1. Open Source Enters The World of Atoms -- an academic statistical analysis of open design. We indicated that, in open design communities, tangible objects can be developed in very similar fashion to software; one could even say that people treat a design as source code to a physical object and change the object via changing the source.
2. Why I Like Redis (Simon Willison) -- coherent explanation of why Simon likes and uses a particular nosql system. I can run a long running batch job in one Python interpreter (say loading a few million lines of CSV in to a Redis key/value lookup table) and run another interpreter to play with the data that’s already been collected, even as the first process is streaming data in. I can quit and restart my interpreters without losing any data. And because Redis semantics map closely to Python native data types, I don’t have to think for more than a few seconds about how I’m going to represent my data.
3. © kiwiright (Vimeo) -- short documentary about copyright, made to raise awareness of the issues in New Zealand. (just as applicable to the rest of the world)
4. Your Movements Speak For Themselves (Jeff Jonas) -- Mobile devices in America are generating something like 600 billion geo-spatially tagged transactions per day. Every call, text message, email and data transfer handled by your mobile device creates a transaction with your space-time coordinate (to roughly 60 meters accuracy if there are three cell towers in range), whether you have GPS or not. Got a Blackberry? Every few minutes, it sends a heartbeat, creating a transaction whether you are using the phone or not. If the device is GPS-enabled and you’re using a location-based service your location is accurate to somewhere between 10 and 30 meters. Using Wi-Fi? It is accurate below10 meters. A thought-provoking roundup of the information leakage with modern locative systems. (via TomC on Twitter)

Four short links: 4 November 2009

Electronics Hacking FAQs, Speech-To-Text Democracy, Open Source Column Database, Massive Online Analysis

1. ChipHacker -- collaborative FAQ site for electronics hacking. Based on the same StackExchange software as RedMonk's FOSS FAQ for open source software.
2. Democracy Live -- BBC launch searchable coverage of parliamentary discussion, using speech-to-text. One aspect we're particularly proud of is that we've managed to deliver good results for speech-to-text in Welsh, which, we're told, is unique. I think of this as the start of a They Work For You for video coverage. I'd love to be able to scale this to local government coverage, which is disappearing as local newspapers turn into delivery mechanisms for real estate advertisements.
3. InfiniDB: Open Source Column Database -- hooks into MySQL, uses MySQL for SQL parsing, security, etc. The commercial enterprise version has multi-server support (parallel scale-out). (via Brian Aker)
4. Massive Online Analysis -- MOA is a framework for data stream mining. Includes tools for evaluation and a collection of machine learning algorithms. Related to the WEKA project, also written in Java, while scaling to more demanding problems. . (via joshua on Delicious)

Four short links: 9 October 2009

Negative Karma, Wal-Mart TQI, Idiot Airlines, and Native iPhone Apps in Lua

1. Don't Display Negative Karma -- A fascinating insight for those building social software, whether for collective intelligence or otherwise: There can be no negative public karma-at least for establishing the trustworthiness of active users. A bad enough public score will simply lead to that user's abandoning the account and starting a new one, a process we call karma bankruptcy. This setup defeats the primary goal of karma-to publicly identify bad actors. Assuming that a karma starts at zero for a brand-new user that an application has no information about, it can never go below zero, since karma bankruptcy resets it. Just look at the record of eBay sellers with more than three red stars-you'll see that most haven't sold anything in months or years, either because the sellers quit or they're now doing business under different account names. (I love finding articles like this, thinking "they should write a book for us!" and then realizing "oh, they already are!") (via Hacker News)
2. Information Wants to be Free, Even At Wal-Mart (Pete Warden) -- an interesting piece on the value of opening up data, sharing information in negotiations so the best outcome can be reached. I'd argue that this trust argument is usually a cop-out, hiding worries about turf and control. In most cases it's clear that it's not in the other party's best interest to screw you over, and if it is, why are you dealing with them at all? The worst cases I saw were between departments within the same company, often we shared more information with competitors than the guys down the hall. The other reason I see people not sharing is shame: many companies (and individuals) work hard to present a facade of competence and quality that facts belie.
3. The Forest, The Trees, and the Bag Fees -- The bean counters can't track the revenue dilution of all these new fees. They don't want to. We miss the forest for the goddamed trees all the time. And the CEO acts as if fees are found cash. Meanwhile, no one asks why our overall revenue is plunging and we're losing money quarter after quarter. Everyone acts as if one thing has nothing to do with the other. A reminder to watch the important numbers, e.g. cash in bank, profit, customer satisfaction. (via Bryan O'Sullivan)
4. Native iPhone Apps Written in Lua -- open source port of Lua with Cocoa bindings for the iPhone. This is a tutorial showing you how to install and get past Hello, World. Apple have already approved one app written using it.

Four short links: 29 September 2009

Bletchley Park No Longer Blech, Contest Mania, Palm Process Fails For Free Software, Open Source Web Analytics

1. Bletchley Park May Have a Future -- the UK birthplace of modern computing, where Alan Turing worked during WW II breaking German codes, is dilapidated and in need of major repair. They appear to have a supporter in the UK National Lottery, who have given them a grant to begin work and prepare for further grants. It should be secured for the future as a place of significant historical merit in the development of computing. (See also The Geek Atlas)
2. Google Opens Voting on Ideas to Change the World -- there are a lot of contests at the moment: Project 10^100, Apps for Democracy, Apps for America, a plethora of X Prizes, the Netflix prize, and more. I wonder whether contests are like communities: you need a manager to cultivate and boost interest, or else your contest withers on the vine.
3. My ongoing Kafka-esque nightmare of dealing with Palm and their App Catalog submission process (jwz) -- This is my story about attempting to simply distribute this free software that I have written, and how Palm has so far completely prevented me from doing so. Epic Palm fail. (via Hacker News)
4. Piwik -- Piwik aims to be an open source alternative to Google Analytics. GPL-licensed.

Four short links: 23 September 2009

Video Art, Synthetic Biology Futures, Crowdsourced Personality, and an 1890s Startup

1. Projections (YouTube) -- the incredible video projection onto an old English manor house by Kiwi Foo Camp alums The Dark Room.
Where Will Synthetic Biology Lead Us? (New Yorker) -- a thoughtful article about the possibilities and cautions of synthetic biology. . “A house pet is a domesticated parasite,” he noted. “ It is evolved to have an interaction with human beings. Same thing with corn”—a crop that didn’t exist until we created it. “Same thing is going to start happening with energy,” he went on. “We are going to start domesticating bacteria to process stuff inside enclosed reactors to produce energy in a far more clean and efficient manner. This is just the beginning stage of being able to program life.”
2. Business Cards and Crowdsourced Personality Assessments -- we scanned images of a person’s business card and asked crowdsourced workers from the Amazon Mechanical Turk channel to write five kind words about the person based on what they saw. I like the idea of being able to crowdsource a quick impartial aesthetic judgement about a design.
3. When Sears Was a Startup (Pete Warden) -- one of the first catalogues from Sears (1897) inspires comparisons to Amazon and other web startups. On a mission with a new business model. They can't stop talking about how they're cutting out the middle men who've been gouging their customers, with pages devoted to messianic rants against the monopolies trying to put them out of business. They contrast their order fulfillment process (dozens of clerks dealing with tens of thousands of orders a day) with the inefficient country stores full of assistants being paid to idly wait for customers, explaining how they can offer such low prices despite the shipping.

Four short links: 21 September 2009

Bad Writing, Tech Immigration, Long Tail Fail?, and The Real McKoi

1. Dan Brown's 20 Worst Sentences -- awful awful writing, and glorious glorious mockery of it.

   Deception Point, chapter 8: Overhanging her precarious body was a jaundiced face whose skin resembled a sheet of parchment paper punctured by two emotionless eyes.

   It’s not clear what Brown thinks ‘precarious’ means here.

2. From Australia to the World: The Story of Google Maps and Google Wave (PDF, HTML Cached here) -- history of Google Maps and Wave, from the creator. This particularly struck me: I know few matters more frustrating than finding funding for a start-up. Immigration tops the list.
3. Rethinking The Long Tail: How to Define 'Hits' and 'Niches' -- the argument comes down to absolute vs relative measurements of popularity. Anderson says that relative hides too much, because percentages are meaningless in a world of infinite inventory. Researchers respond that hits and niches are defined in absolute numbers (top 10, bottom 100). The real takeaway is that infinite inventory requires excellent discovery tools drawing upon collective intelligence systems (hence the Netflix recommendation contest). (via timoreilly on Twitter)
4. The Mckoi Database -- MckoiDDB is a database system used by software developers to create applications that store data over a cluster of machines in a network. It is designed to be used in online environments where there are very large sets of both small and big data items that need to be stored, accessed and indexed efficiently in a network cluster. The focus of the MckoiDDB architecture is to support low latency query performance, provide strong data consistency through snapshot transaction isolation, and provide tools to manage logical data models that may change dramatically in physical network environments that may experience similar dramatic change. (via joshua on delicious)

Four short links: 8 September 2009

Mobile jQuery, API to Google Book Search, Open Learning, Popularity Algorithms

1. jQTouch -- jQuery library for mobile web app development. (via brian on Delicious)
2. GData API to Google Book Search -- search full text, get back metadata, modify "my library" collections, etc.
3. Open and Free Courses at the CMU Open Learning Initiative -- rather than just a lecture and handout dump, it has interactive exercises and questions to help you practice and figure out whether you've learned the subject. (via timoreilly on Twitter)
4. How to Build a Popularity Algorithm You Can Be Proud Of -- description and brief analysis for the popularity algorithms in Hacker News, Reddit, StumbleUpon, Delicious, and Linkibol. A basic collective intelligence technique that's not obvious. (via Simon Willison)

Four short links: 25 June 2009

Twitter Bucks, Nike Numbers, Map Apps, and Digi Shiz

1. How an Indie Musician Can Make $19,000 in 10 Hours Using Twitter -- as Zoe Keating pointed out: "cash made by @amandapalmer in one month on Twitter = $19,000; cash made by @amandapalmer from 30,000 record sales = $0".
2. The Nike Experiment: How the Shoe Giant Unleashed the Power of Personal Metrics (Wired) -- And not only can we collect that data, we can analyze it as well, looking for patterns, information that might help us change both the quality and the length of our lives. We can live longer and better by applying, on a personal scale, the same quantitative mindset that powers Google and medical research. Call it Living by Numbers—the ability to gather and analyze data about yourself, setting up a feedback loop that we can use to upgrade our lives, from better health to better habits to better performance. Collective intelligence + sensor networks can = happiness. (Mathematics gets by with just an "equals" operator. The rest of us need a "can equal" operator ...)
3. Old Map App -- iPhone app with old maps. Reminds me of David Rumsey's keynote at OSCON 2004.
4. Make It Digital -- Digital NZ site that helps organisations wanting to produce digital content, by offering them guidance on formats, metadata, and other issues they'll have to tackle. Includes a voting system to promote the (NZ) content you want to have digitised.

Four short links: 23 June 2009

1. Easter Eggs for Real Life (Neil Gaiman) -- ok, I know easter eggs are already part of real life, but this is still cool. Gaiman recommends a restaurant run by a friend, and the friend has set up a special phrase that to mention to the server, at which point something good and special will happen for them to eat or drink. Think of it as a restaurant Easter Egg. I love language, I love Gaiman's books, I love surprises, and I love that here Gaiman's using the digital sense of Easter Egg (surprise hidden in a program) rather than the analog sense (because there's no searching involved).
2. ASCAP Wants To Be Paid When Your Phone Rings (EFF) -- what the title suggests. You are lost in a twisty maze of rights, all policed by vampires. From ASCAP's point of view, this is a legitimate claim. From anyone else's point of view, it's ridiculous.
3. Tooling Up The Body (MInd Hacks) -- using tools has lots of interesting effects on our perception is the general gist of an intriguing study that provides further evidence for the theory that the brain treats tools as temporary body parts. We talk about using the Internet as our "offsite brain", so it tickles me to learn that the brain treats tools as offsite body parts.
4. Email Patterns Can Predict Impending Doom (New Scientist) -- when Enron was about to collapse, email patterns changed: the number of active email cliques, defined as groups in which every member has had direct email contact with every other member, jumped from 100 to almost 800 around a month before the December 2001 collapse. Messages were also increasingly exchanged within these groups and not shared with other employees. Menezes thinks he and Collingsworth may have identified a characteristic change that occurs as stress builds within a company: employees start talking directly to people they feel comfortable with, and stop sharing information more widely. (via BoingBoing)

Four short links: 29 Apr 2009

4chan, urban redesign, 3d printing, python

1. Moot Wins, Time Inc. Loses -- summary of how the 4chan group Anonymous rigged the voting in Time's 100 Most Influential poll to not just put their man at the top, but also spell an in-joke with the initial letters of the first 21 people. Time tried weakly to prevent the vote-rigging, and ReCAPTCHA gave the Internet scalliwags their biggest setback, but check out how they automated as much as possible so that human effort was targeted most effectively. It's the same mindset that build Google's project management, ops, and dev systems. Notice how they tried to game ReCAPTCHA, a collective intelligence app whose users train the system to read OCRed words, by essentially outvoting genuine users so that every word was read as "penis". Collective intelligence should never be the only security/discovery/etc. feature because such apps are often vulnerable to coordinated action.
2. The old mint in downtown SF painted by 7 perfectly mapped HD projectors -- looks absolutely spectacular. I love the combination of permanent and fleeting, architecture and infotexture. (via BoingBoing)
3. 3-D Printing Hits Rock-bottom Prices With Homemade Ceramics Mix (Science Daily) -- University of Washington researchers invent, and give away, a new 3D printer supply mix that costs under a dollar a pound (versus current commercial mixes of $30-50/pound).
4. Haystack and Whoosh Notes (Richard Crowley) -- notes on installing the search framework Haystack and the search back-end Whoosh, both pure Python. It's a quick get-up-and-go so you can add quite sophisticated search to your Django apps. (via Simon Willison)

Four short links: 27 Apr 2009

Data centers, open research, Jeopardy!, and tombstones

1. Google Server and Data Center Details -- Greg Linden reports on a Efficient Data Center Summit. Google uses single volt power and on-board uninterruptible power supply to raise efficiency at the motherboard from the norm of 65-85% to 99.99%. There is a picture of the board on slide 17. (and this is a 2005 board). Greg has left Microsoft as Live Labs is dissolved.
2. The Economics of Open Access Publishing -- set of papers on the free distribution of research. Pointed to by the RePEc blog. RePEc is Research Papers in Economics, a collaborative effort of hundreds of volunteers in 67 countries to enhance the dissemination of research in economics. The heart of the project is a decentralized database of working papers, journal articles and software components. All RePEc material is freely available. (via Paul Reynolds)
3. Computer Program to Take On Jeopardy! (NY Times) -- move over Turing Test, IBM's working on the Trebek Test: a computer program to compete against human “Jeopardy!” contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward. Really? The system must be able to deal with analogies, puns, double entendres and relationships like size and location, all at lightning speed. Oh, ok. So it's more complex than inverting the hash table of questions and answers. (via ericries on Twitter)
4. The Value of Minimal Data (Powerhouse Museum) -- if you have the ability for passionate users to contribute their knowledge, they can turn "minimal" data into a delicious four course data feast with a vintage port to sip during the dessert course. (via sebchan on Twitter)

Four short links: 22 Apr 2009

Government, Bayes, SMS, and distributed keystores:

1. Government Projects the Agile Way -- Can It Be Done? (NZ Government) -- notes and audio from a workshop at the New Zealand State Services Commission looking to merge agile and government. The pullquotes are mostly generic about agile, but the important thing is that there are agile projects within government and their numbers are growing. Having witnessed the incredibly slow, cautious, and non-agile development processes of government, I know how good this shift can be for budgets and delivery.
2. DivMod Reverend -- general purpose open source Bayesian classifier in Python (the Ruby port is Bishop). Bayes theorem lies behind the 2000-era spam filters, and there have been plenty of open source libraries to do Bayesian classification, but this one caught my eye because it's from the very good DivMod folks who are behind the very good Twisted framework. (via noahgift's delicious stream)
3. RapidSMS -- a free and open source messaging framework for building SMS applications. Integrates with Django. (via straup's delicious stream)
4. Some Notes on Distributed Key Stores (Leonard Lin) -- he had to install and test distributed keystores for a client's project, and posted his notes. Distributed keystores are one of the recent spates of database-like tools intended to solve some of the problems of big data applications. The distributed stores out there is currently pretty half-baked at best right now. [...] Don’t believe the hype. There’s a lot of talk, but I didn’t find any public project that came close to the (implied?) promise of tossing nodes in and having it figure things out. [...] Based on the maturity of projects out there, you could write your own in less than a day. It’ll perform as well and at least when it breaks, you’ll be more fond of it. Alternatively, you could go on the conference circuit and talk about how awesome your half-baked distributed keystore is. (via straup's delicious stream)

Four short links: 2 Apr 2009

Predictions, PDF, source code control, and recommendation engines:

1. Wrong Tomorrow -- track pundits predictions and see how accurate they really are. From the ever-awesome Maciej Ceglowski.
2. PDFMiner -- Unlike other PDF-related tools, it allows to obtain the exact location of texts in a page, as well as other layout information such as font size or font name, which could be useful for analyzing the document. It also infers text running within a page by using clustering technique. Entirely written in Python.
3. Migrating from svn to a Distributed VCS -- to decide which distributed VCS to use, Brett Cannon gathered Python use cases and then showed how they'd be done with different dvcses. The result is a very useful comparison document for svn, bzr, git, and hg.
4. Online Monoculture and the End of the Niche -- interesting post summarising and explaining research into recommendation engines, drawing the conclusion that although Internet World recommendation engines show everybody lots of new stuff, we're all seeing the same new stuff and the end result is less the "riches of niches" Long Tail fantasy and more a monoculture.

Four short links: 31 Mar 2009

Web traffic, web design, hacker spaces, and feature spaces:

1. iPhone and Android Make Up 50% of Google's SmartPhone Traffic Worldwide -- Matt Gross found this interesting tidbit in a TechCrunchIT story.
2. Refining Data Tables -- Luke Wroblewski gives some seriously good tips for designing usable tables in web pages. After forms, data tables are likely the next most ubiquitous interface element designers create when constructing Web applications. Users often need to add, edit, delete, search for, and browse through lists of people, places, or things within Web applications. As a result, the design of tables plays a crucial role in such an application’s overall usefulness and usability. But just like the design of forms, there’s more than one way to design tabular data. (via migurski's delicious stream)
3. Hacker Spaces (Wired) -- "It's almost a Fight Club for nerds," says Nick Bilton of his hacker space, NYC Resistor in Brooklyn, New York. is the must-have quote, but the guts of the article is "In our society there's a real dearth of community," Altman says. "The internet is a way for people to key in to that need, but it's so inadequate. [At hacker spaces], people get a little taste of that community and they just want more."
4. Related Document Discovery Without Algebra -- latent semantic analysis has some scary math, but If the feature space (e.g. the terms/concepts associated to your documents) is small enough, and you make sure synonymy is not a problem, you can do without algebra. One such case is that of your blog postings and their tags. Includes Ruby code. (via joshua's delicious stream)

Four short links: 26 Mar 2009

Books, Money, Collective Despair, and a Dashboard of Doom:

1. Will The Real iPod For Reading Please Stand Up -- Sebastian Mary argues eloquently that we're too focused on long-term writing because of the requirements and constraints imposed upon us by a mass-market paper book, whereas text online is basically an experiment in different lengths and sizes to find new balances for the new medium. a glance at the self-help or business shelves of your local bookshop will show you plenty more. And yet to make economic sense they have to be padded out for publication in 'proper' book size. But to conclude from this (as many unwittingly do) that long-form books are necessarily the best, rather than just the most familiar, way of communicating ideas is mistaken; and to assume that this practice will transplant to e-readers, imagined as a kind of iPod for these long-form essays, is just wrong.
2. What I've Learned in Angel Investing -- fascinating concrete lessons learned by an ex-Yahoo! angel. Be Wary of Entrepreneurs Who are Building for Businesses They Have No Experience In: I don't like it when people are theorizing about how a certain market is or isn't. They will most likely find problems that they have no experience tackling. It's better to find a company who has a veteran of the industry they are tackling so that they have at least have some first hand knowledge of what goes on in that industry.
3. Eco Datamining -- By trawling scientific list-serves, Chinese fish market websites, and local news sources, ecologists think they can use human beings as sensors by mining their communications. Reminiscent of InSTEDD's Golden Shadow project.
4. Check In On the State of the Economy -- a very appealing idea: a dashboard for the economy that is continuously refreshed as new data comes in. The difference between a one-off infographic and a live-updating dashboard is the difference between seeing the train and watching it race towards you. (via >Flowing Data)

Four short links: 25 Mar 2009

IT, AI, IQ, and UK today:

1. Definition IT -- a blog about the ways in which IT is becoming a problem and not a solution. In the absence of any independent global standards or best practice models that guide the delivery of technology into businesses we have relinquished control to the suppliers of our technology. The suppliers are in a mammoth arms race to sell more products and this has become the de-facto controller for the delivery of technology into businesses. No one statement in the blog is outrageous, but they add up and indicate an industry that isn't delivering the value it claims to.
2. What is a Good Recommendation Algorithm? -- Greg Linden starts an interesting conversation at the CACM blogs.
3. Why Money Messes With Your Mind (New Scientist) -- interesting psychological research. the volunteers who had been primed with the money-related words worked on the task for longer before asking for help. In a related experiment, people in the money-word group were also significantly less likely to help a fellow student who asked for assistance than were people in the group primed with non-money words.
4. Obama's Diplomatic Gift to UK Leader Fubared by DRM (BoingBoing) -- we can laugh, but Obama's team is stacked with ex-RIAA lawyers.

Four short links: 16 Mar 2009

Non-interop earphones with DRM, HVAC swarms, paperprints, and product constipation at GOOG:

1. Apple iPod Shuffle (3rd gen) -- "Surprise: the only third-party headphones that will work are ones that haven’t even entered manufacturing yet, because they’ll need to contain yet another new Apple authentication chip, which will add to their price." It's interesting to see Apple prioritising the different interactions with the device: the ease from using standard connectors is less important than how it looks (no buttons!), and both are less important than securing those hardware margins by tithing every corner of the aftermarket thirdparty addon space. I hope users revolt, but I suspect it'll be viewed like the needless diversity of power cords--a non-fatal inconvenience and irritation. I can only imagine the state of mind that thinks it is acceptable to irk a million people so as to be able to make a few bucks on each set of third-party headphones.
2. Managing Energy with Swarm Logic -- intelligent HVAC gear that wirelessly communicates, figures out the individual power cycles of each appliance, then coordinate (with no central control) to figure out how to optimally run the appliances for maximum efficiency.
3. Fingerprinting Blank Paper Using Commodity Scanners -- Ed Felten's latest work of genius, cranking the contrast on scans of paper at different orientation to produce a texture from which can be calculated a fingerprint that survives printing, crumpling, and moisture. (via bos)
4. Tim Armstrong to Head AOL (Battelle) -- interesting quote from former Googler, "It's very hard to take risks at Google."