Entries tagged with “hacking” from Tools of Change for Publishing

CNET Pops the Kindle's Hood and Takes Pictures

CNET/Tech Republic cracks open the Kindle and takes an in-depth look at its hardware. Check out the photo gallery.

How Hackers Show it's Not All Bad News at the New York Times

News of a looming downgrade of NYT stock to "junk" status by Standard & Poor's sadly isn't all that shocking. I'm certainly glad I'm not an investor holding any NYT.

But there's something going on at the Times that probably won't make it to Silicon Alley Insider, much less the mainstream business press, and it's something that's starting to make me think the Times just might succeed in adapting to the changing rules of the media and publishing game (though there will almost certainly be many more casualties before it's over).

So what's the Times doing that's so important? They're hacking.

Not hacking in the nefarious sense, but in the original sense of experimentation, and curiosity, and solving interesting problems (as Paul Graham put it, "Great hackers think of it as something they do for fun, and which they're delighted to find people will pay them for.") How many other publishers are running blogs about their work with open source software? Even fewer are developing and releasing their own high-quality open source software:

Quite frankly, we wanted to scale the front-end webservers and backend database servers separately without having to coordinate them. We also needed a way to flexibly reconfigure where our backend databases were located and which applications used them without resorting to tricks of DNS or other such "load-balancing" hacks. Plus, it just seemed really cool to have a JSON-speaking DB layer that all our scriptable content could talk to. Thus, the DBSlayer was born.

That is not typical newsroom conversation.

But this isn't just about open source software, or even about some developers building cool software to run backend system. The Times has put developers right in the middle of the newsroom. At a MediaBistro event in May, Aron Pilhofer from the "Interactive News Technology" group at the Times (sharing the stage with their Editor of Digital News, Jim Roberts), talked about how the Minnesota bridge collapse was when they realized they needed to develop their own tools to cover the news with the web, and not just on the web. Less than a year later, when Hillary Clinton's infamous public schedule was released, they had the people and the skills in place to crunch 12,000 PDF documents (containing images of scanned documents) through a text-recognition program, on to Amazon's "Elastic Computing Cloud" and finally into a Ruby on Rails Web application providing full-text search across all eight years of calendars.

Just this week, the Times' Derek Gottfrid gave a talk at O'Reilly's Open Source Convention (OSCON) titled "Processing Large Data with Hadoop and EC2" based on work he'd done on the Times' archives. Again, this is the kind of talk you're not likely to hear at most newspapers (or magazines, or book publishers) these days:

I was able to create a Hadoop cluster on my local machine and wrap my code with the proper Hadoop semantics. After a bit more tweaking and bug fixing, I was ready to deploy Hadoop and my code on a cluster of EC2 machines. For deployment, I created a custom AMI (Amazon Machine Image) for EC2 that was based on a Xen image from my desktop machine. Using some simple Python scripts and the boto library, I booted four EC2 instances of my custom AMI. I logged in, started Hadoop and submitted a test job to generate a couple thousands articles — and to my surprise it just worked.

Earlier this month at FOO Camp I had the pleasure of meeting another hacker from the Times, Nick Bilton, part of the Times R&D lab -- the folks who built the impressive NYT iPhone App.

UPDATE: Nick Bilton points out via email that:

There were people from nytimes.com that were instrumental in building the NYT iPhone app also ... Is there anyway you can add a couple of words that the R&D Group 'worked with nytimes.com' to help build the iPhone app?

If you're worried about EBITDA and EPS, then you're rightly worried about the Times right now. But if you're worried about the future of journalism, and about the ability of established media companies to adapt to a digital world, there's also reason to be excited about the Times right now too.

Exploring DIY E-Reader Platforms

I've been working with the EPUB open ebook format a lot lately, but when I want to read a book in it, I have to use my computer. There just aren't any devices which support it yet. Naturally this leads me to wonder whether I could build my own e-reader.

I'm not a hardware person, but the last few years have seen an emergence of open hardware platforms designed to allow even ordinary programmers like me to modify and customize small devices. As far as software goes, an e-reader is pretty straightforward: it's just some text on a screen. That shouldn't be too hard, right?

Surveying the landscape of hardware options, I've ranked below a variety of devices from "friendliest" to "most-intensive DIY." I'm not addressing PDA or phone devices here, largely because I consider their screen size and text rendering insufficient (but plenty of people disagree).

The Chumby -- With a 3.5" touch screen and reasonable $175 price tag, this little wireless computer in a bean bag is an obvious candidate. There's a full-fledged development environment and large community of users. Most Chumby applications are written in a lightweight version of Flash, which is easy enough to develop in.

It has a few downsides, though. The Chumby doesn't have much storage space at all, so any ebooks would have to be saved online and streamed to it, a page or a chapter at a time. Since it's meant to be an always-on wireless device, that seems doable. The screen might be too small to comfortably read lots of text, as it's meant for short bursts like RSS feeds or Twitter updates.

Unfortunately, it's powered by a wall outlet, with only a small 9-volt battery for emergency backup. People on the hardware forums have managed to hack in rechargeable batteries, and I wouldn't be surprised if a totally-wireless Chumby is forthcoming. [Disclosure: O'Reilly AlphaTech Ventures is an investor in Chumby Industries.]

BugLabs -- The most open of the commercial hardware platforms, BugLabs sells individual pluggable modules that support various features, from touchscreens to cameras to GPS. It looks like a great platform, but it's very expensive ($349 for the base module plus $119 for the 2.5" touch-sensitive screen). The screen is probably too small for comfortable reading, but the company Web site promises a larger display soon.

Both the Chumby and BugLabs have touchscreens, which is key for making small screens more usable.

The Kindle -- All the heavy lifting has been done already to get into the Kindle filesystem and peek inside. It's probably too difficult to extend the existing Kindle environment without true source code, but it might be possible to do some simple things, like add new fonts. Few people have really explored hacking on e-ink devices, largely due to high cost and low availability. I suspect when the first color e-ink devices come out, used black and white ones will become popular playthings for enthusiasts.

YBox2 -- For the ultimate DIY experience, the YBox2 platform is a pile of electronic parts you solder together and assemble in an Altoids tin. It doesn't come with a touch-screen, or any screen at all: you connect it to a television or monitor. It uses the tiny Propeller chip, which powers many hobbyist devices and small robots. Like the Chumby, YBox2 comes with networking capability but little storage, and would need to stream book content from the Internet. The networking isn't wireless and of course there's no handy rechargable battery, but if you are the kind of person who can build a YBox2 you probably know how to make those too. I am not that kind of person.

While I'd be happy to crawl around a hacked Kindle, I know I'm not ready to program my own microcontroller. BugLabs seems great from a developer standpoint, especially when they release a larger screen, but I'm unwilling to shell out almost $500 just to experiment. The Sony Reader doesn't have networking, so that's much less interesting. Perhaps a Chumby is in my future. Any other options?

Stay Connected
RSS TOC RSS Feeds
 News Posts
 Commentary Posts
 Combined Feed
 New to RSS?
Newsletter Subscribe to the TOC newsletter.
Tarsier Icon Follow TOC on Twitter.
Newsletter Join the TOC Facebook group.
Newsletter Join the TOC LinkedIn group.
TOC Widget Get the TOC Headline Widget.
Search
Tag Cloud