Software Usability Testing for the Rest of Us

If you're a software developer and you honestly care about the user-friendliness of your software, it is not enough to simply guess which user interface works best for your application: you must test your products with real users.

A new usability testing tool for the Macintosh, called "Silverback," now makes user testing available even to those developers who have, so far, found the cost and effort associated with setting up a proper usability test lab forbidding.

Silverback follows a very simple and effective approach: record the user's actions, facial expressions, and remarks during a test session without interfering with the tests themselves in any way. It does so by both video-grabbing the computer's screen and recording the input from a built-in iSight camera and internal microphone, so all that is technically required to configure a Mac for testing is installing Silverback and preparing the software-subject under test.

Silverback's main window, displaying a Session with a video preview.

Silverback's recordings are organized into "Projects" and "Sessions." A project can contain multiple Sessions, and each Session contains exactly one recording. Besides the Project and Session names, the only meta data that Silverback maintains are a notes field plus automatic date stamp and duration info for each Session.

As soon as you initiate a new recording, Silverback hides the computer's desktop behind a black screen, and only after pressing the space bar will the actual recording commence. This comes in handy for preventing the user from sneaking a peek of the prepared test machine, ensuring a well-defined starting point for each test session. A Session recording is stopped by pressing a configurable keyboard shortcut that brings up a little bezeled dialog box with Pause and Stop buttons. Alternatively, Silverback also supports using an Apple Remote to control the application.

Once you're done recording, the respective Session inside Silverback will sport an image of the user for easier identification and show a preview of the recording, which recording will show the test machine's screen, providing visual indicators for mouse-clicks and "special keypresses" (read: command shortcuts, etc.), as well as a picture-in-picture video of the user's face.

Since Silverback does not yet let you view or edit the recording inside the application, you need to export it to a QuickTime movie before you can watch it. Which, unfortunately, also means that you need to maintain all those Session videos outside of Silverback, too.

Silverback's export preferences.

Silverback turned out to be very intuitive and efficient to use, the quality of the recorded video and audio is perfectly OK, and its user interface visuals look nicely polished. But there is something rather odd about this software. You see, according to its slogan, Silverback aspires to be "Spontaneous, unobtrusive usability testing software for website designers," and while its operation does deserve the "spontaneous" and "unobtrusive" attributes, I was not only confused why its use should be limited to website design, but, much more importantly, what features characterized Silverback as "usability testing" software...

Judging from its current feature set, Silverback is "just" a simple, well-designed screen-plus-webcam grabber with basic organizing features. As such, it finds competition in screencasting applications like ScreenFlow (product website; my review) which costs twice as much, but is dramatically more feature-rich. Then again, most of these features, like audio/video transitions or multi-track editing, are not necessary for usability testing software, thus adding unnecessary complexity and overhead.

That is why Silverback could shine if future features were clearly targeted at usability testing. To compare it to ScreenFlow one more time: just like you basically never have to leave ScreenFlow while working on a screencast, you should never have to leave Silverback for preparing, running, and evaluating usability tests.

If I had any say in the future plans for Silverback, here are some ideas I would add to its requirements specification.

As a first step (and, judging from the Play icon in some of the screenshots on this page, this feature was apparently planned as part of the original specifications), Silverback should provide video playback. I don't see a need for being able to edit the video itself, but being able to add notes on a timeline would be great. Right now, Silverback can already add chapter markers to the video, but without allowing any notes to be associated with them.

As for planning and evaluating a usability test, Silverback would become much more valuable if it had support for test cases: right inside a Silverback project, let me define test scenarios, that I could either print out or -- to save a few trees -- display to the user as on-screen "sub-titles."

In addition to using the Apple Remote to add chapter markers, let me use two further buttons on that Remote to "check off" a specific step of a test, indicating "completed successfully" or "not completed" for later evaluation.

Finally, to make that evaluation painless, give me some basic statistics so I can easily see, based on the test results from a Silverback Session, which areas of my application need an overhaul, and which work particularly well.

All in all, I love the idea of an easy-to-use and even-easier-to-configure application that brings real usability tests to smaller/independent programmers and web designers at an affordable price. Right now, though, Silverback sports hardly any features that are aimed specifically at usability testing, and yet, this application has a lot of promise: if the developers add in testing-specific features -- like maintaining test scenarios and evaluating test results --, they will have a product that is simply unique in its price range.

A 30-day demo version of Silverback can be downloaded from the application's website. A user license costs $49.95.

P.S.: Have you implemented user testing for the software you develop? If so, please share your experiences and tips in this discussion. Thanks!