It's been interesting reading people's views and suggestions about how to handle storage for a large photo library. The feedback I've seen on this topic in the comments to my last Inside Lightroom post has spanned quite an impressive range of solutions ranging from simple multiple disk strategies all the way to Solaris-based servers and remote copies. I've looked at several of these solutions myself and all get the job done with various levels of complexity. Regardless of the details of the solution you take or the technology you use, however, there's a basic strategy that you should follow: Keep at least three copies of your data.

Why three copies? The answer is that you want to make sure that you have a backup of your backup when your primary storage fails. Preferably, you want that third backup to be offsite just in case you have a catastrophic loss in your primary place of computing. Most of all, however, keeping lots of discreet copies of your data on different systems is the approach that people who are managing tons of data, like Google, are now taking. Instead of building fancy systems with stacks of RAID arrays, the vanguard of the storage community has adopted using cheap commodity hardware and are simply making more copies of their data.

An important thing to note in this discussion is that RAID is not a backup strategy. It only counts as one copy of your data. A RAID array should only be considered to be a drive thats faster than a single spindle and only in some cases slightly more survivable, as long as you're not running RAID 0. The theory says that you can replace a failed drive in a RAID 1 or RAID 5 array and everything will be magically come back to normal. My own experience with multiple RAID failures over the years, and that of several other people I've talked to on this issue, doesn't bear this theory out too well. Instead, it seems like a drive failure in a RAID array is a good sign that you have just a bit more time to get your data backed up one more time before things come totally apart.

The people in the storage industry know this. That's why you see lots of work going into advanced next generation file systems like ZFS. ZFS takes a different structural approach to how it keeps data on the spindles and is a much better fundamental design than what's come before. To take advantage of ZFS right now, however, you have to set up a second Solaris system. I find it fascinating that several of the commenters to my last post have taken this approach. But I know that for many photographers with big image collections, setting up a second server—with the attendant system administration duties—is a bit out of scope. As well, setting up a second server has other repercussions, not the least being additional energy use and related environmental impact. And, finally, even if you have a fancy ZFS pool at your disposal, in my mind it still only counts as one copy. You'll still want to have external backups. After all, a lightning strike can fry your computer, or a storm could flatten your house.

The good news is that you don't have to get fancy with additional servers and RAID systems to take advantage of the same fundamental strategy of keeping multiple copies of your photographs. You can do it with one system and simply load up on internal and external drives. The principle of making multiple copies of your data is independent of the technology used. The hard part, of course, is that regardless of the technology you use, you still have to set things up and keeping the flow of photographs from your primary disks to your backups going. The devil, as they say, is in the details.

I'm about half way through the process of migrating to my new storage setup that I'm implementing as part of my New Years Storage Resolution that I posted last week. Next week, I'll post more details.