Inside Lightroom

Digital Media | Spotlight: Photography | Inside Lightroom | Blogs

The Economics of Online Backup


In the comments to my previous posts about storing a photo library, quite a few people have mentioned using an online backup solution like Amazon S3 or Mozy. I've looked into this many times and keep running the two sets of numbers that matter. The first of these is how much the storage costs in dollars. The second is how much it costs in bandwidth.

Note: For me, when I run these numbers, I use a 1TB photo library as a working size. It's a both the size of my current library and a nice round number. Your library is, of course, a different size. I know many of you have smaller libraries. Some have larger. The point I'm going to make is the thought process you need to look at when considering online storage. My own solutions fit my needs, other solutions will probably fit yours. With all that said, let's dive in.

XServe RAID CloseupOne of the most interesting online storage providers, to me, is Amazon S3. It's a high availability storage solution that's being used by lots of people to redundantly store data. It's not easy to use directly, but there are several clients that make it a snap. On the Mac, Panic makes a great solution in their Transmit product, the perennial FTP/SFTP client. The nice part about Amazon S3 is that it provides simple use-based pricing. It costs $0.15 to store a GB for a month, and then there are charges to upload and download that data. Doing the math, storing my TB-sized library on Amazon S3 would cost $150.00/month. This adds up to $1800/year. With the additional charges to upload the data, that comes in at just under $2000.

OK. That's a bit of serious coin. It's less than many other solutions, however. Let's look further.

Mozy provides an unlimited amount of storage for personal, non-commercial use for $4.95/month. I know lots of people raving about this one. The catch for me is that non-commercial bit. My usage would definitely be commercial which would require a MozyPro account with charges of $0.50/month per GB, according to Mozy's pricing information as I write this post. That works out to $500/month or $6000/year. If you subscribe for a year, you get a month free, bringing the cost down to $5500. Even if you are using it for non-commercial purposes, I wonder at what point Mozy would cut you off.

Very Large ArrayOuch. You can see where this is going. Out of all the simple online backup providers I've looked at, Amazon is the one that would currently win my business.

Finally, I should mention that Photoshelter—a company that I use for distributing some of my client work—offers a TB of storage for $1000 year. This is a pretty good deal, especially considering what Amazon S3 and Mozy charge. You can even send them a hard drive with your data files on it to upload for you. The downside, for me at least, is that files on Photoshelter are disconnected from your local filesystem with no ability to sync. This means that the metadata is split from your Lightroom library. If you make metadata edits locally, you would have to propagate those up manually to Photoshelter. The result is that, while I like using Photoshelter as a outbound avenue for my photographs, in addition to Zenfolio and Flickr (two other websites I use to push my photos to clients and the world), it's not yet set up to be a good recoverable backup data store that can be synchronized over time. Of course, you may decide that these aren't issues for you and that you're willing to have just your RAW files without your metadata in case of disaster. If this is the case, you should definitely consider this service.

Beyond the monetary issue, there's also the issue of bandwidth. Photoshelter wins here by letting you send in a hard drive for uploading data. While FedEx and UPS have a bit more latency than a TCP/IP connection, you simply can't beat the bandwidth of sticking a hard drive into a box and having it show up somewhere else the next day with all of its data. With Amazon S3 and Mozy, I don't know of an equivalent service, therefore you have to wait for the uploads to happen. How long? Well, for me, my upload bandwidth is a pretty respectable 768Kb/s. Even at that speed, however, uploading a lot of data takes a lot of time.

KeyboardHere's the math: 768Kb/s is 96KB/s is 0.09375MB/s. Multiply by 3600 and you get 337.5MB/hr. Multiply by another 24 and you get 8100MB/day—equivalent to an 8GB compact flash card per day. From there, you can extrapolate out to about 130 days—or a bit over 4 months—to upload a TB of data. I could always "borrow" a T3 at a friends office, but even with that, it'd take a bit over 2 days to upload the data—assuming, of course, that S3 could intake the data at that speed.

Even once you get your initial data upload taken care of, there's the issue of incremental data uploading. It's not unusual for me to shoot 40GB or more for an event. Even when I'm not shooting conferences, it's still entirely possible for me to shoot 8GB/day or more for days or weeks on end. And, it's not necessarily practical to tie my upload bandwidth to backups for so much of the time.

Now, let's look at the recovery side of the scenario. My current inbound bandwidth is a 6Mb/s DSL connection. I actually see about 5.5MB/s. Doing the math again indicates that it would take 18 days to restore a 1TB image library. Now, granted, in a recovery scenario where you're relying on offsite storage to recover, this might not be a total show stopper, but it is a concern. I'd really rather prefer to have a restore—even from a complete loss of my home—take no more than a few hours or maybe a day.

As I see it, I need a much bigger pipe to the Internet to make these kinds of backups work well long term. Until Verizon FIOS shows up, or I score some really choice client gigs that really add to the bottom line and can put in my own dedicated T3, I'm a bit too bandwidth constrained to make it work out. This, however, is how things stacks up for me. After running the numbers yet again, I'm going to stick with my current program of rotating drives to a safe deposit box which has a nice cafe next door in which to get a cappuccino in. But, I'll keep running the numbers every so often and at some point, maybe the equation will change answers.

If you have less data, or have access to a bigger pipe, the bandwidth economics will be different for you. If you shoot quite a bit less than I do, which wouldn't be surprising as I do generate quite a bit of data, the economics may very well line up. Or, if you're running a more capitalized business than I am and have larger pipes to the Internet, the numbers might work as well. You should go through your own calculations to see if an online storage solution works for you both in terms of money and in terms of bandwidth. And, if you're willing, please share your online backup equations and experiences in the comments. I'm sure that many other people could benefit from your findings.

Photographs in this blog entry are Copyright James Duncan Davidson, All Rights Reserved.





AddThis Social Bookmark Button
Comments (91)

91 Comments

Malte said:

Recently I did similar calculations for myself and already stopped after the bandwidth issue started to materialize. I didn't even look at the costs; at least with todays common bandwidth online-backups aren't really feasible for large amounts of data.

But if you really want to do online-backups, a combination of the Photoshelter and one of the others might work: Snail-mail your massive amounts of data to them and store it in a flat namespace. Then you keep the metadata somewhere else where its easy to update, like S3. Folders are just metadata, too, just pointers into your flat namespace. To glue all that together, you need to create some kind of abstraction layer of course, on Linux I'd probably write a FUSE app. I have no idea how to combine all that with Lightroom, to keep on the overall topic, but that's how you could combine the best of both worlds.

But after all, the deposit box is probably still easier and cheaper :)

Lyle Johnson said:

I could have used this analysis about a week ago, when I experimented with uploading my (much smaller) iPhoto library to S3, via JungleDisk. I only have a few gigabytes' worth of photos to backup, and it still took several overnight sessions to complete that first sync all of the data.

Bob Lee said:

I've been thinking about stowing some external hard drives in my office at work and automatically syncing up remotely via the VPN. I'll have to measure my upward bandwidth though. In the event of a disaster at home, I would just go pick them up, much like you'll do with your safety deposit box.

If you don't have an external office, you could exchange hard drives with a buddy. You keep a couple hard drives for me, and I do the same for you. We both set up ssh accounts so we can sync remotely (assuming you can get enough bandwidth).

Bob Lee said:

Also, the idea behind stowing my hard drives at work is I can do the initial (BIG) sync locally and then just do incremental updates remotely.

Jenny said:

For all online backup and storage related info, I recommend this website:

http://www.BackupReview.info

Fazal Majid said:

For my offsite backup needs, I usea D-Link DNS-323. This small 2-drive NAS runs Linux, can be easily enhanced to support NFS and rsync/ssh. It is inexpensive, compact, power-efficient and reasonably powerful. I keep mine in my office, loaded with 2 1TB drives in RAID1, seeded it by rsyncing on my Gigabit home LAN, then refresh it by having it rsync from my home machine every night while I sleep (my office firewall won't let me push from home to the DNS-323, so I pull from it instead). As a bonus, since my backups include my iTunes library and it has a built-in mt-daapd, I can listen to my entire music collection at work from iTunes over the network.

This is a great post. Actually, as I saw you were starting to talk a lot about that backup issue, I was about to ask you why you didn't talk a bit more about online backup systems. I've done the math myself, but I was still wondering if I wasn't missing something, with all these people talking about S3 as the definitive solution.

The discussion about your use of PhotoShelter as both a delivery website and a backup solution makes me think that we probably need another service : a web site that would take your disks and provide publication of photos on them so that you can monetize them. But do we really need to send them raw files, if it is to publish them on the internet? Do we need so much space?

Also, if you ask a service such as Photoshelter to be a backup of your files, you may not need to restore all your files at once. Don't they provide a 'snail mail download' too?

anon said:

Why do online backup at all?

Why not rsync to a big hard drive, and store it at a friend's house?
You could add a second drive and rotation for redundancy.
Make it automatic with a cron job. You just have to remember to take the disk with you when you leave your house.

Julie said:

I'm using Carbonite http://carbonite.com and very happy with it

Norman said:

Smugmug offers unlimited storage space to account holders, with a cap at 12MB per image for their standard and power user accounts. This still does not address the limitations you found with photoshelter (metadata split with lightroom, etc) at a signifigantly lower price (149/yr for pro accounts).

A comprimise would be a combination of this and offsite storage of a external HDD or archival DVDs, IMO.

(disclaimer, I am a Smugmug customer.)

Malte: Indeed, the problem with the Photoshelter approach is the cost/complexity of recovery. The safe deposit box does win, at least for me.

Bob: Indeed, putting your spare copies at your office is just as good as my using the safe deposit box. The idea is to get your 2nd copy into a place where it can survive anything that's likely to affect your primary location, short of nuclear war of course. And, of course, I know you have mondo bandwidth where you work.. Hrm... I'll be right by! :)

Fazal: That a great way to solve the initial upload issue for sre.

Cyril: There are lots of places to monetize your photos, but RAW files aren't part of that game. I wouldn't ever send a client a unprocessed RAW file. A DNG with initial processing instructions, sure, but never a RAW from the camera. Of course, I may change positions on that over time. As far as a snail mail download, that'd rock.

anon: Storing a clone at a friends house also accomplishes the same thing as my storing clones in the safe deposit box or other people's solution of storing at work.

Norman: Indeed, Smug does offer unlimited storage, but the same bandwidth and restore problems exist. Smug seems best suited as an endpoint for your photography to print, display, and show. But that's just my take.

Peter Krantz said:

For 1Tb data online storage may not be an option. For smaller libraries and if your rimary reason is backup you have a lot to gain. I did the math and for my library of 5+ Gb I pay about $2 per month including the requests and transfer of new photos shot during a typical month (see my name link for some details).

This is of course a lot cheaper (and safer) than getting an external RAID.

ljun said:

has anyone considered trying out crashplan? They offer similar rates, but their client app is pretty nice, and it can store single files bigger than 5GB (limitation of Amazon s3) .....

?


Stanley Seibert said:

I'm using CrashPlan now, and I'm very happy with it. It's very bandwidth efficient, and the ability to store duplicate backups on friend's computers (and restore from several sources simultaneously) is pure genius. The cross-platform aspect is nice as well. I backup my Mac to both Windows and Linux systems.

The economics of CrashPlan are different than Mozy, though. Mozy is a billed like a service, so the software is free, and the storage costs you. CrashPlan costs a fixed amount for the program (incl free updates for a year) and storage on the CrashPlan servers is a secondary service you don't have to use. If you do use it, it costs 10 cents per GB per month, with $5/50 GB minimum.

You can also store your backup data on your friend's computers and avoid monthly charges entirely. I'd suggest trying the 30 day trial, and then if you like it, going straight for the $60 Pro version which allows you to retain previous versions of your files. It's not as sexy looking as Time Machine, but CrashPlan Pro can be snapshotting your work continuously anywhere you have an Internet connection. (Great for a laptop when you don't want an external drive always attached, and don't have Time Capsule.)

But, before CrashPlan, I'd first go buy an external drive and make a bootable image of your Mac once a week with SuperDuper, or Carbon Copy Cloner. (SuperDuper is my favorite.) Only then should you supplement with online backup, since no matter what, it will be SLOW. You should only rely on it to recover things since your last disk clone, or to cover you should you lose both your computer and external disk at the same time. (Fire, flood, theft, etc)

An external drive + SuperDuper + CrashPlan was the best $150 I ever spent.

Pete said:

I recently sent a Mac mini into a data center to act as my backup. I went with Macminicolo.net because you can throw a 1TB drive on there connected to your mini. With that, I was able to send my large library in the mail, and now I can just do updates via transmit.

Grega said:

I second the SuperDuper + CrashPlan suggestion. My backup setup is the following: SuperDuper makes a bootable copy of my system disk each night. That way if my hard disk fails, I can reboot from the external disk and work on with maximum a day of lost data. CrashPlan and Mozy are making online backups off-site. A more detailed story follows.

After a friend of mine got burglarized I realized that on-site backup does not really help when the bad guy takes away both your computer and backup disk. That was when I started looking into online backup solutions. My first stop was Mozy. It worked great for the first 2GB and I bought a subscription plan. After the first full backup (60GB) I started having problems with the client loosing connection and also taking lots of RAM. The issues were so annoying that after weeks of failed backups and maxed out RAM I just uninstalled it and only tried it again now and then when a new beta version was released.

In the meantime I ran across CrashPlan. I liked the idea and having some space available on a Mac located on another continent, I gave it a try. The client is not as flexible as in Mozy in terms of selecting what gets backed up and what not, but the lack of features is made up in simplicity and efficiency. Over a few weeks all 68 GB of data got uploaded and new and changed files get updated each day. I bought the basic license ($20) and am now considering spending additional $40 for the pro license, which enables continuous backup (vs. once a day) as well as storing multiple file versions. If you have access to an off-site computer with some extra space or can convince somebody to share bandwidth and run a local copy of CrashPlan for you (you can make it a reciprocal deal), I believe it to be a great solution. All the data gets encrypted before sending off so there is no worry of it being compromised. To avoid long initial transfer times, one solution could be to perform initial backup on-site over fast ethernet connection, ship the HD to final location and just do incremental backups since.

Just when CrashPlan started working fine for me, new version of Mozy client came out and I gave it another try. While it had serious issues with maintaining connection before, it now works fine (based on some detective work it might have been an issue with my ISP). In the last few days I have updated about 15 GB of files and have 10 GB more to go (estimate is 2 days worth of upload left). The client seems to be behaving better now as well. It can still gobble up to 500MB of RAM and over 1GB of virtual memory, but it seems to more gracefully give way when I'm running lots of applications. I'm hoping the RAM usage will go down once less data needs to be updated.

Future plans? SuperDuper is a must. I need to have a functional mirror to switch to at any time. CrashPlan has worked great for me (I have used it to restore mistakenly changed files a number of times). I still have 6m worth of subscription to Mozy left and will see how that goes as well.

I've previously used a colorful metaphor to describe your bandwidth problem, it's like an old saying called "The Marching Chinese." They say that if you got all the ~1 billion people in China to walk 4-abreast past one fixed spot, they would never stop coming. They'd be lined up so far back, that people would give birth to children while waiting in line, and the children would grow to maturity before they ever crossed the line. And they in turn would give birth to more children who would grow up in line. So the entirety of the population would never cross the line, they'd keep coming forever.

And that's the way it is with the online storage bandwidth problem. I can easily create more data in one day than I can back up in one day with my little DSL connection. I can easily do that EVERY day.

Anyway, apologies for the slightly racist metaphor, it was taught to me by my crazy racist high-school biology teacher (who also had a PhD so I figured he must know something).

There is also http://www.bingodisk.com/ where you can have 100G of online storage space with unlimited bandwidth.

The online disk can be mounted locally (webdav) and you can move files to it as if it was another drive. I'd recommend using Knox to create a file vault on the bingodisk to ensure secure encrypted storage. I'd also recommend logging into the bingodisk via https for a secure channel.

I use rsync to manage my backups to a local external hard drive and also to a colo machine, both using Knox vaults. You may read a blog post I wrote about doing it to a remote machine over ssh also.

I'm very happy with Jungle Disk, which is regularly updated and supported by its developer in combination with Amazon.com's S3. The cost for us photo-less mortals is much less per month, but I really appreciate the economics laid out in this post. As well, I've been trying Jungle Disk Plus, which is a browser-based means to access your backups. It's very impressive.

paul burd said:

I've been looking at this problem for a long time too. The only solution that has been feasible for me in terms of cost, time, and bandwidth is to do nightly backups to an external drive (kept locally), and monthly backups to a second external kept off-site.

I use Time Machine for daily, and SuperDuper for monthly.

Online backups just don't work for extremely large amounts of data (yet).

Jay Kerr said:

I follow the same approach as Paul for my business -- daily backups with Time Machine and monthly backups using SuperDuper.

A couple of years ago my office was robbed. Twice! The only thing that saved me were my backups. I lost almost a week on the G5 that was stolen but I was up and running within a couple of days using backups.

I also archive projects to DVD which takes a lot of time but to me the redundancy is worth it.

I also have a lot of extra storage on my Dreamhost account and tend to use that occasionally when I'm feeling extra paranoid (mostly for email).

Even with much less data to back up (around 100Gb) I came to pretty much the same conclusion regarding time to back up and restore with online services. Really like the idea but it's just not practical for me at the moment.

See the long story on my website: http://www.zx81.org.uk/computing/opinion/backup.html

Chris said:

I'm still looking for the ideal online backup solution.

Right now I'm using Mozy (Home) and it's ok. My biggest complain is that even with the *paid* (non-commercial) service they throttle your upload bandwidth to 128 KB/s—and don't tell you about that loudly and clearly before you painfully find out yourself.

JungleDisk looks promising. But until they finish the file system integration on Mac OS X, it's *not* usable for backups IMHO. That's because of the local WebDAV solution which means, files you copy onto the JungleDisk loose their original permission setting. Quite unacceptable for a real backup!

(Gaah, why do these stupid captchas never work??!!)

Unfortunately online backup will never be viable. It would work for people with a little amount of data, but anyone wanting to backup anything more than basic Word documents will always hit a brick wall. File sizes are increasing, as are hard disk sizes. This means we can store a lot more.

Internet speeds are increasing much more slowly so the gap between the speed we can transfer and the amount to transfer will continue to grow. On top of that, storage amount is pretty much limitless, whereas internet speeds are at some point in the future going to hit a brick wall, meaning the only to make accessing a site faster would be to have servers closer to you geographically. Of course by then uploading a photo would be instant, but I'd assume by then that 1TB of photos would have increased massively.

This may seem strange, but its price/performance is undeniable: I use a bluehost
http://www.bluehost.com/track/mrflipco/text1
webhosting account for remote online backup. (That link is a referral, or you can go straight to http://bluehost.com). For under $100/year, they give you 1.5TB of disk and 15 theoretical TB of transfer, with full ssh and FTP access, all common GNU/linux tools and the ability to install your own.

Now, this obviously does not provide the hardened multiple redundancy industrial security that Mozy or S3 gives. And it /is/ shared hosting -- strongly encrypt anything sensitive. But commercial use is fine, $100/1.5TB is less than the cost of a new commodity hard drive, and they do weekly and monthly backups of your data themselves. The data can be a live copy of what's on your desk, or live as encrypted tarballs, or even be selectively linked into your web space.

For advanced fu, use the rsync-with-hard-links method outlined here
http://blog.interlinked.org/tutorials/rsync_time_machine.html
to do a poor man's time machine to your hosting space.

The only limitation, then, is your upload speed; you need to push 600kB/s upload to fill 1.5TB in a month, 50kB/s upload to fill 1.5TB in a year.

Martin said:

Yesterday I just cancelled my Mozy account. Since I signup in November 07, Mozy client (Mac) never completed backuping my 500GB data.

In the beginning the client was not stable enough, it crashed everytime the configuration window is openened. They fixed that problem in later revision of the software. But then it just backs up things when it wants to; not continuously. The amount stored in the backup stayed around 49GB, never higher. Then I get to see this pop up every day saying no succesful backup has been performed in the last 7 days. Extra annoyance was, my MBP fan was spinning everytime Mozy client did its thing. 400 something bits encryption, cool... but can you do with less encryption and keep my processor cool please?

The good thing is that, when I asked they refunded my money immediately.

Now I am looking for one of those cheap shared hosting that gives hundreds of GBs. With rsync + encrypted disk image (Leopard's sparsebundle) it should work better

It's not unusual for me to shoot 40GB or more for an event.

40GB of backup worthy photos? Every event? Surely you could cull those back a bit before backing up? Even if you're shooting RAW that's an insane amount of photos to backup. Backup the best 50, convert the next best 50 to JPEG and perhaps back them up too. Delete the rest :) Most of your bandwidth problem's are solved.

I have had a somewhat different experience with Mozy and S3/Jungledisk than previous commenters. I found Jungledisk for the Mac unbelievably slow and inefficient at uploading and even just getting it to list my files already backed up required an interminable wait. Meanwhile, Mozy works fine, uploads at high volume and has a number of useful preference settings for scheduling backups and limiting bandwidth usage if need be. I have two computers backing up a total of over 100 GB. And automatic daily updates, I don't have to lift a finger. Mozy also lets you use your own encryption key if you so desire, meaning no one, not even Mozy, can spy on your data even en route.

I do find it hard to believe that online backup services could make any sense at all for someone with 1 TB of data, however, thanks to the lame-o state of broadband connectivity in the U.S. In Korea maybe -- but not here.

Jory said:

While I agree with you for large scale backups, what I find Mozy (and its ilk) useful for is backing up smaller important data, such as my Address Book, QuickBooks & Quicken data, keychains, iCal calendars, and other smallish data. I've been using Mozy Free (since I'm really not using it for commercial purposes, just nightly safety for my laptop) and haven't even exceeded 250 MB of space.

For my fiancee, I've got her desktop G5 and her iBook setup to do the same thing, as well as to backup her vacation photos, as we're currently traveling in Norway and I'd hate to see her lose anything she hasn't already U/Led to Flickr.

Tony said:

Anyone have a recommendation for a Windows equivalent of SuperDuper?

ldw said:

I used Mozy (home version) to back up about 100GB. I do have Verizon Fios, so bandwidth is of no issue to me. I find it a very good service, I have had no issue with it what so ever. My initial archive was about 50GB and it did take almost 2 weeks to back that up, but now it just does delta changes while I am asleep...

Brian Hutchison said:

You could check into ElephantDrive, they offer a 1TB Pro option for $350/year. Works on my Mac, but I am just using the free versionn now while I taste test it.

I have essentially three sets of data. Documents and such (TaxCut files, eBooks I've purchased, etc.). I've used StrongSpace for that so far. The other set of data is multimedia, RAW files from my camera and FLAC files from ripping CDs I own. The final set of data, if you will, is system backups... SuperDuper/TimeMachine backups. I use an external drive for these (I do both SD and TM). Its the massive amount of multimedia files I struggle with... 4.7GBs per DVD, burning two copies of each is a logistical nightmare.

Thanks to prior comments I will check into CrashPlan for some of this data.

John Whitley said:

@Martin Pilkington: "Unfortunately online backup will never be viable. [...]Internet speeds are increasing much more slowly so the gap between the speed we can transfer and the amount to transfer will continue to grow."

This isn't true. Network bandwidth (at all scales, WAN/LAN/local bus/etc.) also has a doubling time (i.e. follows the colloquial version of Moore's Law). It's just that this trending has a very pronounced step-function style of behavior, especially for the broader scales. Intuitively, consider that there's infrastructure that must be built out for each wave of change. National/global infrastructure doesn't change overnight. Using my own personal download bandwidth as an example, I started with 300/1200/2400 baud modem access, then a 28/56k modem, then a 1Mbps up to a 7Mbps DSL line. Each of those main phases lasted for quite some time before the sudden orders-of-magnitude transition to the next phase.

The implications of this are as follows: right now, online backups are useful for some people. By the time the next network bandwidth transition happens, the economics will dramatically change the utility of these services. I doubt that they'll entirely replace local backups for data-intensive users, but just as cheap portable hard drives have recently transformed our backup options, cheap and massive network bandwidth will also extend our backup options.

Raj Prakash said:

BQBackup.com anyone?

Charles: You can state the metaphor the same way in global terms as well. With 6.6 billion people on this planet, we're growing fast as a population.

Michael: Of course there aren't 40GB of worthy photos for every event. But, the rub is that it takes time and effort to cull images. Many people have made the assertion that it's cheaper to keep everything than to take the time to cull. There are other problems with deleting too much as well. If I were just working on personal work, then I could delete more than I can for clients. In my client work, I've found that sometimes I need to go back into my non-selects and dig for other images for particular purposes.

Obviously, what I describe is my own situation, but I've talked to many professionals who don't throw away anything. Nothing. Zippo. Nada. And they all state reasons that fall into the same ballpark.

I typically shoot 14 gigs at a wedding and generate .psd files up to and above 100GB in post processing. When I get home this is backed up to dvd and two separate jbod arrays that are superdupered nightly. One of these is housed in a case I leave near the door in case of fire I can grab it on my way out. Once a week I visit a friend and back up the case to a third array.When i have finished processing an event in Lightroom it's exported as a catalog and popped on another dvd.
I have been looking into online storage, colocation etc for ages and have never found a feasible solution. I think the very best solution would be to run a firewire cable into your neighbors attic and install a bunch of drives up there. In the meantime I'm looking into installing some kind of safe in the garden shed.
Yes I'm paranoid :)

Paul said:

I came across this recently, 1125 gb storage and 7500 gb transfer for $9.95 a month from webhostingbuzz.com. I am not sure how they can stay in business with those prices.

Paul Jameson said:

I worked this out some time ago but couldnt have explained it so well. Nice post. A must for high volume, low value data users in personal or small business scenarios.
I am a freelance photographer. I used multiple free online backup accounts from two pcs and a laptop. I didn't mind the hassle of splitting up accounts to keep them in the free category. I did not see in advance the broadband issue and now realise that to backup online properly I would not alone need a commercial online backup account but signficant broadband in terms of speed and cap. My wife runs a small bookeeping business at home. She backs up all her critical data (mostly word, excel and accounts) with www.backupanytime.com for less than US 40 per month (They are a euro outfit in Great Britain) and she is very happy. I emailed them and got a quote to do the same. They got me to run a broadband test on their site and detail the quantities. They came back and quoted Euro 289 per month and explained I would have to treble my broadband for it to work! Thanks but no thanks. I now undrstand that the online backup model is great for those it suits and a non runner for those it does not. Any inbetween is a coplicated mess.

Paul

michele said:

HI!
Thank you for your comments- from reading the above it confirmed that i made the right choice. We use onlinebackupvault.com - we played around initially with a couple of other guys, but OnlineBackupVault.com is a keeper. THey offer free software and free agents for Exchange & SQL, their backup speeds are fast and they charge us by the space used once compressed.
And probably most important their support is out of this world

Jack Duggan said:


Paul, I use www.backupanytime.com also. Did they contact you recently with a free offer to add another mirror server to your account? They called me and I was too busy but I called back later and not alone did I get an additional mirror server but a software upgrade and a free data audit!
I was informed that the new data centre was in Ireland (I am Manchester) and that if I wanted to backup in UK only I should leave my backupanytime account status as it was. I had no issue with adding an outside data centre and went with it.
Cool or what!

TC60045 said:

The Chinese March analogy is fun, but flawed. Here's the math:

Chinese people: 1,000,000,000
standing in a line, this number abreast: 4
means this number of people in a single file: 250,000,000
assume this many feet between people in that file: 2
total distance, in feet, from 1st to last person in file: 500,000,000
same distance in miles: 94,697
walking speed, in mph: 3
hours for last person to cross: 31,566
hours that last person can walk, per day: 16
number of days required: 1,973
number of years: 5.41

Now, if you use the whole world's population (6B) you'd think that indeed the world would keep coming forever. But if everyone is walking, they wouldn't have, in the words of Bret and Jermane, much "business time."

I've suffered through the bandwidth math, too. From what you describe, your challenge is not the weekly additions to your library, but the initial backup. This is my challenge, too.

I've debated putting a duplicate of my 2TB RAID-6 server at my parents house and using standard DSL between to catch weekly additions, simply using rsync.

If you did this, you'd have your initial backup solved. You then have to keep your additions to 8GB*7=56 GB / week (note: you assumed you get 100% of your upload speed, which is not a practical assumption, but you also didn't factor in any file compression, which can squeeze 20% out of .TIF or .CR2 files if you zip them up, so I'd call it a push).

Thus, a $300 P4 2.4GHz box with a $300 Ciprico raid controller, and 4 1-TB (4@$300) drives comes to $1800. Monthly storage costs = zero.

I could also just gamble and build a lower-end machine on my parents end with two 1-TB drives as JBOD, and that brings it to $900. But when (not if) a drive bonks on their end, I have to make a long drive to replace it and I don't see them *that* often.

Great post to the OP -- thanks for putting this out there!

sandy said:


Most people really missed the point, speaking of online storage / backup, it is not the amount of storage that matters, it is really about the application that matters. In reality, out of 100 small businesses / home users, less than 1% of them would need more than 50GB of storage, unless you backup all the operating system junk files.

I have used DriveHQ Online Backup and am very happy with the speed and reliability. Yes, I have a 120GB hard drive, but my data files are less than 10GB. The first backup does take some time, but the subsequent backup is extremely fast. DriveHQ usually can achieve a 2 -3 times performance boost when uploading files. So even if you are on a 384Kbps ADSL connection, your upload speed could be 3 times faster. In fact, contrary to the author, I think Online Backup is the way to go, at least, good services like DriveHQ Online Backup.

Frank Larou said:

"less than 1% of them would need more than 50 gigs"
I understand your heart is in the right place but this is akin to an old quote from Bill Gates suggesting that noone will need more than 1 meg local storage. The problem really is that most people can not be trusted to select the correct data and watch and amend this on an ongoing basis so complete or near complete backup are more common than you may think. As per the post on backupreview.com above I tested 6 of the sites favourites offerings. I found http://carbonite.com and http://www.backupanytime.com both had a very useful auto select feature which had its own artificial intelligence which familiarises itself with user data types and highlights anomolies. This may be the answer to keeping the selection and transfer amounts below 50 gigs, at least for a while...
Frank

Greg Loomes said:

I agree with Frank. We backup specific critical data only and it works fine. If we tried to backup everything it just wouldnt be feasable or necessary. Online backup is an excellent option if the user is capable of managing it or has local IT support to do it.

arnold jones said:

I use a service from www.onlinedatastore.co.uk , they charge 9.95 per month for a 300GB account and I get free seed loads. For every day use I backup the normal way, upload incremental changes etc. When I dump 10 to 20GB plus on my pc after a shoot I use their software to create a seed load (copy of my data) onto my USB drive, I send the drive to them, they transfer it to their data centres and send the disk back to me within 48 hours.

Daniel said:

I suggest to try http://www.memopal.com/en online storage service. It is really long term storage, versioning and real time backup. Mozy takes very long to backup your files and more than 30 mins to get a single file back. It depends on the fact that the two products have different technology behind. Memopal is really optimized to manage the file too.

I use Memopal online backup service to store my photos. Thay offer 250GB at 99$/year and you can upgrade to more GB (as many as you need). They give you real space (Mozy and Carbonite just stops you at 50GB than you have to pay for every GB extra, they says the service is unlimited, but it is not, read the terms). I’ve chosen Memopal reading this page on Wikipedia and the guide at the end of the page: en.wikipedia.org/wiki/Remote_backup_service those whitepapers are very interesting.

Jeremy Sutherland said:

I've used Vaultwise Secure Backup with good results. They do not cap upload, and download speeds. With their web access you can retrieve files from anywhere, and all files are encrypted before transfer. I did substantial research on some of the others like Mozy, and found they have many stipulations on their "unlimited" data service. I would rather pay more for a premium service that is reliable then a service with hidden agreements.

Steve said:

Skadoit is a great place to store your offsite backup. The great thing is it;s affordable, they do seed loads where you transfer data via external means and the data is uploaded to their servers. After this, they perform in-file transfers which can be done for 5GB in one night. Ther great thing is Skadoit is an opportunity to earn money, real money, by sharing this service with others. They have 5 ways to earn income. Check them out at www.skadoit.com. This is cool because you can get paid for backing up your stuff. And paid good for such a small investment.

Mat said:

Bluehost cannot be used as a backup storage:

From the TOS
http://www.bluehost.com/terms_of_service.html

Backup Storage
Bluehost Web Hosting intends to offer services to host web sites, not store data. Using an account as an online storage space for archiving electronic files is prohibited.

The same applies for dreamhost and probably all other web hosting providers in the consumer segment.

Regards,
Mat

VizionQuest said:

Sounds like you were as discouraged as much as I was with the virtual sea of service providers in this area. There's sooo many online storage and backup service providers out there these days. When trying to find a solution to for my own needs I ran into this issue of the variation of how they display their pricing. I thus embarked on a review of over 35 top providers and decided to share it with everyone. It includes a comparison spreadsheet that evaluates each competitor according to price ($/GB/month), alloted storage amount, bandwidth, and the features they offer. If you have any suggestions for the comparison please speak up.

I hope you find it helpful: http://tomuse.com/2008/09/01/ultimate-review-list-of-best-free-online-storage-and-backup-application-services

data backup said:

dis better to have your stuff backed up and not need than to need your stuff backed up and not have o_O
-jack

Steve said:

I found the best online backup solution at www.skadoit.com. They are affordable at $6.95 monthly or $69.95 yearly, they provide 100% free support, they support home and business users, MACs, Unix, Linux, and windows. They backup databases with their agents: Oracle, SQL Server, MySQL, etc.

PLUS, they do quick loads for large clients, use a technology which backs up only the changes in the file to keep backup space low, they offer continuous backups, and backup files while they're open.

Check them out: www.skadoit.com

Tom said:

Super article, good read on online backup as a whole

Tom
http://www.databackup.ie/

Jack said:

For the best value in online backup, look at http://www.MyOtherDrive.com. They support 128-bit encryption, scheduled unattended backup, and complete public and private file sharing. Their prices are better than any other online storage site I have seen.

Joe said:

Mozy, Carbonite and other online backup services are most definitely built with the consumer and small business in mind. We have reviewed dozens of online backup services and we haven't found one yet that offers a really compelling, turn key solution for the power user/medium size business that doesn't want to integrate an enterprise solution. S3 isn't something we had thought about, so it was great to read your comparison/

JP said:

I have used Mozy and now use secobackup for my personal needs. The GUI is pretty bad but the software is a workhorse and backs up very large files easily. In fact, I have some archives over 10GB that I have tested.
JP

Jaskiran said:

Here is a very good article about ...

"How Do I Choose the Best Online Backup Provider?"
http://www.backupreview.info/2008/02/13/how-do-i-choose-the-best-online-backup-provider/

Anonymous said:

For me, the best, cheapest and highest quality online backup is www.secobackup.com. There is no license fee for software download (free), support large backup sets but like JP says, the GUI is leaves a lot to be improved upon. I hope those guys are listening and improve the UI.

I am even thinking of putting this software for servers which I have at work.

Bye, Youel

Vizion Quest said:

This is just the tip of the iceberg really. I've been trying to keep track of all these online backup providers. I finally had to create a comparison chart to sort and filter them by price, storage amount, and features. Hope you find it helpful, let me know if I left any important ones out.

jay said:

I would have to agree that backing up your data remotely is the way to go. Do yourself a favor and back it up for free.

Danielson said:

I use www.memopal.com and www.megaupload.com

vosi.biz said:

It is interesting to note that many of the online storage / backup service providers are already gone, e.g. XDrive, MediaMax / Streamload, OmniDrive, etc; With today's business environment, it is expected 80% of online storage or online backup companies cannot survive because they never had a profit model other than offering cheaper or even free service!

A company in such a crowded market space must be able to really compete, but not just by offering cheaper / free service. VOSI.biz is a different company, we focus on product quality and service quality, we are keen to develop better products, more features than any of our competitors. We target premium users and business users who are more willing to pay for our services. We create value and save money for our customers.

Please visit us at: http://www.vosi.biz/. We offer a lot more than Online Storage. Our services include Remote File Server, FTP Server, Email Server, Automatic Online Backup and Email Address Book backup services.

Martin said:

Great post and good comments section - I've learned a lot from reading through. Whilst I don't have anything like the need for 1TB of data backup, I would like to do something other than copy onto HDDs that I then leave at work and family members houses! The one I'm most interested in from those above is MyOtherDrive.com. Do people have good reports of using their services? Certainly at $240 a year for 1TB (at time of comment) it is a lot cheaper than the likes of Mozy who still charge the $0.50 per GB.

Martin

Angila said:

Amazon S3 is a great service with a great price. I rate them 10 out of 10 period. I'm using it mostly for videos and it works great.

Angila Jones

Doug Clutter said:

Good post about some of the costs and concerns that come with online backups! Here we are 18 months later and your post is still pretty much right on target.

One point you seem to have missed is that most backup services compress the data pretty hard before uploading it. 30-70% is not uncommon. Since you are working with digital photos (and a lot of them too!), compression wouldn’t help you much.

However, most users are not like you. My 70 year old dad uses our online backup utility to back up literally years of family pics which amount to a little over 1.2GB and it costs him less than 20 cents a month to store on Amazon S3. He loves it!

Bottom line: the Internet is fast enough today for probably 90% of people to use an online backup method. It’s just not fast enough for everyone yet.

Arun said:

In online backup, the most overlooked feature is the security. Mainly because it is a relative term and is not measurable with the normal ways. How secure and fail proof is the storage mechanism is the real question all should ask before buying. Cost per GB changes drastically but reliability drops accordingly.

Pat.R said:

Yeah I lost all of my pics because I did not back them up, that is what i hate about digital photos.

John Doyle said:

For businesses you should be able to request a copy of your data to be shipped on external USB drive next business day to you by fedex etc.Also many on the offsite providers give you the option to also keep a local copy for the last x number of days.
So your DSL pipe should not be that much of an issue unless you are uploading gigs of changes/new files on a daily basis.
In this case you might be better off with a combination of local and offsite storage.

Mehrdad said:

Finding a good vendor on the web is like going on a treasure hunt. They are out there, you just have to do your homework if you don't want to be burned in the long run.

The marketers at multi nationals like Amazon pull buyers in by making them think they are getting something for nothing, but the fact is that not all services can be commodotized. Specially whn it effect people's intellectual property like confidential data.

The Amazon commoditization model will not work in data protection simply because something, like luxury, cannot be commodotized. The state of our economy is a great Macro example.

For quality backup service visit us at carrott.com

Joel said:

One service that I use as Online Photo Backup solution is Backblaze.

Backblaze´s business plan offers unlimited online backup for a flat fee of $50 per computer, per year, unlike Mozy that charges per GB, per month.

The file transfer speed is only limited by you internet connection, and you can decide how much bandwidth you will use for your backups (bandwidth throttling).

The problem is the restoration, it is always a drag.

The only solution that I have come up with is to backup my files to onsite locations, so that I can easily restore them (restoration from onsite locations is more than 30 times faster). And to use an online storage service like Backblaze to be protected against natural disasters.

I also backup my files to other off site locations such as my friends’ computers using great free software called crash plan.

CrashPlan garantees same day recovery!

I guess if you have a lot of friends, you can easily backup a 1TB of pictures with them....lol.

All the best,

Joel

Anyone reading the crashplan comments above. I have written up a diary. In summary memopal bad, crashplan good!

http://martinaharris.com/tag/backup/

jack said:

a great article, you should write a book (or maybe you have already) ... thanks

Kumar said:

I agree wholeheartedly in the importance of chooses a data backup that is a good fit for the individual, I myself use the KiboTec backup software from the company I work for as it was much more cost effective that the other backup software that I found online as there is a pay once option or monthly option and it didn't cost me $1000's per year which I must say I don't mind not having to pay annual fees now for my backups. But when it comes down to it the individual must choose a software that they find easy to use and that will accommodate their backup needs.

Liz Stamos said:

I don't think backups should be done online. Its safe in some way but still unsafe due to the fact that you can get your account hacked, or people working at the backup facility can steal your data. Why would you trust your data with someone else? Personally I run an iphone case store both online and offline so I keep but my data with me and I take it up on an encrypted hard drive.

I just don't trust my data even being left at the office.

Kumar said:

Those are very valid points Liz, that's is why I would stress to ppl to research different forms of backups, ask questions as many as you need to to their customer service support to get the information you need to make a decision and decide on one that best suits the individuals needs. I myself am very confident in the choice to use online backup from the company (KiboTec.com) that I work for as they use only the most updated forms of encryption as well as they do have security features in place. As for being hacked..well I have already in the past (about 5yrs ago) had that not so delightful experience so the way I see it a person can just as easily get hacked either way.

Tracey said:

You did a good comparison of your options in terms of online backup. Cost will always be one of the main points of comparison before acquiring services/products. I've heard a lot of good feedback on Amazon S3. Looks like Photoshelter is a good choice except for the synchronization issue however you get to have good bandwidth which is a big plus already compared to other online backups.

Marcus Reid said:

If you value an open UNIX solution that works like the 'tar' utility, deduplicates, uses strong crypto, and is billed based strictly on actual usage, check out tarsnap.

http://www.tarsnap.com

I use it for some stuff and it's great.

Owen Francis said:

This website has a good upload system and it gives a 5GB free space.

http://absalon.free-online.co.uk/upload/

They also have a unique system for businesses or website owners.

Daruas said:

In the meantime I ran across CrashPlan. I liked the idea and having some space available on a Mac located on another continent, I gave it a try. The client is not as flexible as in Mozy in terms of selecting what gets backed up and what not, but the lack of features is made up in simplicity and efficiency. Over a few weeks all 68 GB of data got uploaded and new and changed files get updated each day. I bought the basic license ($20) and am now considering spending additional $40 for the pro license, which enables continuous backup (vs. once a day) as well as storing multiple file versions. If you have access to an off-site computer with some extra space or can convince somebody to share bandwidth and run a local copy of CrashPlan for you (you can make it a reciprocal deal), I believe it to be a great solution. All the data gets encrypted before sending off so there is no worry of it being compromised. To avoid long initial transfer times, one solution could be to perform initial backup on-site over fast ethernet connection, ship the HD to final location and just do incremental backups since.

Pete said:

Hi James,
i used Amazon S3 and must say i was very happy! all best

Pete

Jenny said:

This has turned into a spammer comment list, to bad it is a very good article and I was considering using amazon s3 but now think that using my own external drives would be a better option...I am not that big yet :) Also to trust a third party with any sensitive data...recipe for disaster I think.

I acclimated Mozy (home version) to aback up about 100GB. I do accept Verizon Fios, so bandwidth is of no affair to me. I acquisition it a actual acceptable service, I accept had no affair with it what so ever. My antecedent annal was about 50GB and it did booty about 2 weeks to aback that up, but now it aloof does basin changes while I am asleep...

jyoti said:

Nice article. Even for online storage one can use the below services such as:
1. SkyDrive(25GB)
2. DropBox
3. Gmail Drive
4. 4Shared

Also there are so many cloud storage options are available which is very economical.

Topi said:

I use Dropbox its great

alex said:

I wonder if cloud computing will help bring prices of data storage down.

Unfortunate about all of the spammers here since it's a pretty informative article. I currently use CrashPlan for online backup and haven't had any issues with them yet

Jeff said:

Then again I forgot to mention that I do have a business in a fairly decent sized city that does not have Fios. So it is not everywhere yet.

Dan said:

I reckon cloud computing will help bring prices of data storage down.

Aletheia said:

I wish somebody would come up with something a little more economical. I have several thousand images that I need to back up. As you spoke to though the cost to do so is somewhat prohibitive.

Belly said:

I’ve been seeking the web in search of the best online storage, backup, and sharing services and applications. Thanks for the detail. I'm still looking for something suitable.

Recommended for You

Tag Cloud

Stay Connected