Digital Media | Spotlight: Photography | Inside Lightroom | Blogs
The Economics of Online Backup
In the comments to my previous posts about storing a photo library, quite a few people have mentioned using an online backup solution like Amazon S3 or Mozy. I've looked into this many times and keep running the two sets of numbers that matter. The first of these is how much the storage costs in dollars. The second is how much it costs in bandwidth.
Note: For me, when I run these numbers, I use a 1TB photo library as a working size. It's a both the size of my current library and a nice round number. Your library is, of course, a different size. I know many of you have smaller libraries. Some have larger. The point I'm going to make is the thought process you need to look at when considering online storage. My own solutions fit my needs, other solutions will probably fit yours. With all that said, let's dive in.
One of the most interesting online storage providers, to me, is Amazon S3. It's a high availability storage solution that's being used by lots of people to redundantly store data. It's not easy to use directly, but there are several clients that make it a snap. On the Mac, Panic makes a great solution in their Transmit product, the perennial FTP/SFTP client. The nice part about Amazon S3 is that it provides simple use-based pricing. It costs $0.15 to store a GB for a month, and then there are charges to upload and download that data. Doing the math, storing my TB-sized library on Amazon S3 would cost $150.00/month. This adds up to $1800/year. With the additional charges to upload the data, that comes in at just under $2000.
OK. That's a bit of serious coin. It's less than many other solutions, however. Let's look further.
Mozy provides an unlimited amount of storage for personal, non-commercial use for $4.95/month. I know lots of people raving about this one. The catch for me is that non-commercial bit. My usage would definitely be commercial which would require a MozyPro account with charges of $0.50/month per GB, according to Mozy's pricing information as I write this post. That works out to $500/month or $6000/year. If you subscribe for a year, you get a month free, bringing the cost down to $5500. Even if you are using it for non-commercial purposes, I wonder at what point Mozy would cut you off.
Ouch. You can see where this is going. Out of all the simple online backup providers I've looked at, Amazon is the one that would currently win my business.
Finally, I should mention that Photoshelter—a company that I use for distributing some of my client work—offers a TB of storage for $1000 year. This is a pretty good deal, especially considering what Amazon S3 and Mozy charge. You can even send them a hard drive with your data files on it to upload for you. The downside, for me at least, is that files on Photoshelter are disconnected from your local filesystem with no ability to sync. This means that the metadata is split from your Lightroom library. If you make metadata edits locally, you would have to propagate those up manually to Photoshelter. The result is that, while I like using Photoshelter as a outbound avenue for my photographs, in addition to Zenfolio and Flickr (two other websites I use to push my photos to clients and the world), it's not yet set up to be a good recoverable backup data store that can be synchronized over time. Of course, you may decide that these aren't issues for you and that you're willing to have just your RAW files without your metadata in case of disaster. If this is the case, you should definitely consider this service.
Beyond the monetary issue, there's also the issue of bandwidth. Photoshelter wins here by letting you send in a hard drive for uploading data. While FedEx and UPS have a bit more latency than a TCP/IP connection, you simply can't beat the bandwidth of sticking a hard drive into a box and having it show up somewhere else the next day with all of its data. With Amazon S3 and Mozy, I don't know of an equivalent service, therefore you have to wait for the uploads to happen. How long? Well, for me, my upload bandwidth is a pretty respectable 768Kb/s. Even at that speed, however, uploading a lot of data takes a lot of time.
Here's the math: 768Kb/s is 96KB/s is 0.09375MB/s. Multiply by 3600 and you get 337.5MB/hr. Multiply by another 24 and you get 8100MB/day—equivalent to an 8GB compact flash card per day. From there, you can extrapolate out to about 130 days—or a bit over 4 months—to upload a TB of data. I could always "borrow" a T3 at a friends office, but even with that, it'd take a bit over 2 days to upload the data—assuming, of course, that S3 could intake the data at that speed.
Even once you get your initial data upload taken care of, there's the issue of incremental data uploading. It's not unusual for me to shoot 40GB or more for an event. Even when I'm not shooting conferences, it's still entirely possible for me to shoot 8GB/day or more for days or weeks on end. And, it's not necessarily practical to tie my upload bandwidth to backups for so much of the time.
Now, let's look at the recovery side of the scenario. My current inbound bandwidth is a 6Mb/s DSL connection. I actually see about 5.5MB/s. Doing the math again indicates that it would take 18 days to restore a 1TB image library. Now, granted, in a recovery scenario where you're relying on offsite storage to recover, this might not be a total show stopper, but it is a concern. I'd really rather prefer to have a restore—even from a complete loss of my home—take no more than a few hours or maybe a day.
As I see it, I need a much bigger pipe to the Internet to make these kinds of backups work well long term. Until Verizon FIOS shows up, or I score some really choice client gigs that really add to the bottom line and can put in my own dedicated T3, I'm a bit too bandwidth constrained to make it work out. This, however, is how things stacks up for me. After running the numbers yet again, I'm going to stick with my current program of rotating drives to a safe deposit box which has a nice cafe next door in which to get a cappuccino in. But, I'll keep running the numbers every so often and at some point, maybe the equation will change answers.
If you have less data, or have access to a bigger pipe, the bandwidth economics will be different for you. If you shoot quite a bit less than I do, which wouldn't be surprising as I do generate quite a bit of data, the economics may very well line up. Or, if you're running a more capitalized business than I am and have larger pipes to the Internet, the numbers might work as well. You should go through your own calculations to see if an online storage solution works for you both in terms of money and in terms of bandwidth. And, if you're willing, please share your online backup equations and experiences in the comments. I'm sure that many other people could benefit from your findings.
Photographs in this blog entry are Copyright James Duncan Davidson, All Rights Reserved.
Comments (42)
Recently I did similar calculations for myself and already stopped after the bandwidth issue started to materialize. I didn't even look at the costs; at least with todays common bandwidth online-backups aren't really feasible for large amounts of data.
But if you really want to do online-backups, a combination of the Photoshelter and one of the others might work: Snail-mail your massive amounts of data to them and store it in a flat namespace. Then you keep the metadata somewhere else where its easy to update, like S3. Folders are just metadata, too, just pointers into your flat namespace. To glue all that together, you need to create some kind of abstraction layer of course, on Linux I'd probably write a FUSE app. I have no idea how to combine all that with Lightroom, to keep on the overall topic, but that's how you could combine the best of both worlds.
But after all, the deposit box is probably still easier and cheaper :)
I could have used this analysis about a week ago, when I experimented with uploading my (much smaller) iPhoto library to S3, via JungleDisk. I only have a few gigabytes' worth of photos to backup, and it still took several overnight sessions to complete that first sync all of the data.
I've been thinking about stowing some external hard drives in my office at work and automatically syncing up remotely via the VPN. I'll have to measure my upward bandwidth though. In the event of a disaster at home, I would just go pick them up, much like you'll do with your safety deposit box.
If you don't have an external office, you could exchange hard drives with a buddy. You keep a couple hard drives for me, and I do the same for you. We both set up ssh accounts so we can sync remotely (assuming you can get enough bandwidth).
Also, the idea behind stowing my hard drives at work is I can do the initial (BIG) sync locally and then just do incremental updates remotely.
For all online backup and storage related info, I recommend this website:
http://www.BackupReview.info
For my offsite backup needs, I usea D-Link DNS-323. This small 2-drive NAS runs Linux, can be easily enhanced to support NFS and rsync/ssh. It is inexpensive, compact, power-efficient and reasonably powerful. I keep mine in my office, loaded with 2 1TB drives in RAID1, seeded it by rsyncing on my Gigabit home LAN, then refresh it by having it rsync from my home machine every night while I sleep (my office firewall won't let me push from home to the DNS-323, so I pull from it instead). As a bonus, since my backups include my iTunes library and it has a built-in mt-daapd, I can listen to my entire music collection at work from iTunes over the network.
This is a great post. Actually, as I saw you were starting to talk a lot about that backup issue, I was about to ask you why you didn't talk a bit more about online backup systems. I've done the math myself, but I was still wondering if I wasn't missing something, with all these people talking about S3 as the definitive solution.
The discussion about your use of PhotoShelter as both a delivery website and a backup solution makes me think that we probably need another service : a web site that would take your disks and provide publication of photos on them so that you can monetize them. But do we really need to send them raw files, if it is to publish them on the internet? Do we need so much space?
Also, if you ask a service such as Photoshelter to be a backup of your files, you may not need to restore all your files at once. Don't they provide a 'snail mail download' too?
Why do online backup at all?
Why not rsync to a big hard drive, and store it at a friend's house?
You could add a second drive and rotation for redundancy.
Make it automatic with a cron job. You just have to remember to take the disk with you when you leave your house.
I'm using Carbonite http://carbonite.com and very happy with it
Smugmug offers unlimited storage space to account holders, with a cap at 12MB per image for their standard and power user accounts. This still does not address the limitations you found with photoshelter (metadata split with lightroom, etc) at a signifigantly lower price (149/yr for pro accounts).
A comprimise would be a combination of this and offsite storage of a external HDD or archival DVDs, IMO.
(disclaimer, I am a Smugmug customer.)
Malte: Indeed, the problem with the Photoshelter approach is the cost/complexity of recovery. The safe deposit box does win, at least for me.
Bob: Indeed, putting your spare copies at your office is just as good as my using the safe deposit box. The idea is to get your 2nd copy into a place where it can survive anything that's likely to affect your primary location, short of nuclear war of course. And, of course, I know you have mondo bandwidth where you work.. Hrm... I'll be right by! :)
Fazal: That a great way to solve the initial upload issue for sre.
Cyril: There are lots of places to monetize your photos, but RAW files aren't part of that game. I wouldn't ever send a client a unprocessed RAW file. A DNG with initial processing instructions, sure, but never a RAW from the camera. Of course, I may change positions on that over time. As far as a snail mail download, that'd rock.
anon: Storing a clone at a friends house also accomplishes the same thing as my storing clones in the safe deposit box or other people's solution of storing at work.
Norman: Indeed, Smug does offer unlimited storage, but the same bandwidth and restore problems exist. Smug seems best suited as an endpoint for your photography to print, display, and show. But that's just my take.
For 1Tb data online storage may not be an option. For smaller libraries and if your rimary reason is backup you have a lot to gain. I did the math and for my library of 5+ Gb I pay about $2 per month including the requests and transfer of new photos shot during a typical month (see my name link for some details).
This is of course a lot cheaper (and safer) than getting an external RAID.
has anyone considered trying out crashplan? They offer similar rates, but their client app is pretty nice, and it can store single files bigger than 5GB (limitation of Amazon s3) .....
?
I'm using CrashPlan now, and I'm very happy with it. It's very bandwidth efficient, and the ability to store duplicate backups on friend's computers (and restore from several sources simultaneously) is pure genius. The cross-platform aspect is nice as well. I backup my Mac to both Windows and Linux systems.
The economics of CrashPlan are different than Mozy, though. Mozy is a billed like a service, so the software is free, and the storage costs you. CrashPlan costs a fixed amount for the program (incl free updates for a year) and storage on the CrashPlan servers is a secondary service you don't have to use. If you do use it, it costs 10 cents per GB per month, with $5/50 GB minimum.
You can also store your backup data on your friend's computers and avoid monthly charges entirely. I'd suggest trying the 30 day trial, and then if you like it, going straight for the $60 Pro version which allows you to retain previous versions of your files. It's not as sexy looking as Time Machine, but CrashPlan Pro can be snapshotting your work continuously anywhere you have an Internet connection. (Great for a laptop when you don't want an external drive always attached, and don't have Time Capsule.)
But, before CrashPlan, I'd first go buy an external drive and make a bootable image of your Mac once a week with SuperDuper, or Carbon Copy Cloner. (SuperDuper is my favorite.) Only then should you supplement with online backup, since no matter what, it will be SLOW. You should only rely on it to recover things since your last disk clone, or to cover you should you lose both your computer and external disk at the same time. (Fire, flood, theft, etc)
An external drive + SuperDuper + CrashPlan was the best $150 I ever spent.
I recently sent a Mac mini into a data center to act as my backup. I went with Macminicolo.net because you can throw a 1TB drive on there connected to your mini. With that, I was able to send my large library in the mail, and now I can just do updates via transmit.
I second the SuperDuper + CrashPlan suggestion. My backup setup is the following: SuperDuper makes a bootable copy of my system disk each night. That way if my hard disk fails, I can reboot from the external disk and work on with maximum a day of lost data. CrashPlan and Mozy are making online backups off-site. A more detailed story follows.
After a friend of mine got burglarized I realized that on-site backup does not really help when the bad guy takes away both your computer and backup disk. That was when I started looking into online backup solutions. My first stop was Mozy. It worked great for the first 2GB and I bought a subscription plan. After the first full backup (60GB) I started having problems with the client loosing connection and also taking lots of RAM. The issues were so annoying that after weeks of failed backups and maxed out RAM I just uninstalled it and only tried it again now and then when a new beta version was released.
In the meantime I ran across CrashPlan. I liked the idea and having some space available on a Mac located on another continent, I gave it a try. The client is not as flexible as in Mozy in terms of selecting what gets backed up and what not, but the lack of features is made up in simplicity and efficiency. Over a few weeks all 68 GB of data got uploaded and new and changed files get updated each day. I bought the basic license ($20) and am now considering spending additional $40 for the pro license, which enables continuous backup (vs. once a day) as well as storing multiple file versions. If you have access to an off-site computer with some extra space or can convince somebody to share bandwidth and run a local copy of CrashPlan for you (you can make it a reciprocal deal), I believe it to be a great solution. All the data gets encrypted before sending off so there is no worry of it being compromised. To avoid long initial transfer times, one solution could be to perform initial backup on-site over fast ethernet connection, ship the HD to final location and just do incremental backups since.
Just when CrashPlan started working fine for me, new version of Mozy client came out and I gave it another try. While it had serious issues with maintaining connection before, it now works fine (based on some detective work it might have been an issue with my ISP). In the last few days I have updated about 15 GB of files and have 10 GB more to go (estimate is 2 days worth of upload left). The client seems to be behaving better now as well. It can still gobble up to 500MB of RAM and over 1GB of virtual memory, but it seems to more gracefully give way when I'm running lots of applications. I'm hoping the RAM usage will go down once less data needs to be updated.
Future plans? SuperDuper is a must. I need to have a functional mirror to switch to at any time. CrashPlan has worked great for me (I have used it to restore mistakenly changed files a number of times). I still have 6m worth of subscription to Mozy left and will see how that goes as well.
I've previously used a colorful metaphor to describe your bandwidth problem, it's like an old saying called "The Marching Chinese." They say that if you got all the ~1 billion people in China to walk 4-abreast past one fixed spot, they would never stop coming. They'd be lined up so far back, that people would give birth to children while waiting in line, and the children would grow to maturity before they ever crossed the line. And they in turn would give birth to more children who would grow up in line. So the entirety of the population would never cross the line, they'd keep coming forever.
And that's the way it is with the online storage bandwidth problem. I can easily create more data in one day than I can back up in one day with my little DSL connection. I can easily do that EVERY day.
Anyway, apologies for the slightly racist metaphor, it was taught to me by my crazy racist high-school biology teacher (who also had a PhD so I figured he must know something).
I'm very happy with Jungle Disk, which is regularly updated and supported by its developer in combination with Amazon.com's S3. The cost for us photo-less mortals is much less per month, but I really appreciate the economics laid out in this post. As well, I've been trying Jungle Disk Plus, which is a browser-based means to access your backups. It's very impressive.
I've been looking at this problem for a long time too. The only solution that has been feasible for me in terms of cost, time, and bandwidth is to do nightly backups to an external drive (kept locally), and monthly backups to a second external kept off-site.
I use Time Machine for daily, and SuperDuper for monthly.
Online backups just don't work for extremely large amounts of data (yet).
I follow the same approach as Paul for my business -- daily backups with Time Machine and monthly backups using SuperDuper.
A couple of years ago my office was robbed. Twice! The only thing that saved me were my backups. I lost almost a week on the G5 that was stolen but I was up and running within a couple of days using backups.
I also archive projects to DVD which takes a lot of time but to me the redundancy is worth it.
I also have a lot of extra storage on my Dreamhost account and tend to use that occasionally when I'm feeling extra paranoid (mostly for email).
Even with much less data to back up (around 100Gb) I came to pretty much the same conclusion regarding time to back up and restore with online services. Really like the idea but it's just not practical for me at the moment.
See the long story on my website: http://www.zx81.org.uk/computing/opinion/backup.html
I'm still looking for the ideal online backup solution.
Right now I'm using Mozy (Home) and it's ok. My biggest complain is that even with the *paid* (non-commercial) service they throttle your upload bandwidth to 128 KB/s—and don't tell you about that loudly and clearly before you painfully find out yourself.
JungleDisk looks promising. But until they finish the file system integration on Mac OS X, it's *not* usable for backups IMHO. That's because of the local WebDAV solution which means, files you copy onto the JungleDisk loose their original permission setting. Quite unacceptable for a real backup!
(Gaah, why do these stupid captchas never work??!!)
Unfortunately online backup will never be viable. It would work for people with a little amount of data, but anyone wanting to backup anything more than basic Word documents will always hit a brick wall. File sizes are increasing, as are hard disk sizes. This means we can store a lot more.
Internet speeds are increasing much more slowly so the gap between the speed we can transfer and the amount to transfer will continue to grow. On top of that, storage amount is pretty much limitless, whereas internet speeds are at some point in the future going to hit a brick wall, meaning the only to make accessing a site faster would be to have servers closer to you geographically. Of course by then uploading a photo would be instant, but I'd assume by then that 1TB of photos would have increased massively.
Yesterday I just cancelled my Mozy account. Since I signup in November 07, Mozy client (Mac) never completed backuping my 500GB data.
In the beginning the client was not stable enough, it crashed everytime the configuration window is openened. They fixed that problem in later revision of the software. But then it just backs up things when it wants to; not continuously. The amount stored in the backup stayed around 49GB, never higher. Then I get to see this pop up every day saying no succesful backup has been performed in the last 7 days. Extra annoyance was, my MBP fan was spinning everytime Mozy client did its thing. 400 something bits encryption, cool... but can you do with less encryption and keep my processor cool please?
The good thing is that, when I asked they refunded my money immediately.
Now I am looking for one of those cheap shared hosting that gives hundreds of GBs. With rsync + encrypted disk image (Leopard's sparsebundle) it should work better
It's not unusual for me to shoot 40GB or more for an event.
40GB of backup worthy photos? Every event? Surely you could cull those back a bit before backing up? Even if you're shooting RAW that's an insane amount of photos to backup. Backup the best 50, convert the next best 50 to JPEG and perhaps back them up too. Delete the rest :) Most of your bandwidth problem's are solved.
I have had a somewhat different experience with Mozy and S3/Jungledisk than previous commenters. I found Jungledisk for the Mac unbelievably slow and inefficient at uploading and even just getting it to list my files already backed up required an interminable wait. Meanwhile, Mozy works fine, uploads at high volume and has a number of useful preference settings for scheduling backups and limiting bandwidth usage if need be. I have two computers backing up a total of over 100 GB. And automatic daily updates, I don't have to lift a finger. Mozy also lets you use your own encryption key if you so desire, meaning no one, not even Mozy, can spy on your data even en route.
I do find it hard to believe that online backup services could make any sense at all for someone with 1 TB of data, however, thanks to the lame-o state of broadband connectivity in the U.S. In Korea maybe -- but not here.
While I agree with you for large scale backups, what I find Mozy (and its ilk) useful for is backing up smaller important data, such as my Address Book, QuickBooks & Quicken data, keychains, iCal calendars, and other smallish data. I've been using Mozy Free (since I'm really not using it for commercial purposes, just nightly safety for my laptop) and haven't even exceeded 250 MB of space.
For my fiancee, I've got her desktop G5 and her iBook setup to do the same thing, as well as to backup her vacation photos, as we're currently traveling in Norway and I'd hate to see her lose anything she hasn't already U/Led to Flickr.
Anyone have a recommendation for a Windows equivalent of SuperDuper?
I used Mozy (home version) to back up about 100GB. I do have Verizon Fios, so bandwidth is of no issue to me. I find it a very good service, I have had no issue with it what so ever. My initial archive was about 50GB and it did take almost 2 weeks to back that up, but now it just does delta changes while I am asleep...
You could check into ElephantDrive, they offer a 1TB Pro option for $350/year. Works on my Mac, but I am just using the free versionn now while I taste test it.
I have essentially three sets of data. Documents and such (TaxCut files, eBooks I've purchased, etc.). I've used StrongSpace for that so far. The other set of data is multimedia, RAW files from my camera and FLAC files from ripping CDs I own. The final set of data, if you will, is system backups... SuperDuper/TimeMachine backups. I use an external drive for these (I do both SD and TM). Its the massive amount of multimedia files I struggle with... 4.7GBs per DVD, burning two copies of each is a logistical nightmare.
Thanks to prior comments I will check into CrashPlan for some of this data.
@Martin Pilkington: "Unfortunately online backup will never be viable. [...]Internet speeds are increasing much more slowly so the gap between the speed we can transfer and the amount to transfer will continue to grow."
This isn't true. Network bandwidth (at all scales, WAN/LAN/local bus/etc.) also has a doubling time (i.e. follows the colloquial version of Moore's Law). It's just that this trending has a very pronounced step-function style of behavior, especially for the broader scales. Intuitively, consider that there's infrastructure that must be built out for each wave of change. National/global infrastructure doesn't change overnight. Using my own personal download bandwidth as an example, I started with 300/1200/2400 baud modem access, then a 28/56k modem, then a 1Mbps up to a 7Mbps DSL line. Each of those main phases lasted for quite some time before the sudden orders-of-magnitude transition to the next phase.
The implications of this are as follows: right now, online backups are useful for some people. By the time the next network bandwidth transition happens, the economics will dramatically change the utility of these services. I doubt that they'll entirely replace local backups for data-intensive users, but just as cheap portable hard drives have recently transformed our backup options, cheap and massive network bandwidth will also extend our backup options.
BQBackup.com anyone?
Charles: You can state the metaphor the same way in global terms as well. With 6.6 billion people on this planet, we're growing fast as a population.
Michael: Of course there aren't 40GB of worthy photos for every event. But, the rub is that it takes time and effort to cull images. Many people have made the assertion that it's cheaper to keep everything than to take the time to cull. There are other problems with deleting too much as well. If I were just working on personal work, then I could delete more than I can for clients. In my client work, I've found that sometimes I need to go back into my non-selects and dig for other images for particular purposes.
Obviously, what I describe is my own situation, but I've talked to many professionals who don't throw away anything. Nothing. Zippo. Nada. And they all state reasons that fall into the same ballpark.
I typically shoot 14 gigs at a wedding and generate .psd files up to and above 100GB in post processing. When I get home this is backed up to dvd and two separate jbod arrays that are superdupered nightly. One of these is housed in a case I leave near the door in case of fire I can grab it on my way out. Once a week I visit a friend and back up the case to a third array.When i have finished processing an event in Lightroom it's exported as a catalog and popped on another dvd.
I have been looking into online storage, colocation etc for ages and have never found a feasible solution. I think the very best solution would be to run a firewire cable into your neighbors attic and install a bunch of drives up there. In the meantime I'm looking into installing some kind of safe in the garden shed.
Yes I'm paranoid :)
I came across this recently, 1125 gb storage and 7500 gb transfer for $9.95 a month from webhostingbuzz.com. I am not sure how they can stay in business with those prices.
I worked this out some time ago but couldnt have explained it so well. Nice post. A must for high volume, low value data users in personal or small business scenarios.
I am a freelance photographer. I used multiple free online backup accounts from two pcs and a laptop. I didn't mind the hassle of splitting up accounts to keep them in the free category. I did not see in advance the broadband issue and now realise that to backup online properly I would not alone need a commercial online backup account but signficant broadband in terms of speed and cap. My wife runs a small bookeeping business at home. She backs up all her critical data (mostly word, excel and accounts) with www.backupanytime.com for less than US 40 per month (They are a euro outfit in Great Britain) and she is very happy. I emailed them and got a quote to do the same. They got me to run a broadband test on their site and detail the quantities. They came back and quoted Euro 289 per month and explained I would have to treble my broadband for it to work! Thanks but no thanks. I now undrstand that the online backup model is great for those it suits and a non runner for those it does not. Any inbetween is a coplicated mess.
Paul
HI!
Thank you for your comments- from reading the above it confirmed that i made the right choice. We use onlinebackupvault.com - we played around initially with a couple of other guys, but OnlineBackupVault.com is a keeper. THey offer free software and free agents for Exchange & SQL, their backup speeds are fast and they charge us by the space used once compressed.
And probably most important their support is out of this world
Paul, I use www.backupanytime.com also. Did they contact you recently with a free offer to add another mirror server to your account? They called me and I was too busy but I called back later and not alone did I get an additional mirror server but a software upgrade and a free data audit!
I was informed that the new data centre was in Ireland (I am Manchester) and that if I wanted to backup in UK only I should leave my backupanytime account status as it was. I had no issue with adding an outside data centre and went with it.
Cool or what!
The Chinese March analogy is fun, but flawed. Here's the math:
Chinese people: 1,000,000,000
standing in a line, this number abreast: 4
means this number of people in a single file: 250,000,000
assume this many feet between people in that file: 2
total distance, in feet, from 1st to last person in file: 500,000,000
same distance in miles: 94,697
walking speed, in mph: 3
hours for last person to cross: 31,566
hours that last person can walk, per day: 16
number of days required: 1,973
number of years: 5.41
Now, if you use the whole world's population (6B) you'd think that indeed the world would keep coming forever. But if everyone is walking, they wouldn't have, in the words of Bret and Jermane, much "business time."
I've suffered through the bandwidth math, too. From what you describe, your challenge is not the weekly additions to your library, but the initial backup. This is my challenge, too.
I've debated putting a duplicate of my 2TB RAID-6 server at my parents house and using standard DSL between to catch weekly additions, simply using rsync.
If you did this, you'd have your initial backup solved. You then have to keep your additions to 8GB*7=56 GB / week (note: you assumed you get 100% of your upload speed, which is not a practical assumption, but you also didn't factor in any file compression, which can squeeze 20% out of .TIF or .CR2 files if you zip them up, so I'd call it a push).
Thus, a $300 P4 2.4GHz box with a $300 Ciprico raid controller, and 4 1-TB (4@$300) drives comes to $1800. Monthly storage costs = zero.
I could also just gamble and build a lower-end machine on my parents end with two 1-TB drives as JBOD, and that brings it to $900. But when (not if) a drive bonks on their end, I have to make a long drive to replace it and I don't see them *that* often.
Great post to the OP -- thanks for putting this out there!
Most people really missed the point, speaking of online storage / backup, it is not the amount of storage that matters, it is really about the application that matters. In reality, out of 100 small businesses / home users, less than 1% of them would need more than 50GB of storage, unless you backup all the operating system junk files.
I have used DriveHQ Online Backup and am very happy with the speed and reliability. Yes, I have a 120GB hard drive, but my data files are less than 10GB. The first backup does take some time, but the subsequent backup is extremely fast. DriveHQ usually can achieve a 2 -3 times performance boost when uploading files. So even if you are on a 384Kbps ADSL connection, your upload speed could be 3 times faster. In fact, contrary to the author, I think Online Backup is the way to go, at least, good services like DriveHQ Online Backup.
"less than 1% of them would need more than 50 gigs"
I understand your heart is in the right place but this is akin to an old quote from Bill Gates suggesting that noone will need more than 1 meg local storage. The problem really is that most people can not be trusted to select the correct data and watch and amend this on an ongoing basis so complete or near complete backup are more common than you may think. As per the post on backupreview.com above I tested 6 of the sites favourites offerings. I found http://carbonite.com and http://www.backupanytime.com both had a very useful auto select feature which had its own artificial intelligence which familiarises itself with user data types and highlights anomolies. This may be the answer to keeping the selection and transfer amounts below 50 gigs, at least for a while...
Frank
I agree with Frank. We backup specific critical data only and it works fine. If we tried to backup everything it just wouldnt be feasable or necessary. Online backup is an excellent option if the user is capable of managing it or has local IT support to do it.