Does your Backup System Protect your A$$

My backup system was put to the test this last week. I had my main photography desktop, an Apple iMac, it’s time machine backup drive, and two other hard drives containing pictures of family, clients, and my recent trip to India. I also had a laptop, an Apple MacBookPro with client info, documents, and some miscellaneous pictures. So, was this a disaster, or an inconvenience? How well did my backup system work? Was I prepared?

The answer isn’t quite so clear cut. It wasn’t a disaster, but I found out I didn’t have a perfect system. Lets look at what happened, and what could be done differently.

So what backups do I still have? Didn’t they take my hard drives? Well, they did take a drive that was my time machine backup for that machine. I have no backup for that. I have lost several documents. I had PDF’s of client contracts, bank statements, time sheets from my computer coding jobs, eBooks, and other things. Many of these things I didn’t have a backup for. Some of these I can re download, but some like the client contracts, I only have a paper backup. Good job I have that, but for something that important, I really needed an off site backup.

As a side note: I do have a password on my login, but none of the data was in an encrypted volume. I hadn’t anticipated that I would lose my desktop. I figured a laptop, sure, but how would I lose my desktop? This ostrich head in the sand plan isn’t a plan.

So, did I have any off site backup? Yes. But my system needs work. The hard drives that were taken consist of my RAW files, and I had off site backup of those. They are manually mirrored drives. Every week or so (I have to admit that the time could grow) I would bring the drive home from a remote location and use RSYNC to mirror the data, and then bring the drive back. I have family pictures in one off site location, and client RAW images in another. Problems with this: it relies on my getting the drives to and from home manually. I was also only syncing RAW files. That was all that was on those drives. I didn’t have off site backup for my other documents.

The laptop I got lucky on. I have a wireless backup through time machine to a Apple Time Capsule. It is not in the open, and wasn’t taken. I just got a new laptop, and was able to pull in the data from that time machine backup. I don’t keep as much on the laptop however, partly as a way to minimize loss if stolen. But at least it was up an running quickly.

I was also somewhat lucky in that I have a NAS. It was not taken, as it was not out in the open. This should be a spot that I keep much of my documents, but I still haven’t set something up to get data here easily, or automatically. It is really there so that other computers on my home network can share data. If I don’t need to share it, I haven’t been putting it there. Also, this set of drives is raided, but not remotely backed up.

So, do I have all my photography data? Sort of. There are three types of images in my system. The RAW files, the lightroom ‘developed’ files, and photoshop adjusted files. There is also other data in lightroom catalogs, and things like presets, and export and print presets are not saved in regular lightroom catalog backups.

I did have backups of my catalogs. There were also on my external drives, and I had a fairly recent version on my off site backup, so we were good there. But was it good enough? I had to postpone two client meetings where they wanted to order images, but I didn’t know what I had, and couldn’t show them the images on a computer. I didn’t have them with out my RAW images, and a Lightroom catalog, and LR3 to process them. What would have helped? Having jpeg copies of all my ‘finished’ client images. I will get back to that.

The last bit of my system is finished PDF files. Well, some of them are with my Lightroom RAW files. When I head to photoshop from Lightroom, the PSD (or TIFF) gets stored in the same folder. But, I have (had) some photoshop files that are composites, or HDR’s, or other images that didn’t start with Lightroom. Because they were not getting saved with my RAW files, they were only on my desktop (and the time machine backup that disappeared) and not synced to an off site backup drive.

As well, the presets that I had developed my self, and others that I had downloaded are gone. Same as the PDF files, no off site backups. I had recently read on the lightroom blog about syncing those to dropbox, but didn’t get around to doing that.

Ok, so what have I learned, and what can I do different? First of all, nothing at home (or an office/studio) should be considered safe. You need to have an offsite backup of everything. Just having a backup at the same location is not enough. It protects you from drive failure, but it will not protect you from theft, or a worse disaster like a fire. So how do we get all this data protected off site?

Well, lets look at where we can put our data. First would be a service like Mozy, or Carbonite¬†(PC mag review). There are issues with using these for lots and lots of RAW files. First, they take a long long time to sync up. Second, they can get expensive. Mozy home might be free for 2gig, but move to the pro package or get 500gig of photos, and it starts to add up. I don’t think that it is a cost effective solution for a working pro and their RAW files. These systems are set up for home users with a few gig, not someone with lots and lots of gigs.

How about what ever web site you use? For example, I have a service with a web host and run my own blog, I could just put my RAW files there right? Well, look into the TOS. Most say that you can only put files there that are in support of the web site. Backups are not allowed. Hmmm… Well, how about Photoshelter? I have a Photoshelter account. How about putting full size RAW files there. Looked at their prices for add on data storage? Yikes. (Plans) I started putting full size images up there at first. Sounded like a good backup, but I was filling up my space too quickly. So, I have another idea. First, I am not sold on Photoshelter. It doesn’t work perfectly for selling events and weddings anyway. I am thinking of switching to Smugmug. One of the things they offer is unlimited storage of jpegs. So, what I am thinking is that I will export full size images of everything that I put up there, and I would put up images for every client, even if I didn’t intend to send them there. That way I always have a full size finished image backed up on the internet somewhere. I wouldn’t have been stuck like I was this week needing both my raw files and a light room catalog. I would always have images I can review with clients.

Ok, so having client images readily available is a good thing, but I don’t want to lose my raw images. I think that I will still need to follow my manual sync, off site strategy for these images. It is just too much data, and too expensive to save RAW files to the cloud. I just need to be more diligent about the backups.

Ok, what about the other documents? That is what I think I will use something like Mozy home for. I won’t have multi hundreds of gigs for regular documents. This will be an affordable use of these cloud services. This is where the presets and things should be synced. The other photoshop files need to get saved to the same drive I have my RAW files on so that they will get synced to an off site backup. Ok, so how about my NAS? It provides a great way to share documents on my network, store video, music, and provide local backup of files, but to get it backed up would be a lot of data. It will need to be a combination of manual off site hard drives for video and music, and then making sure that everything stored on the network is also on backed up into the cloud service from the computer I was using. The NAS can’t be the only location I have those files.

So, the biggest thing I learned was that I need to have off site backup for everything. That can get expensive. The solution is to figure out how to use hard drives manually, home cloud services, and a better thought out plan to make sure that everything is getting backed up.

What do you think? Are you backing everything up off site? What’s your strategy? What do you think of mine?

New gear I could maybe afford

David du Chemin has a post up about the hyperdrive he bought. Sounds like something you add to your car to get you 400 miles in 4 minutes (I wish) but it is a on site backup device. I have wanted one of these, not that I would really use it that much, but they still come in handy. Would have been great on my summer vacation. Epson makes the P-5000 that will store 80 gigs for OVER $600. Yikes. I am buying a lens before that. Amazon lists the 40 gig model for $279 and the 100 gig for $319. Much more reasonable. Not as big a screen, but getting approachable in price.

New storage needed.

I am getting tired of my linux based setup. It just doesn’t work well enough. The biggest issue I have is that everytime it gets powered down (vacation, storm, etc) and comes back up, it doesn’t quite. I always have to execute some command line magic to get everything recognised. I have tried Gentoo, Ubuntu, Mythbuntu, all with the same issue. I think it is the add in SATA card I have in there, but I don’t want to swap out more computer hardware this time.

Continue reading

Check your backups

In my backup strategy, I mentioned, like Scott, I had a small 2.5 inch drive that I could use to keep a backup of my data when on the road. I don’t use it much. I don’t travel often, and I am not allowed to use it at work.

So, with a trip coming up in July, I thought I would test it out. From a short trip to Rochester MN last weekend, I had some images on my laptop. (I hadn’t deleted the cards before getting the images to the main computer though) I copied the masters to the portable, then imported them into Aperture from the external drive. Oops. Several garbled images. Reformated the drive and I tried again. Same thing, different images. Dead drive, or housing. Guess I am going shopping.

I need more space.

So after I talked about my backup strategy, I went and started my command to sync my pictures. I went to checkup on it, and it was stuck. I checked, and I was out of space. Oh oh. I stopped the sync. Now what? Well, the drives are sporting LVM2, and I had extra space available, so I just added some.

First add some more space to the volume:
# lvextend -L +1G /dev/vgnas/pictures

Then tell ext3 about the extra space.
# resize2fs /dev/vgnas/pictures

Thats it. I did it live, without unmounting. I then went back to my rsync command, and it continued right where it left off. Sometimes I really like linux.

Backup Strategy

Scott Kelby just posted his backup strategy, and I thought I would follow up with mine.

The thing that is really interresting about Scott and my solutions, is that there are a lot of similarities. But mine cost a lot less.

His first step is to save on location to an Epson P-500 (link). That would be cool. I have looked at those wistfully before. I don’t have anything comparable. I don’t travel as much as he does, so I just have to put in my empty card and go from there.
Continue reading