A back up strategy to avoid saying “I lost my photos!”

I’ve known family members and even experienced photographers who have lost the photos on the memory card even before offloading them to a computer and those who have lost their photos from failed computer storage. Take a moment to think about the time invested and the many beautiful memories you could potentially lose if your current photo catalogue were just gone. I feel my anxiety rising just thinking about it. Ugh!

So, if you're curious about it, here's what my backup and recovery plan looks like!

Stuff happens, and if you do not have a coherent strategy for protecting your images from data loss, at some point, you will be the one to utter the phrase, "I lost my photos." Here's the strategy I used to mitigate the risks.

I use the internal macOS hard drive for installing applications only. I wanted a way to restore my applications and settings if the internal hard drive failed. I wanted assurances that the local and remote backups would be in sync. I wanted to import my images from the SD card and have assurance that the copy on the local external hard drive is backed up to another local external disk as soon as possible. I considered fire a threat vector. In case of fire, I wanted assurances that I could create a tertiary remote back as quickly as possible. I wanted to manage my local backups and remote backups with one tool.

Backup While Shooting

My risk mitigation strategy starts before I click the shutter release for the first image. I use two similarly sized SD cards in my Fuji X-T2, configuring my camera to store full-sized RAF (Fuji's RAW image format) on both cards. If one card entirely craps out on me or if I accidentally format a card in the camera, I’ve got all the RAW images safely stored on the other card. I never remove the second memory card from my camera.

If your camera has two memory slots, even if each slot is a different storage type, there is no reason not to use this two-card arrangement to reduce the risk of data loss. Memory cards used to be quite expensive. My first digital camera was a Sony DSC-S70, a 3.3-megapixel camera with one storage card slot that used 128 MB Memory Sticks. That 128 MB Memory Stick barely held 36 JPEG-quality images. Today, for the same prices, I fill the card slots on my Fuji X-T2 with 32GB Lexar Professional SDHC UHS-II super fast cards. I can store over 1200 RAW quality 24-megapixel images. If you get a low-speed memory card for the backup card slot, you may experience the slow performance of the camera during burst shooting. I recommend buying two of the same brand and type of card and buying the fastest storage card your camera can support.

Local and Online Backup

Time Machine

Hard drives fail, computers get stolen, and there’s any number of ways a whole bunch of data can get lost.

macOS, Time Machine
macOS Time Machine

I primarily use the internal macOS hard drive for installing applications. One of my main concerns was having a reliable way to restore my applications and settings in case the internal hard drive decided to call it quits. I’m a Mac, so my backup strategy includes the built-in Time Machine as well as a third-party backup solution, Code42’s CrashPlan Pro for Small Business.

Using Time Machine is simple. Plug in an external drive to the Mac and follow the on-screen wizard. Then forget about it. Time Machine automatically runs in the background doing its thing. Additionally, I used CrashPlan to backup the internal SSD of the Mac to an external USB hard drive.

If the internal SSD fails, I can boot and run from the Time Machine drive until I get a replacement and ca recover from the Time Machine drive. If the Time Machine drive fails, I can recover from the CrashPlan backup. These multiple backups reduce the risk of a single point of failure. Time Machine is only used for data loss prevention on my Mac itself.

CrashPlan

I wanted to import my images from the SD card and be sure that the copy on my local external hard drive got backed up to another local external disk. If I have the computers and the backup in the same place, what happens if they both get destroyed by a fire or another disaster? I wanted a solid plan in case that threat ever became a reality. I wanted to create a tertiary remote backup. Additionally, I wanted to ensure that my local and remote backups were always in sync.

CrashPlan Pro Console
CrashPlan Pro Console

Last year, I wrote an article detailing my backup strategy using CrashPlan PRO for Small Business. Nothing has changed for 2019, except that I now use a Fuji X-T2, and sometimes I import my images as Fuji RAF. You should read that article for all the details.

Here are all the benefits of CrashPlan.

CrashPlan is a cloud-based backup and data protection service that offers several key features for individuals and businesses. Some of its notable features include:

  • Automated Backup: CrashPlan provides automated and continuous backup for my files and data. It monitors my selected files and folders for changes and ensures they are backed up to the cloud in real-time.
  • End-to-end Encryption: CrashPlan prioritises security by encrypting my data during transit and storing it encrypted in their cloud servers. This ensures that my files are secure from unauthorised access.

  • Versioning: The service allows me to store multiple versions of my files. This feature is valuable if I need to restore a previous version of a document or file.

  • File Restoration: I can easily restore files from my backup, whether a single file or an entire folder. This flexibility is helpful in case of accidental deletions or data loss.

  • Customizable Backup Sets: I can define which files and folders I want to include in the backup sets, giving me control over what gets backed up.

  • Alerts and Notifications: The service provides alerts and notifications to keep me informed about the status of my backups, ensuring I am aware of any issues that may arise.

  • Large Storage Capacity: CrashPlan offers various subscription plans with different storage capacities, allowing me to choose the one that suits my needs.

  • Support for External Drives: I can back up external hard drives, USB drives, and network-attached storage (NAS) devices, providing comprehensive data protection.

  • Hybrid and Local Backup: Besides cloud backup, CrashPlan offers the option for hybrid and local backup, which can be useful.

It's important to note that CrashPlan may have different features and plans depending on whether you are using it for personal or business purposes. Before you subscribe, you'll need to review their latest offerings to ensure they align with your specific backup and data protection needs.

CrashPlan is a backup software that I installed on my Mac. It is highly configurable, lightweight and runs automatically in the background. I configured CrashPlan to back up when my iMac is idle and to use more bandwidth at night when my home network is not being used. The software is set to make local and encrypted cloud backups.

I pay about $10 per month per computer for a subscription to their online backup. My cloud backup is configured to use 256-bit AES data encryption for data at rest has no file size restrictions or additional charge for space. Since I have the encryption keys locally, neither CrashPlan or anyone else can read my files.

I can control what files are protected, when, and how often. CrashPlan backs up my most recent files first, then ensures I have a full backup at one destination as soon as possible before backing up to the other target.

When I run into a problem, CrashPlan’s support staff are available by phone, chat, and email. For recovery, I can restore via CrashPlans’ desktop app or web browser. I have a fast internet connection via Xfinity, so recovery times are not too time-consuming.

In the recent past, CrashPlan offered a disk recovery option whereby they would ship an encrypted disk with a copy of my backup. However, the company recently re-organised to focus on small to medium-size businesses and shuttered the consumer arm of the company.

All of my images are stored in an Adobe Lightroom Catalogue on a 5 TB external hard drive. CrashPlan’s backup software continuously creates copies of my pictures to another local 5TB external hard drive and encrypted copies to CrashPlan’s cloud service. A real-time file watcher checks for new files and file updates and backs up changes every 15 minutes by default.

That means I have three copies (two local and one in the cloud) of my images. The likely data loss is minimal. If one of the local drives fails, I can work off the other drive. I can replace the defective drive with a quick trip to Best Buy or an Amazon.com Prime delivery.

If both local drives fail (unlikely) or my house floods, electrical surge, or burns down in a fire, I can rest easy knowing that I can restore from cloud storage. No technology is zero risks, but it is highly unlikely that I would lose both drives and cloud backup. These multiple backups reduce the risk of a single point of failure.

Using a similar three-copy strategy, I use CrashPlan to back up all of my other files - my iTunes library (a separate hard drive from my Adobe Lightroom Catalog), home movies, financial records, etc.

CrashPlan Notification Email
CrashPlan PRO Notification Email

One of the features of CrashPlan that I often need to remember is email notifications. My wife doesn’t use her MacBook Air daily, preferring to use her iPad for most tasks. Sometimes, she can go an entire week without using her MacBook; when she does, she may only use it for a few hours.

That means that often, CrashPlan still needs to back up her MacBook. CrashPlan’s cloud service will send me an email notification.

Bhavna also needs to remember to attach an external backup drive. However, CrashPlan will pick up where it left off the next time I plug in the drive. Compared to other cloud backup options, CrashPlan Pro for Small Business is an excellent value.

I’m happy with CrashPlan for Small Business because it always works.

Image Formats and Backup Strategy

This started as a comment to Alex Kunz's post on DNG and Backups. After commenting on Alex's post, I decided to write a post detailing my backup strategy.

Comment on Alex’s Post

It is interesting to see the perspective of a professional photographer when it comes to backup and file formats. As a cyber-security professional my perspective is different.

For me, standard file formats are more important because I can expect broad support for that format and longevity. With DNG I have all the originating RAW data plus all my edits in one file. I think proprietary file formats can lead to vendor lock-in and being left stranded if the vendor changes format or abandons the format. And as long as the software supports it, I can edit my DNG (and any changes) in any photo editor.

I am more concerned with my ability to recover from data loss than how much storage my files consume or how long backup takes. And I test my backups. I have friends who are professional photographers and I know for a fact they never test their back-ups. I have a few times, deleted everything from my hard drives and restored them from back to test my backup strategy.

Current Backup Strategy

All of the images are stored in an Adobe Lightroom Catalogue on a 5 TB external hard drive. The Lightroom catalogue consumes about 846.41 GB of disk space. CrashPlan’s backup software continuously backup backups my images (encrypted) to CrashPlan’s cloud service and to a separate 5TB external hard drive. A real-time file watcher checks for new files and file updates, and backs up changes every 15 minutes by default.

If the drive holding my catalogue fails I can buy a new drive and restore from the local backup. If both hard drives fail, I can purchase new drives, and I can recover from the CrashPlan cloud.

Settings for Adobe Lightroom are also backed-up to the local CrashPlan disk and cloud, as well as via Apple’s Time Machine software.

Recovery

If I suffer catastrophic failure — a fire in my house consumes the computer and all attached hard drives — I am confident that I could recover my entire system in a few days.

I know some people baulk at spending over a hundred dollars on big drives but that cost is nothing compared to the value of the images. Only an idiot puts cheap tyres on a car carrying their kids.

Wait a minute ... backup!

 

Recent blog posts by web developer/designer Chris Bowler and photographer François Arnold started me thinking about my own computer backup strategy.  I would like to think I've developed a reliable and workable system over the years but on inspection I think have a few gaps.

Chris's blog posting is about how he uses Time Machine as part of his backup strategy.  He also compares his strategy to that of others.

My own process is similar, making use of a 1 TB external drive for weekly clones of my hard drive and video storage, a 750 GB external drive for Time Machine backups, a 120 GB external drive for monthly clones of my hard drive, and Dropbox for everything that is important (even my music and photos).1

François on the other hand focuses on creating a detailed backup strategy for Adobe Photoshop Lightroom.

 

As photographers, we mainly want to backup our master files with their editing (so we don’t have to do it all over again) and our catalogs.2

François's strategy starts from the moment he imports his photos into Lightroom which includes choosing the right format to keep his images and edits. François is a Mac user so his strategy also includes Time Machine.

My current strategy is not quite as organised or well thought out as Chris's or François'.

My current backup system involves three external hard drives and a cloud backup service. My Lightroom catalog is kept on one of the external drives; a 1TB G-Drive Q attached to my iMac. My wife prefers iPhoto so I import photos from her iPhone and any images of family. The iPhoto library is kept on the same 1TB drive as the Lightroom catalog. iMovie files are stored here as well. The second external hard drive, a 500GB G-Drive Q attached to the iMac, stores my iTunes library. Some of my important files are stored in the Documents folder on the iMac's 250GB internal drive and some are stored on the 500GB G-Drive Q. I use Dropbox to store my 1Password keychain as well as documents that I've import onto my iPad via GoodReader. These documents might have come to my iPad via Mobile Safari or the Mail app.

The backup process is both automated and manual. I use the Backblaze3 cloud backup service to backup the entire Lightroom catalog, the files stored in the internal Documents folder and the Documents folder on the external 500GB G-Drive Q. The Backblaze agent runs continuously and copies any changed files up to the cloud service. Occasionally — there is no set schedule — I connect a 250GB Seagate FreeAgent drive and run Time Machine to backup. This backups everything except the default exclusions4. I also have a 500GB Time Capsule that I never use. The Time Machine backups were taking too long over Wi-Fi.

After reading Chris' and François' post I realise that I have a gap in my strategy. The first issue is that I only have two copies of my photos and documents. The copy on the external drives and the backups in the cloud (for the photos and documents) and the Seagate drive (for documents). I think a good backup strategy requires at least three copies of important files — the original and two backups — with one of the copes stored off site. The second issue is that I don't have a backup for my iTunes library. All my purchases — movie, music, and apps — are at risk5.

I think I need a new system. I have not bought any equipment yet but I think I have a strategy that will work.

  • Move all important documents — tax paper work, receipts, etc — to the Documents folder on the Mac's internal drive. I'll use TimeMachine Editor to automate the schedule. Continue to backup the iMac via Backblaze but add a twice weekly backup to the FreeAgent via Time Machine. That's one off site backup and one onsite backup.
  • Continue to keep the iTunes library on the 500GB G-Drive Q. There is still the issue of a second and third backup. I could backup to Backblaze but media files tend to be quite large and I'm worried about exceeding my Comcast bandwidth cap.
  • The photo libraries are already being backed-up to Backblaze but as with the iTunes library the second backup is an issue. My Time Capsule does not have enough storage to keep my iTunes library and my very large photo libraries. I could add another FireWire drive to the iMac but I've already got two drives hanging of the back of the thing and the desk could get crowded. The Seagate FreeAgent Drive is too small to hold iTunes and the photo catalog.

I'm not sure what to do about the other backups. I guess I could perform manual backups on a weekly basis to another external drive but I really want an automated solution. I need to do more thinking.