How I organize my photo library

Since I got my first professional digital camera in 1999 (if you don’t count an exotic Apple QuickTake 150 in 1995), I needed a future compatible system to store my images in a file system. I quickly settled for a very simple system and now 10 years and almost 400,000 photos later I realized how smart I was back then, it still works perfectly for my needs.

I shoot mostly RAW photos, but occasionally JPG or exotic formats from cameras I borrowed or rented and decided early on to keep my originals separate from edited images. The reason is simple: originals have more value, because they are the raw material and always can be re-edited and converted again. You cannot go back from an edited image to the original.

My goal was to create a simple, hierarchical folder system with some redundancy holding unique filenames and I came up with this file name concept:

YYYYMMDDNNN(N).extension

YYYY      = Year, for example 2009

MM        = Month, for example 09

DD        = Day, for example 12

NNN       = Number, a sequential number of
            photos taken on this specific day,
            usually 3 digits for a maximum of
            999 images per day

(N)       = Overflow Number, in rare occasions
            I take more than 1000 photos
            per day, so I allow to overflow
            the usual 3 digit long number
            into 4 digits.

extension = the original extension
            of the image file, for
            example JPG, NEF, CR2, DNG etc.
            converted to lowercase

With this concept, the 24th JPG photo I would have taken on September 12, 2009 would be renamed to the file name:

20090912024.jpg

There is a lot of information stored in this file name already. The exact date when this photo was taken plus the sequence, which implies that I must have at least 23 more photos taken on this very day. I don’t need any other helping text in the file name like “John’s Wedding“, because this is not specific enough and this reminder might not work after a few years (who was John?).

You might also notice that I pad each number with leading 0’s (zero), because this makes sorting in file systems so much more easy and each filename has basically the same length. For example the month 9 for September becomes 09, and the 24 for the 24th photo of the day becomes 024.

By using the date the images were taken, I can be absolutely be sure that my image names never repeat and I am independent of strange decisions camera makers made or will make in future. The current system used in cameras is based on the DCF standard which allows only for max 9999 unique file names (DCF_0001 – DCF_9999), which are stored in a max of 99 folders, but if a file gets accidentally moved out of a folder, the damage is done.

All other information except the date and sequence don’t belong into the file name, because the so called Meta-Data are stored inside the file and contain the exact date/time, camera information, EXIF etc. Further organization should be done with DAM (Digital Assessment Management) software, which is a topic for another day, this post just describes the way how I store images in a file system so that DAM software actually can access the digital photos.

My storage concept requires some discipline, which I obviously have or my collection would not have survived the last 10 years. Especially it is important to rename new files at ingesting time. For a variety of reasons I don’t use camera specific copy software, because it might alter the files from the camera. I prefer to copy each file straight from the memory card into the file system and rename them there or use vendor independent ingestion software, which renames the files while copying from the memory card.

To store the files in the file system I create subfolders inside a master folder, which basically are based on the year, month and day when the images are taken. To stay with the example above, the file 20090912024.jpg would be stored in this folder hierarchy:

/PhotoLibrary/2009/09/20090912/20090912024.jpg

As you can see, there is some redundancy in the folder naming, for example the day folder contains the entire date of the day, which just repeats the file name structure. I have a reason for that, so let me explain it from the left to the right.

Maybe a necessary side note here: My concept is independent of the operating system it is stored on. Personally I use Macs as my standard workstation, but I developed this concept years ago on Windows and most of my images are now stored on Linux based NAS systems, the folder and file naming structure works everywhere because it is based on the lowest common denominator by using only numbers and characters based on the ASCII character set. No spaces or strange characters here which might mess things up.

Back to the folder structure. The first folder is the enclosing folder which holds the hierarchical subfolders is called “PhotoLibrary“. Inside this enclosing folder are just 10 sub folders, one for each year since I started taking digital photos. The folders are based on the years, so in this case 1999-2009. Even if I take photos for another 30 years, there will be just 40 subfolders. Each of the year folders contains 12 subfolders for the specific months, in this case (padded with 0’s) 01-12. Since every year has a fixed amount of months, it’s a constant that every year folder holds max 12 month folders.

Inside the month folder I could have used just codes like 01-31 for the individual days, but I decided for redundancy and repeat the year and month and add the day for it. This has the advantage if a folder gets accidentally moved or I need all pictures of a single day, I can just copy this folder out of my Photo Library and still know where they belong.

Finally the file name again, the structure explained above makes it simple to move an accidentally moved file easily back into the folder where it belongs.

Another side note: Most file systems regardless of operating system have problems (mostly speed related) with folders holding more than about 6000 files. By using this folder hierarchy, I never have more than the small amount of images in a folder I was able to take in a single day. And even at my absolute record taking day I managed to take “only” 2600 photos, still much smaller than the 6000 file limit.

A picture says more than a thousand words, so here is the example how my Photo Library is organized by showing a selected single day and the hierarchical structure enclosing it.

A lot can be learned from this screen grab above. For example it shows instantly that I haven’t taken any photos on the 1st, 5th, 9th, 19th, 22nd, 28th and 31st of July 2004. What a lazy photographer I am!

That’s basically all, my photo storage is perfectly organized. As a DAM I currently use Apple’s Aperture and all images are “referenced”, but my photo library concept survived already several other DAM software (FotoStation and IMatch) until I settled with Aperture for now.

Lets talk about Backups. Harddisks fail, that’s a given. Its just a matter when. In theory I can just copy the entire PhotoLibrary folder to different disks and have a perfect 1:1 copy. In fact I do just that, but its not that simple because my entire library contains about 6TB of image data at this stage, which is a huge amount of data. But again, my simple concept works even for smaller chunks, for example it would be possible to copy individual year folders to different disks, so that the structure remains the same, just the library is split up. It could look like this:

Disk A:
/PhotoLibrary/1999/
/PhotoLibrary/2000/
/PhotoLibrary/2001/
/PhotoLibrary/2002/

Disk B:
/PhotoLibrary/2003/
/PhotoLibrary/2004/
/PhotoLibrary/2005/
/PhotoLibrary/2006/

Disk C:
/PhotoLibrary/2007/
/PhotoLibrary/2008/
/PhotoLibrary/2009/

In a future post I will explain how I create automated and save backups, but so much for now. The outlined storage concept works very well for me and I also converted my wife with her photo collection to use the exact same concept. Please feel inspired, but remember: It works for us and it might not work for you.