kbImport: A Photo/Audio/Video Filing System

Sometimes, you just have to do it yourself.

There are easy ways for a creator to manage digital media, but they tend to be a bit inflexible, or they don’t scale well, or you just never can figure out where your photos are actually stored, which can make backups confusing (or expensive, if you’re paying a monthly fee for cloud storage).

A couple of years back I decided to unify a single storage method for photos, video footage, and audio field recordings. My method needed to apply to all three media types, could handle external drivees, and needed to be able to run on most any kind of device: Mac (at the time, my homme machine), Windows (at the time, my office machine), or Linux (added for travel, using a tiny Asus Eee PC and a La Cie rugged drive).

The result was a python script called kbImport3 – it’s worked for me for over a decade now, with a few minor tweaks like adding DNG support, AVCHD video support, and a minor tweak for running on Chromebook.

Note: this article was split-out from a larger article on Chromebook storage in 2020, but it’s not Chromebook-specific)

Structured or Unstructured Filing Systems

Ask ten photographers about their filing systems, and you’ll probably get a dozen different answers, even accounting for the three photographers who report: “I don’t have one.”

Windows, MacOS, Dropbox, Adobe Bridge and Lightroom (and others) all offer super-simple media storage methods, usually triggered when you insert an SD card or reader into your PC. While all of these can be useful, I found them frustrating, since they tended to lock you into a single proprietary, pur-way-or-the-highway mechanism and pattern for storage. I prefer a scheme that’s flexible in terms of what’s being stored and how I might use it later. None of the “default” methods seemed to be transportable between multiple drives, mixed kinds of computers, different brands of software, different cloud services, etc.

As is typical for digital media, convenience usually comes with padded handcuffs. So I devised my own very simple system, based in part on how I’d already been filing 35mm contact sheets: by date and “roll number.” For digital workflow, I think of “roll number” as “job name” – e.g., “GarciaWedding” or “Marseille” rather than “Roll_B.”

This isn’t a sales pitch: while I do share kbImport.py on GitHub, it’’s possible to use the same filing scheme – or nearly so – using tools like PhotoMechanic or Adobe Bridge. What those don’t offer is the wider range of computer, drive, or media types. And they’re not free.

For folder and file arrangement, I use a simple pattern of easily-sorted dates, owner, and project: YYYY-MM-DD-owner-jobName, repeated not only in the hierarchy of folders, but also in the file names themselves, so I can keep track of loose files even if they’ve been copied elsewhere (say, to this website).

Here’s the basic pattern of archive folders and files:

media_type/
  YYYY/
    YYYY-MM-Mon/
      YYYY_MM_DD[_job]/
        [prfx_][job_]original_name1
        [prfx_][job_]original_name2
        [prfx_][job_]original_name3 ...

…where media_type will be Pix, Audio, or Vid – the script can manage imports from both cameras and audio recorders. If more than one kind of media is present on a single SD card, each file will be sent to an appropriate folder.

To use the script, I connect the SD card (and usually an archive hard drive), open a terminal window and run kbImport from python (or python3). Here’s a typical case:

python3 kbImport.py -p photographerName -j Sample

The script will look for available media (the SD card), an available archive location (the external drive, or if not available, an appropriate internal-drive location), set up the appropriate folders and start copying.

The dates used are by default the dates of the original files: if I don’t get to make backups right away, no problem, they’ll all be correctly labelled & filed. In the example, since we’ve set the kbImport prefix to photographerName and the job name to Sample then if the SD card has a camera-original image named “DSCF8513.JPG” shot on 6 April 2020, the archive folder on the hard drive will be something like this:

Pix/2020/2020-04-Apr/2020_04_06_Sample/photographerName_Sample_DSCF8513.JPG

a video shot on the same camera on the same day will get its own parallel path:

Vid/2020/2020-04-Apr/2020_04_06_Sample/photographerName_Sample_DSCF8516.MP4

Joining Archives

Sometimes I may want to archive my SD cards but don’t have my external archive drive. Or I might want to collect archives into a larger, permanent store or network archive. If there’s no external-drive archive, the script will just aerchive to the local hardrive. It’s easy to later copy it to an external drive when one is available: since the archive folder structure is always consistent, in almost every case archives can be joined by just drag-and-drop to the appropriate location on the Mac, Windows, or Linux desktop.

Cloud Folders

On some machines, that “local archive” is in a folder that’s automatically synced to Google Drive. This makes an easy way for me to grab them on any other device, including my phone or the Chromebook Files app.

I used to use the same strategy with Dropbox, the mechanisms are similar. I’ve recently found that you can connect Dropbox to Files thanks to a Chrome extension, so the strategy can still work if you prefer that service.

Work Directories

This filing system is file for importing photos etc, but where do they go while being worked on? I use a manual system that plays well with kbImport by defining a different work directory for each year, and then sort work directories within that folder.

For this year’s photo jobs:

Pix/
  2020/
    Work2020/
      projectA/
        (edited pictures that are part of project "A")
      projectB/
      projectC/
      insta2020/
        (misc pictures resized for instagram, flickr, etc)
      pix2020/
        (pictures for this blog)
      sms2020/
        (small pictures for sending in messages and email)
      (plus any "loose" pictures)

This copies as easily between drives and computers as the automated directories.

I usually have some other random source-image folders, say pictures I’ve scrounged from the web, mailed to me, etc. I just keep them in their own dated subfolder(s).

Pix/
  2020/
    from_web_2020/
      (random photos I found online)

Doppelganger Avoidance, aka “Smart Copy”

I often forget to erase content from my SD card(s) after I’ve archived them. kbImport catches this, and won’t re-archive media that’s already stored – even if the second import has specified a different job name. It can reconize that, say, bjorke_Hedge_XT1A9633.jpg and bjorke_Scoot_XT1A9633.jpg are the same file.

Overrides

Some behaviors can be also changed by command-line options. For example, if you want to unify the archive dates to today rather than the original file-creation times, just add -u – a complete list can be found on the github site or by just typing python kbImport3.py -h

It’s open source python, easily changed. Sometimes, you just have to do it yourself.