Canon 1Ds Mark 2

I take the archiving of my digital photos seriously. My photos are the archived memories of my family. I have developed over the years, a means whereby I sort, store, and archive them. I’ve been asked repeatedly how I do this, so I thought I would write it up once and for all.

I use a Canon 1Ds Mark II camera which is 16.7 Megapixels. I also shoot exclusively in Raw with the DSLR, which yields files that range in size between 13 and 22 megabytes each. Each of thes .CR2 raw files must be “developed” using special software. The resulting .jpg images create an additional file of about  two to four megabytes. Then I may crop or alter the file, making a new copy of the full-sized .jpg. Then there are the web-sized versions and the thumbnails which are only 100k or so. After all my editing, each single image capture from my camera might consume a total of 25 megabytes of disk space with all copies considered – more if there are many versions.

My first rule, is that I let the camera name the files according to whatever scheme it uses. I may configure it once, but I do not rename my image files. Thus I might get an image name entitled P1040730.jpg from my Panasonic point and shoot camera, or _B0Z6573.CR2 from my DSLR. Back in the days of my 1.2MP Kodak DC120 I would rename the photos, but now I take in excess of 10,000 images a year, and I just don’t have the time. Honestly I just don’t care about the image names anyway, opting instead to use directory names to identify each event.

As soon as I copy the images from the camera’s card, they are put onto a mirrored RAID pair of drives. I once had a drive fail during the transfer and I lost 6 Gig worth of pics. That was not a happy day. The next day I set up RAID so that a drive failure wouldn’t hurt me. Once they’re on the RAID pair, I delete them from the card. I have two drives for photography:

• Current – two 250G mirrored drives
• Archive – One 1TB drive

The Archive drive gets upgraded every year because it fills up. Luckily my disk space needs seem to run right behind what $100 will buy me that year, so it works out. Every couple of years I have to upgrade the mirror pair as well.

All of my photography web pages have an underlying hierarchy on the server. The top level will be a master archival index called archive-index.html. The next level will contain all of the years. Within each year are all of the event folders for that year. Each event folder contains the images and HTML for that event. The hierarchy can be thought of like this:

Archive-Index
   \-YYYY
       \- YY-MM-DD_EventNameWithNoSpaces
       \- YY-MM-DD_EventNameWithNoSpaces
   \-YYYY
       \- YY-MM-DD_EventNameWithNoSpaces
       \- YY-MM-DD_EventNameWithNoSpaces

On my computer, there is more complexity than on the web server, but the basic format is the same. The folder hierarchy for both the Archive and Current drives is the same.The root contains only year folders. The year folders contain event folders. The event folders contain that event’s raw files. The event folder might contain a Develops folder for processed jpgs, but will not if a point and shoot camera was used. The Event folder will also contain a duplicately named folder that will contain web-sized copies, thumbnails and HTML code. This folder will eventually be copied or moved to my HTML drive for inclusion into my Family page.

YY=Year, MM=Month and DD=Day. Thus:

YYYY                                         # There is nothing here but folders
   \- YY-MM-DD_EventNameWithNoSpaces         # This folder contains all the Raw files
       \- Develops                           # .jpgs developed from raw
       \- YY-MM-DD_EventNameWithNoSpaces     # Web sized versions of .jpgs

In practice, it might look like this:

2009
   \- 09-08-16_AnnieAndTheTrash
       \- Develop
       \- 09-08-16_AnnieAndTheTrash

   \- 09-08-17_GibsonLesPaulR8
       \- Develops
       \- 09-08-17_GibsonLesPaulR8

You might have notice the odd way in which I depict the date in my folder names. The year is listed first, then the month, then the day. In this way the folders always sort properly. If I were to use a normal American date format like 08-16-09_Event, then August-2009 would sort with August-2008 which makes me twitch. By using my format, the system will always sort properly. This is less of an issue with the event folders separated into year folders, but being this detailed always pays off in the long run.

There are some very specific aspects to the folder names. They have saved me countless hours of coding and have let me do some pretty cool things over the years.

• Each section of the date: YY-MM-DD is separated by a hyphen
• The date is separated from the event name by an underscore
• There are never spaces or any non-alphanum characters in the event name
• Each word in the event name is capitalized

These may looks like the random rantings of a crazed old programmer, and they are, but there is logic, and logic is our friend. By separating the date from the event name with an underscore, I can write scripts and trust that everything to the left of the underscore is the date, and everything to the right is the event name. By using hyphens, I can always parse the date. By never using spaces, I can guarantee that the folder name will work in all operating systems, and be understood by all browsers. Similarly, by not allowing characters like apostrophes, I can ensure that my script will work on multiple operating systems.

The folder with the web-sized versions of the images has the same name as the original because it will be copied in whole to my HTML drive where I will add it to my web page. The same logic works there as well. Once this folder is copied to the HTML drive, the folder names will be the same, and will be sorted the same way as the originals. Additionally, I’ll easily be able to tell from the folder name – which is included in the thumbnail HTML page – where to find the image on my drive.

Once I have all the pics processed and settled, I make two copies to DVD-DL. One DVD-DL stays home in the safe, and the other one goes to a safety deposit box at the bank. Seriously. I have a box at the bank that has nothing but hundreds of DVD disks in it. I could rebuild both my home PC and my servers in the event of a catastrophe.

At the end of each year, I copy the entire year over to the archive drive and create a new folder on the Current drive for the new year. I also delete the current year from the RAID pair which frees up space for the new year. Since the archived files are also backed up to multiple DVDs, there is no longer a need for RAID.

Having recently bought a Mac Powerbook Pro, I had to come to grips with using a new operating system. I have used Windows XP since it was a beta in 2001. This being the end of summer in 2009, you could say that I’m a tad set in my ways when it comes to my computing tasks.

The original Macintosh

The Mac Powerbook Pro was so enticing a piece of hardware though, that I was willing to look at Mac OSX. I was no stranger to Macs having used them since my roommate bought an original Macintosh back before time began. Though at the time I liked my Tandy 1000 better, the Mac was pretty darn cool too.

Still, having one around and using one as my primary mobile computing platform were two different issues. I’m a working consultant, so I need my laptop to work – period. I also need to not be trying to figure out how to use my computer while charging my client by the hour. That’s just bad form.

My first issue was the fact that I have many XP-only pieces of software. The thumbnailer program that I use for my photos is Windows-only, and I like it so much that I will forever keep a windows machine around to use it. Luckily with the Mac, there’s a better way.

In the apple store, one of the salespeople showed me Apple Parallels, which allows Windows XP (or most other operating systems) to run in a virtual instance at the same time as OSX. Think VM-Ware if you know what that is. In fact VM-Ware has a similar product called Fusion that many people prefer.

While Parallels looked cool, I wondered about moving files back and forth between OSX and XP. I was then shown how files can be dragged between OSX and XP windows and my jaw hit the floor. I told them to wrap it up. Fusion is also capable of this type of integration.

Once I had the safety net of being able to run XP within my OSX environment, I knew that I could install my normal XP software and just boot XP when and if I needed it. There was no no danger of needing to figure something out at a client site.

I then moved onto trying to get myself to a point where I wouldn’t need XP. Sure certain things like thumbnailer needed XP, but I only use that stuff when I’m on big trips. I usually do all my photography processing on my big home machine.

My biggest needs on a portable computer are:

• Writing – both personal and business
• SSH and terminal
• Visio
• Email
• Web browsing

Business writing and Visio both demand Microsoft Office. I was willing to buy a Mac version of office, but from all the reviews I’d read, the latest version sucked, and the old version was for power-PCs. While the older version would work with my Intel processors, I’m not a fan of paying for old versions. I’ve relegated business writing to XP using Parallels. It works, I have the license, and I’m not messing with it.

With Visio there is no choice. It simply does not exist on the Mac. Yes I know there are other packages that do the same thing, but my clients use Visio, and I am expected to give them Visio files. The easy fix for me is to use MS Office under Parallels, so that’s what I do, and it works without incident.

For personal writing, I don’t need MS Office, so I set about finding something else. I tried the demo version of iWork and hated it. I tried Open Office and had stability problems. I then discovered Neo Office, which is an OSX-specific port of Open Office. I love it. I’m using it to write my book and have had zero problems with it over the past two months.

SSH is a big deal to me since I primarily use PINE for email. Call me old school, but it works, and it’s impossible to get a virus, worm, trojan horse or anything other than offensive words with PINE. I’m using Alpine now on Ubuntu, but still, I need a nice SSH app to get to my servers.

In XP I use Secure CRT and love it. There is, as of yet, no OSX version of Secure CRT though. Luckily, for 95% of my needs, I don’t need it. OSX is UNIX, and it inherently contains SSH. Once I got my terminal window to look the way I liked, I was set.

While I use PINE for my text based email, I do occasionally need a graphical client. I prefer Thunderbird, and I tunnel the connection through SSH to my server. Works like a charm just the way I like it to.

Web browsing was similarly solved with Firefox, though I’m sad to say that since version 3.0, Firefox has been irritating me in a fairly regular basis. I find that Safari is a surprisingly good and stable browser, which I use about 75% of the time while on the Mac.

Some things are still mildly bothersome as a long-time XP user. For example:

• I dislike the fact that I can only resize a window from the bottom right corner
• I dislike the fact that with most apps, I cannot maximize the window
• I dislike the fact that there does not seem to be the idea of an insert key, even with a 101-key keyboard
• I dislike that there are no dedicated pg-up/pg-down keys on the Mackbook

Still, these are minor complaints. My insert key complaint is due to my habit of using shift-delete to cut and shift-insert to paste in windows. I’ve learned to use Apple-x and Apple-v like I should. The other stuff is mostly habit, and I’m finding that over time these things bother me less and less.

I’ve found that I can reproduce the Page-up and Page-down functionality by using the spacebar and shift-spacebar in every app I’ve tried so far.

Many people worry or complain that there is no right-click on a Mac. They’re wrong. Not only is right-click supported if you get a third-party mouse, but the included trackpad has a right-click button. For some reason it’s disabled by default! Turn that on and you’re golden.

It took me a lot less time than I expected to become proficient in OSX. Sure I still might need to hunt for some odd feature here and there, but overall I’m thrilled with the experience so far. I like it so much that when my kid’s PCs expire, I’ll likely replace them with Macs.

By the way, in the few months since I’ve owned my Powerbook, I have never once had to find or load a driver. I’ve not had issues installing or removing any software. I’ve not seen a conflict of any kind. I hate to repeat hackneyed sayings, but this thing just works. Oh wait! I did have it lock up on me – twice! Each time XP had consumed all the memory and was stuck in a race condition within Parallels. I got into terminal and killed the process and got it working again. I love Unix.

GAD

Apple Macbook Pro

I hate laptops. In fact I have a statement that I repeat whenever anyone asks me to recommend a laptop:

“All laptops suck – buy what you like”

The purists will note that we haven’t seen a laptop in the market for over 10 years. Laptops were huge beastly things that harken back to the days of floppy disks and orange plasma screens. What we’re talking about of course, are notebook computers. The word laptop just rolls of the tongue better than notebook does, so I, like most people, will continue to misuse the word. Besides, I use it on my lap all the time. Thanks, I feel better.

At any rate, I found myself recently needing a laptop because my HP widescreen laptop blew a hard drive and developed an LCD problem that I couldn’t live with. I fixed it up as best as I could, and donated it to my youngest daughter who now uses it exclusively for important tasks like surfing Webkinz.com.

Being a business expense, and thus tax deductible, I decided to hunt out the best, most powerful machine I could get. I had the following requirements:

• 17 inch non-glare wide-screen w/ 1920×1200 resolution
• dual-core processor
• 4G RAM
• Non-squishy keyboard
• Sturdy case – metal if possible

That’s it. Doesn’t seem hard does it? I quickly discovered that it was. Most laptops on the market are disposable pieces of crap. Anything under $1000 seems to be made of thin plastic with toy keyboards and low-quality parts. The good side of this is that for $500 you can get some serious horsepower and RAM. I considered this because the alternative was a $3000 solution. If a laptop lasts me about three years on average, I’d be saving money if I just bought a $500 laptop every year and ran it into the ground.

The non-glare aspect of my requirements killed a LOT of contenders. The other thing that killed most laptops under $1000 was the non-squishy keyboard requirement. The keyboards on most laptops suck. Sure some have full keyboards with number pads and a lot of other cool features, but type on them for more than a few minutes and you’ll feel the cheapness of them. Cheap keyboards fail me. Nothing will make me want to hurl a laptop through an open third story window like a failing keyboard.

I’m a working consultant, and I’m also a writer. I Can type 130 words per minute, so I need a keyboard that can keep up with me and won’t fail after typing 10,000 words per day. Additionally, since I often have to sit in odd places while I’m working, the glossy screens that are so prevalent don’t work for me. Sit near a window with a glossy screen and you’ll know what I mean.

I am also a photographer, so a quality wide screen is important to me. As a writer I like the widescreen so I can put two full pages side by side. I’m spoiled, I’ll admit it. My HP laptop, though widescreen, was only 1280×960, which was damn-near useless for me. It looked fabulous watching movies, but for real work, it left me cold. I need pixels, and I need a lot of them.

Luckily, and a bit surprisingly, the dual core processor and 4G RAM requirements were no big deal. I quickly narrowed my choices down to two contenders: The Lenovo W700ds and the Apple Macbook Pro 17″

Lenovo W700ds

Loaded the way I wanted, they were both around $3000. I had used Lenovo laptops in the past, and was excited at the prospect of getting one, until I started reading reviews. Apparently, Lenovo had changed out the best feature of the business class laptops – the keyboard. Additionally, there were some reports that the sevice from Lenovo had gotten pretty bad. The days of killer support from IBM was gone. For $3000, I want excellent support.

I’ve had my Apple Macbook Pro 17″ non-glare laptop for abour four months now, and I like it more each day. It routinely get’s over seven hours of battery life, the screen is magnificent, and  the keyboard rocks. I didn’t like the keyboard at first. There are no insert keys, no home/end/pgup/pgdown keys, and some other niggles, but it’s not squishy, and it’s kept up with me admirably.

This Macbook, is in fact, the first laptop that i’ve ever had that doesn’t suck. The case is aluminum and is rock solid. The screen is magnificent. The keyboard is wonderful. Even the trackpad is great, though I use my Logitech wireless mouse 90% of the time.

I had to learn about OSX, which I’ll write about later, but even if I need Windows XP, I bought Parallels which works wonderfully.

I’m a convert. My next computer purchase will likely also be a Mac, even if I have to buy one used.

GAD