Go to TODAY TUESDAY

What I find personally frustrating is that I cannot use the computer inside my camera in any way other than what the manufacturer intended. Who today would be happy buying a home computer that could only run the software installed at the factory?

Why is my camera not programmable? Why can I not load new abilities onto it just as I do at my desk? Even my phone takes new programs!

What I want to see is a camera that has a language from which I can write programs to control its functions. This could be Python, Java, or even Pawn. It should be obvious from history that as soon as something is completely flexible, people start using it in ways that were never expected.

I like night photography, but the High Dynamic Range of such exceeds the capabilities of modern-day digital sensors. To compensate, I take multiple exposures with different shutter times and merge them using special “hdr” software.

The mixing is a fairly easy process with the exception of the alignment of the photos. Even with a tripod, touching the camera to change the shutter time moves the camera slightly and the resulting photos end up offset by a few pixels in some direction.

If the camera were programmable, I would have it take multiple exposures all by itself,
stopping automatically when it has all that are needed. I would write something like this:

function TakeHDR() {
    raise mirror        // do it once for all shots
    wait 5 seconds      // let mirror shake settle out
    let T = 30 seconds
    do {
        set shutter time to T
        take photo
        save photo
        let T = T / 4   // next at -2 stops exposure
        create histogram of photo
    } while (photo has any pixel's R,G,B at maximum value)
    lower mirror
    signal "all done"
}

With that simple program, I’d get a series of shots, each 2 stops faster in exposure, until there were no overexposed pixels. Because there was no need to touch the camera, all of the images would be perfectly aligned.

How important is this feature to me? I would actually consider switching my equipment from Nikon to another to get this.

If you haven’t heard, there’s a very common problem with capturing multiple images for stitching. It’s called “parallax” and it’s what makes close-by things appear to move faster than far away ones when looking out the window of a moving car. This affects your images when you rotate the camera to capture an adjacent area. If your camera and lens are not pivoting on the optical center of the lens (the point at which light appears to be entering it) then close things will move faster that distant ones as you turn and the final stitched image will have odd discontinuities at image boundaries. A standard tripod will move around the focal plane (the film or CCD sensor) which is not the same thing.

You can spend hundreds of dollars for a nice, heavy “panoramic mount” that will make this problem go away… or you can cheat. The trade-off, of course, is time and accuracy. The trick to reducing parallax errors is to push them to places where it won’t be noticeable after blending. The sky is usually the best choice but anything without long, straight lines will usually work.

Before starting, set everything on the camera to “manual” so nothing changes between shots. This is supposed to look like a single exposure when all is said and done. Take some quick shots in all directions and adjust the exposure so that you’ll capture everything.

Start by capturing the entire scene. A level shot every 45° (vertical mount) or 90° (horizontal mount) plus a few of the sky and the ground will ensure that you have everything. This is important because it’s easy to miss sections in the next step but having this will allow you to fill in any gaps.

Once the images are captured, they have to be warped and stitched. You can pay an obscene amount of money for PTgui or you can use the free Hugin software. There’s no question that PTgui is more robust and has a nicer user interface but there’s no difference in the final output quality. In fact, PTgui uses many of the same (free) back-end programs such as “nona” and “enblend” when creating the final image.

The most time-consuming part of the stitching process is defining the control points. Both programs mentioned above have built-in automated tools to create control points but don’t use them! You want to do this by hand and only add control points along or on either side of the intended seam. Placing control points elsewhere in the image is detrimental since the program will sacrifice some accuracy in the important points to try to satisfy these unimportant ones. You want these boundary regions to align after warping because that is where the seams will go and the closer the areas match between warped images, the less noticable the seam. What happens elsewhere, where there are no seams, isn’t important since we’re distorting the final image so much that any error won’t be noticeable.

To get the “tiny planet” effect, use a “stereographic” or “stereographic down” output with about 300° on both the horizontal and vertical. When you preview the output, you’ll likely find that it looks “wrong” but it’s just a matter of setting the center to where the tripod can be seen. Then you can push, pull, rotate, etc. the preview until you get the general result you want.

Now render the final result. If you’re supremely lucky, the blending program will automatically place the seams in the correct locations and you’ll be done.

However, if you’re not that person, it’s not going to be perfect and you’re going to have to adjust it by hand. To do this, render the output again but this time have it output all the individual images separately.

After loading the blended image into Gimp, Photoshop, or whatever, find the blending mistakes and load the appropriate image into a higher layer. You can then make transparent all of this new layer from the boundary areas outwards, effectively pushing the seam out to those areas. Rinse & Repeat until all seams have been fixed.

If the ground has lines, you’re going to find that the seams are noticeable. It’s going to take some work with Gimp or Photoshop to transform/warp various parts of the images to create something that doesn’t have noticeable errors. In the end, though… It’s all worth it!

215 of these images are 3 rows of 50-some vertical photos at full zoom (200mm) plus another two rows capturing the tops of buildings and the top of the bridge.  There was no point snapping a photo of empty space — there is nothing to anchor it to when stitching.

The remaining 5 images were taken horizontally with an angle big enough to capture the full vertical.  This would provide background imagery of the night sky and harbor that could not be photographed at full zoom and be attached to a landmark.  Since neither the night sky nor an 8-second exposure of rippling water have any detail, nothing is lost.  Note when using this technique: capture full-zoom images that just touch the foreground imagery so that the final stitch will have plenty of “quiet space” around the main content for blending smoothly into the background.

And so ended the easy part.

All of these images were shot in RAW mode because my camera has a 12-bit analog-to-digital converter and every bit is important given the huge dynamic range of night photography.  I usually do these with HDR but if you’ve read my previous HDR adventure (and the follow-up) using only 11×3 images, you’ll understand why I was not about to do it with 220×3 images.

Using Photoshop RAW converter, I optimized the exposure as best I could to reduce blown-out areas to the bare minimum and then converted them to 48-bit TIFFs (16 bits per channel) that the stitching software can read.

When it came to the stitching, I tried both PTgui (v7, not the latest v8) and Hugin.  Both behaved pretty much the same and both died horribly when trying to produce the final output.

Creating the control points took a number of hours.  The automated placement did a good job, covering about 80% of the pairs but some had none at all.  I started the process with PTgui since it’s what I’ve used in the past; for many of the pairs with no control points, asking it to auto-generate points for just that pair usually produced good results.  For the rest, I added a bunch by hand.  Later, when using Hugin (which will read PTgui project files), I also added “vertical line” control points.  These marvelous items tell the optimizer how to make perfectly straight panoramas without having to fiddle with it by hand in the preview pane.

Hugin can also handle shots taken with different lenses or the same lens at different focal lengths.  This was essential in making sure that the 5 “background” images were reasonably well aligned with the 215 “foreground” ones.  PTgui (v7) had no such capacity.

If I had known Hugin had these last two features (vertical line control points and support for different lenses), I never would have tried PTgui, but instead I didn’t switch to Hugin until PTgui proved useless at producing a final image.

The problem, quite simply, was size.  Once everything was assembled, I learned that the final image was to be over 3 GIGApixels in size!!!  Multiply that by 3 colors and 16-bits (2 bytes) per pixel and you get an image that is over 18GB in size.  Ouch!  This is so big that PTgui couldn’t allocate a big enough memory chunk to even warp the images at full resolution.

Hugin (or specifically, “nona”, the warping engine) has a nifty feature of keeping the warped output nearly the same size as the
original and include a position within a larger image encoded in the output file.  PTgui wanted every warped output to be the full 3Gpx in size.

Unfortunately, when it came to combining all the individual warped images, Enblend (the backend “blender” program of Hugin) died for the very same reason as PTgui: It couldn’t manage an 18GB image buffer, even held on disk, when compiled for a 32-bit operating system.  I’m actually running Vista-64 but these programs were built in 32-bit mode and nobody had build 64-bit versions that I could find.

I should mention that PTgui v8 is available as a 64-bit binary that may work…  If you want to shell out USD$216 or so.

Here’s another reason Hugin beats PTgui: It’s “free software”.  The source code is readily available.  I could easily download it and compile it as I needed.  I chose a different path, though.  I transferred the warped images to my laptop and then my Linux workstation at the office which is running a 64-bit distribution of Linux and has Hugin/Enblend available.  A little investigation into the Enblend command-line and a day later (including a lot of disk thrashing), I had a 6GB (compressed) TIFF file with the final image.

Using my laptop as a network again, I transferred the result back home for final exposure adjustment in Photoshop.  No luck.  Photoshop (CS2) won’t read a 6GB TIFF file.  In fact, TIFF doesn’t support files larger than 4GB — that darned 32-bit size thing again.

Back at the office…  This time I have Enblend stitch together one row of images at a time, producing 5 files only 1.5GB in size.  These load into photoshop just fine but I don’t want to hand-blend the rows.

Then I asked Enblend to combine adjacent rows (1&2, 2&3, 3&4, and 4&5).  Since there is a gap between rows 1&3, and 2&4, there must be a stripe in which both the 1&2 and 2&3 composites are identical, meaning blending is just cutting one off with a straight line through that area.  The output size of these row-pair images is about 2.7GB each.  But…

Photoshop still barfed on them.  It seems that it cannot load TIFF files larger than even 2GB.  What now?  I’d already split it down to row-pairs.  To make the files smaller, I’d have to start splitting it vertically as well.  I didn’t want to do that much hand blending of the pieces, though, even if it would be just finding the overlapping regions.

It occurred to me that perhaps I could split these large, blended files into smaller-sized tiles and then just butt them up against each other in Photoshop.  A Google-search later, I found a program to split images into tiles.  It’s $20 but has a trial version.  I try it.  It doesn’t support TIFF.  Ah well, it probably wouldn’t handle huge images either.  Another search later, I find a TIFF splitter.  Same pay/trial deal, but it turns out this only splits whole images from a multi-image file — completely useless to me.

Okay, back to my roots.  Way way back many centuries ago, not long after the Internet began, there existed a collection of image tools called PBM and eventually NetPBM.  This collection of command-line programs consists of binaries to convert pretty much every conceivable non-proprietary image format into a simple, uncompressed format and back, plus a large collection of programs that do various operations on images in this uncompressed format.

For example, if you wanted to scale a PNG up 50% and convert it into a JPEG, you could do something like:

    pngtopam myimage.png \
        | pamscale 1.5 \
        | ppmtojpeg -quality=85 \
        > mybigimage.jpg

The “ppm”, “pbm”, and “pam” names all mean different styles of the same basic format and almost all tools will read all formats.

In this case, the tool I was looking for is “pamdice” which will take an input image and dice it into a number of rows and columns.  By applying this to each row-pair, I can create files of manageable (read: loadable by Photoshop) size and put the whole thing together.

Now that I had files that Photoshop could load, I created a canvas of the desired final size and pasted in the bottom row-pair.  First the left half and then the right half.  So far, so good.

Adding the second row pair involved matching up the overlap.  This is easily done coarsely with a full view.  To align it precisely, zoom in to 100% (ctrl-alt-0) and set the blending option to “difference”.  Then drag the new piece around until swaths of the screen turn black.  Draw a line through this black area and “clear” (make transparent) the lower part of the upper row.  Then change the blending back to “normal” and “merge down”.  Repeat this with all row pairs.  Forget keeping independent layers and using layer masks — the image is simply too large.  There was enough waiting for swapping to disk as it was.

Once all the rows were done, it was time to fix the things that Enblend did wrong.  On a perfectly static scene, I might not have had to do anything.  However, I had a scene with boats going by and some of the choices Enblend made were not correct.

But no problem…  Where a wrong choice was made, I simply cut & pasted the raw warped image and position it using the same technique mentioned above.  Then I would select the area I want, feather it, clear the rest, and merge it.  After every major step, I’d save as PSB since PST won’t handle files this large.

Finally, it was time to combine the foreground and background images. I loaded the latter, scaled it up 4× to match the resolution of the foreground, waited for photoshop, cut & pasted the foreground, wait, wait, and wait some more while Photoshop writes 30⁺GB to its swap file, performed coarse placement of the foreground, wait, switched to “difference” blending and do fine placement, wait some more, etc.

For final blending of the two, I used the lasso to draw about 2/3 way between the landscape and the edge of the image.  I then feathered this selection to make the transition go from 1/3 of the way out to the end and cleared the outside.  This turned out more difficult than I anticipated because, though I had used the same F-stop and shutter speed, the background images were brighter than the foreground ones.  This required some compensation and very careful blending, but the result is nice, gentle transition that is invisible in the night sky and indistinguishable from depth-of-field or natural long-exposure blurring of water.

Since I had deliberately underexposed all of the images to avoid blow-out in the highlights, as the last step I used some “curves” to brighten up the dark areas.  With the extra bits from the raw capture, though, there’s no notable noise or posterization.

A quick crop (well, okay…  no operations are quick on an file this size) to the horizontal boundaries of the high-resolution imagery and it’s all finished.

Final Version (click for full-size viewing on GigaPan)

Bonus:  I’ll email the final full-size image to the first person who can identify the Andrew Lloyd Webber musical quoted above. 

For the most part, it isn’t the eye that “sees” color. It’s the brain. Perhaps that it is why fantastic color is so pleasing. And when it comes to fantastic color, there is just no competing with Mother Nature.

Of course, as soon as you put a frame around it, the image loses something. I remember this moment quite well — I was sitting on an island beach with my new wife when I triggered the shutter. It was probably the most beautiful sunset of our two weeks in Greece, including those we saw in Santorini which is known for fabulous sunsets. I believe this image to be beautiful, but I’m also somewhat biased as I see it as I did then… without borders and my new love at my side.

If color stimulates the brain directly, what about something that stimulates in indirectly? Imagination, for example. It’s this that creates vivid dreams and makes appealing things like naughty on-line chat rooms. Imagination is often better than reality, and why not? There’s no limits to our imagination. As long as one doesn’t live there, it’s a great place for frequent visits.

Look at this image closely. How do you know that the walls are white, the stains are brown, the grass is green, and the little stripe at the top of the hull is blue? Your imagination fills in all the details. Unlike the color image above where you take it in all at once, this one draws the eye all around. Every place you look, you see more that what is there. There is color, but it’s like an overlay; it’s not there and yet you see it anyway.

Go on… Click on the image to see a larger version. Take a look around.

Now I’ll admit, I cheated. The image of the boat isn’t truly colorless. It’s 90% desaturated so you can’t actually see the color, but there is enough present to hint to the mind what it should be. Perhaps a child wouldn’t need it, but I find the imaginations of us adults sometimes need just a little push to get going. Could it be that we don’t use it enough?