Macro Depth of Field Technique

Discussion forum for Tawbaware's PTAssembler software, Helmut Dersch's Panorama Tools software and any other photography related software

Moderator: spamkiller

Lin Evans
Posts: 67
Joined: Wed May 12, 2004 4:16 pm
Location: Berthoud, CO
Contact:

Post by Lin Evans » Mon May 17, 2004 1:25 am

Though these threads seem to be primarily focused on panoramas and associated issues, I thought some may be interested in a technique to enhance depth of field in situations where one can take multiple images.

First let me say that I'm using a rather loose definition of "macro" as a close-up, and not necessarily 1:1 or greater, even though the same technique is applicable.

Since depth of field is a major issue for those who love macro photography, and since I don't recall it being discussed here - at least not recently if at all, I thought I would do a few samples showing one way to get enhanced depth of field, provided the subject doesn't move and you have time to shoot a few frames from a tripod.

Essentially, those using a fixed lens digicam have an enormous advantage in many respects over the photographer using a dSLR. In some ways it's a two edged sword though. The fixed lens digicam offers much more depth of field at even F6 than is practical with a good macro lens and a dSLR simply because the tiny sensor and lens is shooting macros at very short focal length. Typically it may be as low as 8mm to 12mm. Most of our macro lenses for larger format begin at around 50mm and many like to use 100mm or 180mm to get a bit of working room.

So when we shoot with our 35mm platform dSLR's, and even with our fixed lens digicams, we always would love greater depth of field. Fortunately for us, we can "sometimes" borrow a technique from photomicroscopy of combining multiple images taken of the same subject with differing focal points into a single image with great depth of field. The reason I say "sometimes" is that there are a couple conditions which must be fulfilled. First, with fixed lens digicams, it's less of an issue because of their lens design, most do not greatly change the "size" of the object when changing focus. With 35mm dSLR's we must find a macro lens which doesn't greatly change image size with fine focus adjustments. Fortunately there are a few around. Second, we must be able to shoot from a tripod and preferably with a remote release device.

For this exercise, I selected a Huchol art object (beaded deer) which stands about 7 inches in height. I shot it from above at an angle of around 50 degrees which gave plenty of room to show depth of field.

To make it even more difficult, I let the camera (Canon 10D with 100mm Canon F2.8 macro lens) select the aperture and it averaged around F3.5. I used the built-in flash and took a total of 8 frames. I began by focusing on the very top of the ear and progressively changed the focus point until at the 8th frame I had focused on the base.

Because I didn't use a cable release (one should) there was a tiny bit of frame shift which caused some areas to not be perfectly focused because the software was not able to perfectly align before combining the frames into a single image.

To avoid wasting bandwidth, and because there are a total of 9 frames, I'll provide links rather than embed the images.

If you look at each frame, you will see the progression of "in-focus" areas. All in all, the software did a rather nice job of making one image of 8 differing frames.

Now the best part :-) The software is free. If I get some interest, I'll post another message with a link to the free software and to commercial software (not cheap) which serves much the same purpose.

Best regards,

Lin

http://www.lin-evans.com/combine/file001.jpg
http://www.lin-evans.com/combine/file002.jpg
http://www.lin-evans.com/combine/file003.jpg
http://www.lin-evans.com/combine/file004.jpg
http://www.lin-evans.com/combine/file005.jpg
http://www.lin-evans.com/combine/file006.jpg
http://www.lin-evans.com/combine/file007.jpg
http://www.lin-evans.com/combine/file008.jpg
http://www.lin-evans.com/combine/combine.jpg



[Edit by Lin Evans on Friday, May 28, 2004 @ 12:01 PM]

johnh
Posts: 2118
Joined: Sun Jul 20, 2003 2:43 pm
Location: UK
Contact:

Post by johnh » Mon May 17, 2004 3:22 am

Originally posted by Lin Evans
Because I didn't use a cable release (one should) there was a tiny bit of frame shift which caused some areas to not be perfectly focused because the software was not able to perfectly align before combining the frames into a single image.
I aligned the images with Pano Tools (optimizing y,p,r and fov of individual images), and ran them through Helicon Focus for comparison. There's not a huge difference:

Image

John

Lin Evans
Posts: 67
Joined: Wed May 12, 2004 4:16 pm
Location: Berthoud, CO
Contact:

Post by Lin Evans » Mon May 17, 2004 11:44 am

Actually, there's probably no difference at all in the outcome. There are several ways to "skin a cat" :-) It's possible to do this manually in PhotoShop with layers, masks, history brush, etc., but the "advantage" of doing it the way I did is that the software is "free" and there is no text message affixed as in the freeware version of the Helicon Focus.

I use Helicon noise filtering and the Helicon Focus product is quite reasonably priced as is the Pro version when compared to the "giant" in the industry, Syncroscopy's Auto-Montage:

http://www.syncroscopy.com/syncroscopy/am.asp

But even the entry level product at Syncroscopy is around $600 (about the price of the Pro-level helicon product:

http://helicon.com.ua/pages/index.php?heliconfocus

But the great "deal" of the century is the amazingly powerful CombineZ freeware which is nearly as powerful as the nearly $3000 Auto-Montage:

http://www.hadleyweb.pwp.blueyonder.co. ... uction.htm

It's easy to use (took about 1 minute to do the eight frames), priced right and powerful.

Best regards,

Lin


[Edit by Lin Evans on Monday, May 17, 2004 @ 12:33 PM]

djcheetah
Posts: 16
Joined: Sat Feb 21, 2004 10:49 am
Contact:

Post by djcheetah » Mon May 17, 2004 12:18 pm

Lin,

Could you post the link to the free software?

Thanks!

DJCheetah

Lin Evans
Posts: 67
Joined: Wed May 12, 2004 4:16 pm
Location: Berthoud, CO
Contact:

Post by Lin Evans » Mon May 17, 2004 12:36 pm

Sorry, I forgot the proper syntax here (not used to using the "" in front and " behind - it's posted in the link above and now corrected - but here it is again for the CombineZ software I used:

http://www.hadleyweb.pwp.blueyonder.co. ... uction.htm

Lin

karlg
Posts: 73
Joined: Thu Jul 31, 2003 7:39 pm
Location: Silicon Valley
Contact:

Post by karlg » Wed May 26, 2004 5:53 pm

Lin

Thanks for starting this thread. I've been interested in this
general technique for a while. Here is a pic I made from
multiple shots, hand blended in PS:

http://www.pbase.com/image/28622076/large

larger version
http://www.pbase.com/image/28622076/original

Sorry, I don't have the component shots online at the moment, but there
were about 6 shots, each with just a part of the flower in focus. BTW:
they were shot with the Canon 100 f2.8 USM Macro lens.

I think I have a pretty good method for doing the combining in PS. If
someone is interested, I can post an explanation (it's fairly simple once
you get the hang of it, but an explanation would take more typing than
I want to do unless someone is interested).

After this came up a few weeks ago (on this board or some other one, can't
remember) I tried the Helicon software (on this image or a similar one,
can't remember that either -- getting too old), but the results weren't
good enough for me and I couldn't see an easy way to fix things up in PS.
Ideally, I'd want the program that combines the images to have an output
capability like the Panorama Tools "PS with masks" so I could adjust the
blending in PS similar to adjusting the seams in a pano. I looked a bit
at the CombineZ pages and see something about outputting a depth map
file, but it wasn't obvious to me whether there would be an easy way to
get the images and the depth map into PS in some useful form. Any
insights on this would be appreciated.

Actually, now that I think about it, I realize that I could use the single
output image from the combining software (Helicon or CombineZ) as a bottom
layer that would have most things in focus and then I could put all the
original shots as layers above and use my manual technique to adjust the
layer masks to fix just the parts that the algorithm got wrong. That
would probably save a lot of time.

On the issue of "registering" the component images (that is: moving, rotating,
and scaling the images so they line up):

First, my experience with the 100 f2.8 macro is that even though it doesn't
change length when you focus at different points, the image size _does_
change. You can see this by looking at the scales on the lens: setting a
different focus distance sets a different magnification. IIRC, I read in
a John Shaw book that this will happen with any lens. I have a macro
slider plate and believe that in theory I could use it to move the camera
to counteract the change in image size from changing the focus ring
position, but my experience is that in practice it would be impossible to
actually do this -- you'd have to memorize the "old" size of the subject and
move the camera just enough to get the subject back to that size.

So, second, can I use PTA/PT to "register" the images? In the past, I've
had some success using PT to register images I wanted to blend to extend
dynamic range (with the nice side effect that I get one file that I can
load into PS to get all the images in as layers -- does anyone know an easy
method to load N image files into PS as layers of one image?) I tried
using PTA/PT with my Columbine flower images. As suggested on this board
some time in the past by Max or johnh or one of our other gurus, I even
optimized for FOV to try to get PT to scale the images as well as align
them (ie. P, Y, R). However, the output image had the
big "projected onto a sphere then back to a flat surface" bulges on each
edge. So big that I would lose a large portion of the image if I cropped
them off. So my question is, how do I get PT to do the registering of the
images without distorting them? IIRC, I was able to reduce the bulging
by setting a bogus initial FOV (90 degrees) for the images. Is this the
best way to reduce the distortion? Is there an optimal initial FOV? If
yes, but it isn't a fixed value, how do I determine it?

(Just to be clear, the image linked above didn't use PT. I hand aligned,
rotated, and scaled the component images in PS before blending. Later
I experimented with PTA/PT and Helicon to see if they would make it easier
to do the registering or blending.)

Thanks in advance to anyone who provides answers.

Karl


[Edit by karlg on Thursday, June 3, 2004 @ 02:40 PM]

Lin Evans
Posts: 67
Joined: Wed May 12, 2004 4:16 pm
Location: Berthoud, CO
Contact:

Post by Lin Evans » Fri May 28, 2004 12:33 pm

Hey Karl, it's been a while!!

Actually, I'm probably not the best source for the PT alignment - JohnH would likely know that answer.

You should definitely download and try CombineZ (tiny code) which does its own alignment (and apparently, some sizing too) and seems to do a pretty good job in general.

All of these various tools appear to work better with tiff or bmp files which have no jpg artifacts to interfere with the algorithm. In any case there are frequently some "halo" artifacts which need to be cloned out in PhotoShop. I think your idea of using the output from the combine process in PhotoShop layers has great merit and could help with removal of the halo artifacts without resorting to clone techniques.

Another technique I find useful is to use deconvolution algorithms such as Focus Magic to touch up small areas (masked in PhotoShop) and fine tune the finished product.

It's nice to have choices in ways to enhance the DOF - it's absolutely true that even with ideal lenses there will be some changes in image size (magnification) between frames. The smaller the changes the easier the alignment, at least for the automated processes. But the main advantage I find is in the amazing speed which these algorithms can do the majority of the "rough" work.

Some who looked at the Huichol Deer in my samples noticed the OOF on the bottom of the deer's forward most ear. It was out of focus in all the originals because of the very limited depth of field per frame and does show that the algorithm works properly in that it neither added to nor deleted from the "best" available edge focus points in the individual frames. This is a situation where Focus Magic could easily "correct" the few pixel misfocus locally. Of course had I used F8 or greater, this would not have been an issue at all, but I purposely used very wide apertures because it more easily demonstrated what the algorithm was doing.

Best regards,

Lin


[Edit by Lin Evans on Friday, May 28, 2004 @ 01:15 PM]

johnh
Posts: 2118
Joined: Sun Jul 20, 2003 2:43 pm
Location: UK
Contact:

Post by johnh » Fri May 28, 2004 1:49 pm

Originally posted by karlg
So, second, can I use PTA/PT to "register" the images?
Yes you can. You should not get any "bulges" as there is minimal change to each image. If you specify the input and output projections to be rectilinear, then the output image should look very similar to the input image, even though it has undergone a process of mapping to a spherical surface and back again. If you fix the HFOV of one image, then the HFOV of the other images can be optimized to take account of changes in the lens focusing distance.

John

ns
Posts: 85
Joined: Wed Dec 10, 2003 4:38 am

Post by ns » Sun May 30, 2004 12:21 pm

About registering the images:
I combined two images for dynamic range expansion.
I have aligned them with PTA.
I found two important points:
1. Autopano did not perform well, and I had to select control points manually. I assume it was because of the difference in the exposure that make the two images look different to autopano to the point it gave bad results.

2. The micropositioning had similar problem, probably from the same reason, i.e. the two images are not similar enough.
and it actualy have shifted the points to wrong positions,
so I had to reposition all the control points.

Any suggestions for better workflow (not manual)?

maxlyons
Posts: 3647
Joined: Fri Jun 20, 2003 8:55 pm
Location: USA
Contact:

Post by maxlyons » Sun May 30, 2004 1:17 pm

The micropositioning had similar problem, probably from the same reason, i.e. the two images are not similar enough.
The microposition feature compares the intensity patterns of each image. If the two images use different exposures, then micropositioning will not work as well.

Max

richayd
Posts: 1
Joined: Mon May 31, 2004 5:30 pm

Post by richayd » Mon May 31, 2004 6:05 pm

I would greatly appreciate it if Karl would be willing to take the effort to share his PhotoShop methods. I've been trying to get it right in PS for almost a year and haven't been very successful. Many thanks and best regards,

Richard

karlg
Posts: 73
Joined: Thu Jul 31, 2003 7:39 pm
Location: Silicon Valley
Contact:

Post by karlg » Thu Jun 03, 2004 2:28 pm

Originally posted by richayd
I would greatly appreciate it if Karl would be willing to take the effort to share his PhotoShop methods. I've been trying to get it right in PS for almost a year and haven't been very successful. Many thanks and best regards,

Richard
OK, here is my photoshop method for blending for DOF. Maybe it will be of
use to some or better still, maybe some gurus will suggest improvements or
alternatives.

The general approach is to build a stack of layers from the bottom up.
Each layer has a layer mask that lets everything from below pass up
the stack except for the part of that image that is in focus. That is,
each layer mask will be mostly black and just have white where that image
"contributes" an in focus section to the overall image.

The key to the technique is being able to toggle the "current image" on
and off so you can spot the areas where it is in focus but the current
image (ie as computed from lower layers with their masks) is not.

Steps:

0. If necessary, use Photoshop or PTA/PT or something else to adjust the
images so they will be "registered" (translated, rotated, scaled) so they
line up correctly when stacked. (This is mainly what I was asking about in
my post.)

1. Decide what order you will stack the images. For example, if you carefully
took the images moving focus from near to far, then you may want to stack
them in the order you took them or the reverse. If as in my Columbine flower
picture you took a bunch of pictures with focus set for different parts of
the subject, you might want to order them from those with the most in focus
areas to those with the least amount of in focus area. The ordering shouldn't
affect the final result, but it might save you work. With my columbine
image, I made some notes and drawings ahead of time about which part of
the flower I expected to get from each image. This may help, but the beauty
of this technique is that it helps you decide which portions of each image
are used in the final image.

2. Make a copy (Image>Duplicate) of the first image. This will be the "work
image" where we build up the stack of layers.

3. Add the next image to the stack. (Open the image, select all, copy,
switch to the work image, paste). It can be helpful to rename the layer
so you know which picture it came from.

4. Make a copy of this new layer (Ctrl+J) So now the top of the stack
has two copies of the current image on top. The upper one will be a
temporary "flipper layer" and the lower one will be the one that we keep
and will have a layer mask to select its in-focus areas.

5. Add a layer mask to the lower of the top two layers. (Don't have PS here
at work, but it's something like Layers>Add Layer Mask>Hide All or
Reveal All -- don't know which, but the one that makes the initial mask
all black. Another way is to click the New Layer Mask icon at the bottom
of the layers palette, then click the layer mask icon on the layer, then
use the paint bucket tool to fill it with black)

6. Now toggle the flipper layer on and off (ie click its eye icon). Each
time you turn it on, look for areas that go from out-of-focus to in-focus.
Mentally note that area, if any.

7. If you didn't note any areas, you are done adding this image to the
stack. Delete the flipper layer and go to step 3.

8. Turn off the flipper layer.

9. Using a soft edged brush with foreground color set to white, paint that
area on the layer mask. (That is, click the layer mask icon then paint on the
image.) As you paint, the area will go from OOF to in-focus. If you paint
too far, undo or switch to black and paint where you went to far.) Of course,
you can always toggle the flipper layer on and off to test whether you have
gotten the whole in focus area or have gone too far. When you are satisfied
that you have the current area right, go to step 6 to look for other in focus
areas in the current image.

Once you get the hang of it, it goes pretty fast.

Of course, if you used PTA/PT to register the images, you can output a
layered Photoshop file that already has all the images as layers. This is
what I actually do, so I don't actually add images in step 3 but rather I
have all the upper images that I haven't worked on turned off.

Although the process sounds completely algorithmic, there is still a lot of
art to it. For example, you paint on the layer mask of one of the images
to make it contribute an in-focus version of one of the petals of a flower.
But if you just paint the area covered by that petal in that image, the
edges of the blurry version of that petal from another image may show. Most of
the time, you can just paint a bit beyond the edges of the in focus petal,
but sometimes you then include something that's blurry in the image that has
the in-focus petal. So you end up having to adjust brush sizes and brush
edge softness. In some cases, you may have to do other things like using
the clone tool, as I did a few times when I blended the Colombine shot.

I hope this explanation is clear enough. Feel free to ask about anything
that isn't clear. (However, often I only get around to checking this forum
one or twice a week, so please be patient for an answer.)

Karl





[Edit by karlg on Thursday, June 3, 2004 @ 02:32 PM]

rjlittlefield
Posts: 5
Joined: Tue Jul 06, 2004 2:36 am
Location: Richland, WA, USA
Contact:

Post by rjlittlefield » Tue Jul 06, 2004 2:47 am

Readers of this thread may be interested in some material that I have posted at http://www.janrik.net/ptools/ExtendedFocusPano12/ . This is strictly experimental code that I am working on as time permits, but it may be in good enough shape to help out other people.

Quoting from the web page...

"...here is an experimental version of pano12.dll that determines
visibility and computes masks from estimated focus instead of just
image overlap.

The images produced by this pano12.dll are similar to those produced by other extended focus software, but are easier to edit because the visibility masks are preserved in the output.
...
The computed masks are present in "Photoshop with feather" (PSD_mask)
and "Multi-image feathered TIFF" (TIFF_mask) outputs.
...
This code uses the classical variance method of estimating focus. An
array of "best source" image numbers is computed, recording at each
pixel position the PTStitcher image number that has the largest
variance of pixel values within a focus-estimation window. Then
masks are computed as follows. For mask type m0, at each pixel, the
mask value is 255 for the best image and 0 for all others. For mask
type m1, at each pixel, the mask value is 255 for the best image and
all lower-numbered images, and 0 for all higher-numbered images. For
mask type m2, the array of "best source" image numbers is smoothed by
a simple averaging filter. Then at each pixel position, the mask
value is computed as 255 if the image number is less than or equal to
the average, 0 if the image number is greater than the average+1, and
linearly interpolated between 255 and 0 as the image number is
between average and average+1. I make no claim that this is a
particularly good way to do the job, but it was quick to get working
and generates perhaps surprisingly good results."

Check out the pictures and maybe the code, and let me know what you think.

--Rik


[Edit by rjlittlefield on Tuesday, July 6, 2004 @ 02:50 AM]


[Edit by rjlittlefield on Tuesday, July 6, 2004 @ 02:51 AM]


[Edit by rjlittlefield on Tuesday, July 6, 2004 @ 02:53 AM]

karlg
Posts: 73
Joined: Thu Jul 31, 2003 7:39 pm
Location: Silicon Valley
Contact:

Post by karlg » Tue Jul 13, 2004 6:38 pm

rjlittlefield wrote:Readers of this thread may be interested in some material that I have posted at http://www.janrik.net/ptools/ExtendedFocusPano12/ ...
Rik

Hopefully I can find time to give it a try (I still haven't gotten around to trying CombineZ that I downloaded weeks ago.)

I really like how this gives you the images and masks, so you can adjust after the fact. It looks like the m0 option results in masks similar to what I'd get using the manual method I described earlier in the thread. I'm having a bit of trouble getting my mind around the differences between the mask modes and why I'd want to use a given one. More explanation would be appreciated.

Karl

Lin Evans
Posts: 67
Joined: Wed May 12, 2004 4:16 pm
Location: Berthoud, CO
Contact:

Post by Lin Evans » Wed Jul 14, 2004 2:50 pm

Hopefully I can find time to give it a try (I still haven't gotten around to trying CombineZ that I downloaded weeks ago.)
Hi karl,

You might want to go back and download the latest version since there has been an upgrade with some substantial improvements in areas of image quality. I received an email from Alan Hadley, the author of CombineZ a couple days ago. Alan had seen the thread here and had experimented with the images I linked (beaded deer) and had made some improvements resulting in better image reproduction. These changes are reflected in the latest release. The file name will be identical to the one you downloaded, so just overwrite the old with the new.

Best regards,

Lin

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests