| View previous topic :: View next topic |
| Author |
Message |
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
 Posted: Sat Feb 09, 2008 8:03 pm Post subject: TuFuse: Focus and Exposure blending software |
 |
|
I've just released a new software program called TuFuse. TuFuse is a command-line program that performs both exposure-blending (i.e. combining multiple images taken at different exposures into a single "extended dynamic range" image), and focus-blending (i.e. combining multiple images taken at different focus settings into a single "extended depth of field" image). TuFuse can perform either of these functions individually or simultaneously on a stack of images. The following image is comprised of 27 source images (only 9 shown here) taken at different focus settings and exposures. The result is a single image in focus from front to back, with both highlights and shadows well exposed.
TuFuse operates as a standalone program and I'm distributing it as freeware. However, it is also packaged with PTAssembler 4.0, and is PTAssembler's default "stacking plugin" program. By using TuFuse with PTAssembler 4.0, exposure and focus blended panoramas can be easily created. Alternatively, PTAssembler's new "auto-align" feature can be used to make sure a single stack of images is properly aligned before invoking TuFuse. In addition to its panoramic image creation capabilities, PTAssembler works as a GUI for TuFuse, allowing for the alignment of images before stacking by TuFuse.
Below is an example of a focus and exposure blended panoramic image created by PTAssembler and TuFuse. 21 images were combined into three focus and exposure-blended images and then combined into this "vertical panorama" by PTAssembler.
For more details and a download link for TuFuse, consult the TuFuse web page.
For some examples of focus and exposure blended panoramas created with PTAssembler, check here.
Max |
|
| Back to top |
|
 |
michaelshea
Joined: 30 Nov 2004
Posts: 30
Location: Brisbane, QLD, Australia
|
| Posted: Sun Feb 10, 2008 4:20 pm Post subject: |
|
|
| Nice work Max, can't wait to give this a try. |
|
| Back to top |
|
|
smallfluffykat
Joined: 19 Dec 2005
Posts: 50
Location: Bristol, UK
|
| Posted: Sun Feb 10, 2008 4:26 pm Post subject: Which is Better? |
|
|
So I guess I'll ask....
Which is better, the Enfuse+CombineZ or TuFuse? (Apart from that the former you have to hunt for on the interweb and the latter you don't?) |
|
| Back to top |
|
|
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
| Posted: Sun Feb 10, 2008 6:07 pm Post subject: Re: Which is Better? |
|
|
| smallfluffykat wrote: |
| Which is better, the Enfuse+CombineZ or TuFuse? (Apart from that the former you have to hunt for on the interweb and the latter you don't?) |
I haven't really done exhaustive testing, but I guess it depends on what you are trying to do! Tufuse has the advantage that you can throw whatever you like at it (images with different exposure, images with different focus), and it will "do the right thing". CombineZM is a one-trick pony (focus blending only), and while Enfuse can do both, my experience is that the optimal settings for focus blending aren't the same as the optimal settings for exposure blending...
Enfuse seems a bit quicker than TuFuse. But, TuFuse seems to use less memory. CombineZM seems a bit slower than TuFuse. However, they all seem to work pretty well!
Max |
|
| Back to top |
|
|
mvejerslev
Joined: 20 Jul 2004
Posts: 172
Location: Denmark
|
| Posted: Tue Feb 12, 2008 10:44 am Post subject: |
|
|
| Great work on this and the new PTAssembler, Max! I can't wait to try it out, although my pano shooting is a bit low atm. I'll pass this along to my Photoshop list. |
|
| Back to top |
|
|
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
| Posted: Thu Feb 14, 2008 9:11 am Post subject: |
|
|
Here's another sample of en exposure blended image using TuFuse.
I usually don't try and shoot buildings like this at night because it is impossible to retain some light in the sky without overexposing the artificially illuminated building. However, using TuFuse I was able to capture both the sky and the building and fuse them into this single image.
The exposures in this example varied from 15 seconds @ F2.8 to 0.6 seconds @ F4, a difference of about 6 stops.
I also realized that the fusion of multiple images tends to have the nice side effect of reducing image noise in the final result.
Max |
|
| Back to top |
|
|
NEUH
Joined: 26 Sep 2007
Posts: 23
|
| Posted: Thu Feb 14, 2008 11:46 am Post subject: |
|
|
Hi, Max!
Very interesting direction in your works. The combination mosaicing, exposure blending and focusing stacking and all in one is a digital photographer dream.
Congratulations !
On my opinion that would be required at least two o three year before a code without mistakes. And I wish a good way ! |
|
| Back to top |
|
|
alexandre jenny
Joined: 27 Nov 2003
Posts: 140
|
| Posted: Fri Feb 15, 2008 4:05 am Post subject: |
|
|
Congrats.
That's a really interesting project, Max. Good job.
I'm wondering if you would need help to make this project available on mac and linux too. I'm ready to help. |
|
| Back to top |
|
|
Pablo
Joined: 02 Sep 2004
Posts: 13
Location: Germany
|
| Posted: Fri Feb 15, 2008 11:42 am Post subject: |
|
|
Hi Max,
Indeed, TuFuse is very interesting.
The idea to combine both focus and exposure blending is a good one.
Since TuFuse and enfuse are based on the same algorithm, their inner working is quite similar, but due to the two steps made in TuFuse internally, it is really simpler to use than enfuse for this task.
In theory it would be nice to have an algorithm that can deal with both objectives at the same time (eg. fusing a long exposed, far focused background image with a short exposure focussed foreground image), but for almost all practical purposes, the two step algorithm as used in TuFuse is probably appropriate.
From what I read on the TuFuse page, the similar results can be obtained by using enfuse multiple times (for those on OSX and Linux:):
First blend blend the individual focus stacks, for example with:
enfuse -o focus1.tif --wContrast=1 --wSaturation=0 --wExposure=0.001 --HardMask <images>
then blend these with the default parameters (exposure blending):
enfuse -o final focus1.tif ...
Some comparisons between TuFuse and enfuse (Max, please correct me if I'm wrong):
--wMode 1 seems to be equivalent to the enfuse --HardMask parameter
The TuFuse page does not specify how the contrast is measured. While implementing the contrast criteria in enfuse, I found that using a variance based criteria provides better results on massive stacks (like the ones by Rik Littefield) than using a laplacian filter as suggested in the MKR paper.
The gamma (-k) and Sigma (-s) arguments do not have an equivalent in enfuse.
I'm wondering if the gamma parameter has much influence on the result (and if its used during computation of the weights or during the pyramid blending, or both). The -c parameter of enfuse (Use CIECAM02 to blend colors) might have a similar result, although I haven't tested it.
As for the sigma parameter, I assume that the weights of each image are multiplied with with G(relative exposure distance), where G is the gaussian function. I should probably experiment a little with it to see in which cases it is useful.
ciao
Pablo |
|
| Back to top |
|
|
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
| Posted: Fri Feb 15, 2008 12:34 pm Post subject: |
|
|
Pablo,
Thanks for the thoughtful comments!
| Quote: |
| The idea to combine both focus and exposure blending is a good one. Since TuFuse and enfuse are based on the same algorithm, their inner working is quite similar, but due to the two steps made in TuFuse internally, it is really simpler to use than enfuse for this task...From what I read on the TuFuse page, the similar results can be obtained by using enfuse multiple times |
I think this is probably correct. I suspect that you'd get very similar results using this approach. Of course, you need to sort out which images belong to which groups (i.e. which ones get blended during the first iteration, and which ones get blended during the second iteration), and feed them to Enfuse appropriately in multiple invocations. With TuFuse, you can just feed it all the images at once, and it will sort things out appropriately by itself.
| Quote: |
Some comparisons between TuFuse and enfuse (Max, please correct me if I'm wrong):
--wMode 1 seems to be equivalent to the enfuse --HardMask parameter
|
I'm not sure...I haven't actually looked at the source code for Enfuse, and the Enfuse documentation that I've seen didn't make it clear to me exactly what the HardMask parameter does. But, for TuFuse, if you use wMode 1, then it doesn't created a weighted average using multiple input images when creating the laplacian pyramid...it just picks the "best" pixel from one of the input images.
| Quote: |
| The TuFuse page does not specify how the contrast is measured. While implementing the contrast criteria in enfuse, I found that using a variance based criteria provides better results on massive stacks (like the ones by Rik Littefield) than using a laplacian filter as suggested in the MKR paper. |
TuFuse uses the absolute value of the laplacian at each pixel. I think this is an area where I'll be doing some more experimenting in the future, as I suspect there are more elegant ways of detecting focus. Can you explain a bit more about Enfuse's variance based approach? One of the problems with the current approach is that it does tend to enhance image noise. I've thought about using a laplacian at a higher level in the pyramid as a measure of contrast that might reduce noise but have not yet tested this approach.
| Quote: |
The gamma (-k) and Sigma (-s) arguments do not have an equivalent in enfuse.
I'm wondering if the gamma parameter has much influence on the result (and if its used during computation of the weights or during the pyramid blending, or both). The -c parameter of enfuse (Use CIECAM02 to blend colors) might have a similar result, although I haven't tested it. |
Currently the gamma parameter is only used to calculate the relative exposures in each image. It is on my list to use the gamma parameter to "linearize" the image data before calculating gaussian and laplacian pyramid levels, but have not yet done it. I think that, in theory, it ought to reduce the possibility for color shifts and other oddities, but in practice, I haven't seen much of a problem with the current approach.
| Quote: |
| As for the sigma parameter, I assume that the weights of each image are multiplied with with G(relative exposure distance), where G is the gaussian function. I should probably experiment a little with it to see in which cases it is useful. |
Yes...the sigma parameter is the same one described in the "Well-exposedness" bullet point in section 3.1 of the MKR paper. I've done a little testing with this, and for values that are lower, the tails of the Gauss curve are smaller, and very bright and/or very dark pixels get less weight. In fact, here is an example of varying the sigma parameter.
You can see that in the image with a small value for sigma (0.1), really dark (e.g. the night sky) and really light (e.g. the overexposed brick in the building) portions of the input images are given less weight, and so the resulting image shows less overall contrast. In this particular image, it actually looks pretty good...the grass is a little brighter, and so is the sky. I suspect that in other imges, a value of 0.1 might end up producing a "washed out" looking image. I haven't done enough testing with other images to figure out if something other than 0.2 would be a better default value.
Max |
|
| Back to top |
|
|
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
| Posted: Fri Feb 15, 2008 12:37 pm Post subject: |
|
|
| alexandre jenny wrote: |
Congrats.
That's a really interesting project, Max. Good job.
I'm wondering if you would need help to make this project available on mac and linux too. I'm ready to help. |
Thanks for the offer...I'll let you know!
Max |
|
| Back to top |
|
|
Pablo
Joined: 02 Sep 2004
Posts: 13
Location: Germany
|
| Posted: Fri Feb 15, 2008 1:13 pm Post subject: |
|
|
Hi Max,
| Quote: |
| Quote: |
Some comparisons between TuFuse and enfuse (Max, please correct me if I'm wrong):
--wMode 1 seems to be equivalent to the enfuse --HardMask parameter
|
I'm not sure...I haven't actually looked at the source code for Enfuse, and the Enfuse documentation that I've seen didn't make it clear to me exactly what the HardMask parameter does. But, for TuFuse, if you use wMode 1, then it doesn't created a weighted average using multiple input images when creating the laplacian pyramid...it just picks the "best" pixel from one of the input images.
|
Indeed, this is what the --HardMask parameter does. maybe I should have spend more than a few seconds before giving it a name ;-)
| Quote: |
| TuFuse uses the absolute value of the laplacian at each pixel. I think this is an area where I'll be doing some more experimenting in the future, as I suspect there are more elegant ways of detecting focus. Can you explain a bit more about Enfuse's variance based approach? |
It takes all pixel values inside a window (3x3 by default), and calculates the variance. This is then used as focus measure of the center pixel of the window. For RGB images I use the norm as a contrast measure. sqrt(var_r + var_g + var_b). By this I get something like a mean standard deviation instead of the variance. If used with --HardMask, one could obmit the sqrt altogether, though.
Multiply by some factor to scale the value to a similar level as the other criteria.
| Quote: |
One of the problems with the current approach is that it does tend to enhance image noise. I've thought about using a laplacian at a higher level in the pyramid as a measure of contrast that might reduce noise but have not yet tested this approach.
|
This is a general problem of this method. Both in focus details and noise are high frequency image content. Its the same dilemma the various noise removal algorithms have to deal with, separating noise from image content.
By using a the laplacian on a lower frequency, one will loose image details, but get a smoother picture. A locally adaptive measure could be used, maybe similar to the anisotropic smoothing of the greycstoration program, to decide if a pixel is an image feature or just noise. However, this is a lot more complex than a simple variance or laplacian filter.
| Quote: |
Yes...the sigma parameter is the same one described in the "Well-exposedness" bullet point in section 3.1 of the MKR paper. I've done a little testing with this, and for values that are lower, the tails of the Gauss curve are smaller, and very bright and/or very dark pixels get less weight. In fact, here is an example of varying the sigma parameter.
You can see that in the image with a small value for sigma (0.1), really dark (e.g. the night sky) and really light (e.g. the overexposed brick in the building) portions of the input images are given less weight, and so the resulting image shows less overall contrast. In this particular image, it actually looks pretty good...the grass is a little brighter, and so is the sky. I suspect that in other imges, a value of 0.1 might end up producing a "washed out" looking image. I haven't done enough testing with other images to figure out if something other than 0.2 would be a better default value.
|
I see. I think this is probably a nice tuning parameter, might have a nicer effect than adjusting the exposure and saturation weights. A low sigma will make the result look less contrasty, in the sense that middle tones are preferred even more. In enfuse it is currently hardcoded to 0.2.
Pablo |
|
| Back to top |
|
|
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
| Posted: Fri Feb 15, 2008 8:10 pm Post subject: |
|
|
| Pablo wrote: |
| Quote: |
One of the problems with the current approach is that it does tend to enhance image noise. I've thought about using a laplacian at a higher level in the pyramid as a measure of contrast that might reduce noise but have not yet tested this approach.
|
This is a general problem of this method. Both in focus details and noise are high frequency image content. Its the same dilemma the various noise removal algorithms have to deal with, separating noise from image content.
By using a the laplacian on a lower frequency, one will loose image details, but get a smoother picture. A locally adaptive measure could be used, maybe similar to the anisotropic smoothing of the greycstoration program, to decide if a pixel is an image feature or just noise. However, this is a lot more complex than a simple variance or laplacian filter.
|
I just tried using a lower frequency laplacian level as an alternate measure of focus, and the results don't look too good. The resulting focus blended images are less noisy, but not nearly as "well focused". I think the gains (less noise) don't come nearly close enough to the cost (less overall focus) to use this approach...I'll keep experimenting with other ideas!
Max |
|
| Back to top |
|
|
maxlyons
Joined: 20 Jun 2003
Posts: 3347
Location: USA
|
| Posted: Mon Feb 18, 2008 6:55 pm Post subject: |
|
|
I've just uploaded version 1.11 of Tufuse. No big changes in this version, but a couple of small bug fixes, and a minor speedup.
Max |
|
| Back to top |
|
|
rjacobs
Joined: 04 May 2007
Posts: 81
Location: Virginia
|
| Posted: Tue Feb 19, 2008 5:50 pm Post subject: Re: TuFuse: Focus and Exposure blending software |
|
|
Max,
How do you determine the number of focus increments for a scene? This one for example. Thanks.
Richard
Below is an example of a focus and exposure blended panoramic image created by PTAssembler and TuFuse. 21 images were combined into three focus and exposure-blended images and then combined into this "vertical panorama" by PTAssembler.
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
Subscribe
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
