20D Noise Reduction Questions

RichardRichard Administrators, Vanilla Admin Posts: 19,893 moderator
edited June 25, 2006 in Cameras
Custom function 02 enables in-camera noise reduction when exposure time is greater than 1 sec. I set up my camera with this enabled when I first got it and never much thought about it since. Last night, I tried to shoot some fireworks and was reminded that noise reduction greatly increases the time between shots. In fact, the delay is equal to the exposure time of the last shot. So that got me wondering:

1) Is noise reduction applied to RAW files at all or only to JPG? I was shooting RAW only but the delay was certainly there.

2) How does the quality of in-camera noise reduction compare to PP software? I have generally used RSE or CS2 noise reduction when I needed it.

3) Is there any shooting situation where in-camera noise reduction has an advantage over PP?

Thanks for your wisdom. :thumb

Comments

  • ziggy53ziggy53 Super Moderators Posts: 23,764 moderator
    edited June 24, 2006
    rsinmadrid wrote:
    Custom function 02 enables in-camera noise reduction when exposure time is greater than 1 sec. I set up my camera with this enabled when I first got it and never much thought about it since. Last night, I tried to shoot some fireworks and was reminded that noise reduction greatly increases the time between shots. In fact, the delay is equal to the exposure time of the last shot. So that got me wondering:

    1) Is noise reduction applied to RAW files at all or only to JPG? I was shooting RAW only but the delay was certainly there.

    2) How does the quality of in-camera noise reduction compare to PP software? I have generally used RSE or CS2 noise reduction when I needed it.

    3) Is there any shooting situation where in-camera noise reduction has an advantage over PP?

    Thanks for your wisdom. thumb.gif

    Richard,

    The way this particular noise reduction works is to allow the camera to take a "black frame" exposure, which basically builds an image mask, based on "your" imaging chip "at that point in time". The idea is that the noise varies from pixel site to pixel site, and it varies according to the duration of the exposure and according to ambient temperature (and some other stuff, but those are the biggest contributors.) This mask is used to offset the "predictable" noise produced by that sensor during a long exposure. There is still random noise, but that tends to be reduced automatically by the averaging effect of the long exposure itself.

    Since this frame is not the same from camera to camera, or even day to day in the same camera, this is truly a customized mask, optimized for "that" exposure. There is no way to duplicate this action or its effect in software.

    That said, noise reduction software can still be used to good effect against the problem, it just won't be as effective. It may be effective enough, but you need to run tests for yourself.

    ziggy53
    ziggy53
    Moderator of the Cameras and Accessories forums
  • ziggy53ziggy53 Super Moderators Posts: 23,764 moderator
    edited June 24, 2006
    I should also mention that some astro-photographers make a seperate "black frame" exposure every half-hour, or so, and then use that frame as a layer to semi-automatically reduce the effects of long exposure noise. It is considered usually "close enough" to what the camera can do automatically, and can eliminate the wait you describe when the camera does it automatically.

    So you have another choice available to you. :D

    ziggy53
    ziggy53
    Moderator of the Cameras and Accessories forums
  • RichardRichard Administrators, Vanilla Admin Posts: 19,893 moderator
    edited June 24, 2006
    ziggy53 wrote:
    I should also mention that some astro-photographers make a seperate "black frame" exposure every half-hour, or so, and then use that frame as a layer to semi-automatically reduce the effects of long exposure noise. It is considered usually "close enough" to what the camera can do automatically, and can eliminate the wait you describe when the camera does it automatically.

    So you have another choice available to you. :D

    ziggy53
    Thanks for the explanation, Ziggy. I don't understand the black frame layer option, though. I gather that you shoot an entirely black frame, say, with the lens cap on. Then you import that as a layer into a subsequent shot. Right? Anything that is not 0,0,0 on that layer is presumably noise. But what do you do next? headscratch.gifscratch

    Cheers,
  • ziggy53ziggy53 Super Moderators Posts: 23,764 moderator
    edited June 24, 2006
  • erich6erich6 Registered Users Posts: 1,638 Major grins
    edited June 24, 2006
    Thinking about this as a "layer" will confuse the issue. It really isn't a layer and in reality it is not "noise" removal. Noise is random in nature by definition. What you do with a dark frame is capture the inherent pixel-to-pixel nonuniformity in the array (there's always some distribution of dark current flowing through). Think about it as removing a "bias" from your image.

    Generally this isn't an issue because it is such a small amount compared to the scene signal. But, when you are doing long exposures you're usually dealing with low signal levels and now your long exposure integrates not only the scene but this dark current as well. The dark subtraction effectively gets rid of this.

    Your question about the RAW image is a good one. I think the RAW image also gets the dark frame subtracted otherwise I would expect to see a RAW dark frame out of the camera and I've never seen that. Anyone out there know the answer to this question?

    Erich
  • erich6erich6 Registered Users Posts: 1,638 Major grins
    edited June 24, 2006
    OK. I was intrigued by this and I've now confirmed that the RAW file also gets the dark frame subtracted when the noise reduction option is set. I performed two tests one at 15 sec exposure and another at 80 sec. In both cases I left the lens cap on and used the remote shutter to take two frames: one without noise reduction and one with.

    What I found is that for both cases the noise reduction is effective at removing nonuniformity due to stray light but at the expense of slightly more random noise in the 15-sec exposure (the 80-sec exposure was improved in both uniformity and noise).

    I'll post the images and stats from the histogram later. I checked both RAW and in-camera JPG's and both result in the same conclusion. Stats were all derived from the RAW image.

    Erich
  • gusgus Registered Users Posts: 16,209 Major grins
    edited June 24, 2006
    rsinmadrid wrote:
    and was reminded that noise reduction greatly increases the time between shots. In fact, the delay is equal to the exposure time of the last shot.

    3) Is there any shooting situation where in-camera noise reduction has an advantage over PP?

    Thanks for your wisdom. thumb.gif

    I cant shed light on the technicalities RS but i was in all sorts of bother when i tried it. I thought something was wrong with the camera after my first & only star trail shot. It works great though. I should have tried to do the same shot without it turned on to see the difference.

    This is 55 min exposure.

    50436409-M.jpg
  • NikolaiNikolai Registered Users Posts: 19,035 Major grins
    edited June 24, 2006
    What Ziggy said
    In-camera noise-reduction is hardware-only, RAW or not. It simply "takes ANOTHER shot" with all the same parameters (like ISO and shutter speed) without lifting the mirror and thus blocking the light from the lens hitting the sensor. Since it happens immediately following the actual shot, the idea is that the dark currents in a non-uniform array should produce a distribution similar to what happened in the actual shot. After that built-in software takes this distribution into account and adjusts the original image while it is still in the buffer (i.e. before writing to your card, even in case of RAW file).

    It definitely can be helpful, but you have to be ready to pay the price of waiting (exactly!) twice as much. Thus, if you're doing super long shots (like in Gus case, you'd have to wait almost an extra hour), you may simply miss your next one, unless you count for this effect.

    I normally keep this setting off, since I rarely do very dark and very long exposures. However, I agree that it helps with the noise if switched on.

    I have my own idea how to clean with the "stars trails" shots, however. We were discussing Zion stars shots while being at the post-shootout and the idea dawned upon me.:):

    See, in those shots most of the noise is coming from the skies. Trees and rocks (and other ground-based objects) are usually pure silhouettes, or can be easily made into such, so noise is not the problem there.

    Start with making a new copy of your layer (Ctrl+J).
    To get rid of the sky noise you need to find an exact "center of rotation", which is relatively easy task. Mark it with two rulers (H and V). Then Select All (Ctrl+A), Free Transform (Ctrl+T) and then - move the central point to the exact intersection of the rulers (it will normally snap to it). Then rotate the image a little. If the trails skew (you can verify it by changing the layer's opacity to 50% or less) - you made a mistake in selecting the center and need to repeat the procedure from the start (ESC to undo the Free Transform). If the trails remain in place - you're golden. Rotate image just so the most (or all) of the "noise particles" completely clear their original locations. Press ENTER (to lock the results of the Free Transform).
    And now - drum roll - change the blending mode of your upper (rotated) layer to Darken (or Multiply for an even stronger effect). You will notice that *all* the noise is wiped out - just for the price of slightly affected length of the trails. You can always adjust opacity of the layer if the effect is too strong, of course.
    If the center of rotation is "off the grid" (not on your image), you still can try to figure out where it is by zooming out (Ctrl+"minus") and using a "larger than life" selection rectangle before proceeding to Free Transform.
    As a variation of this method you can apply it in several different steps, with different radii, each time masking out the areas you don't want to be affected.
    Try it, you may like it:-)

    HTH
    "May the f/stop be with you!"
  • NikolaiNikolai Registered Users Posts: 19,035 Major grins
    edited June 24, 2006
    Doppler effect in action
    gus wrote:
    50436409-Ti.jpg

    It's really funny how such a simple thing as a digicam on a tripod at night can efectively tell you which stars fly away from the Earth (red ones) and which one come towards it (bluish ones)..
    They should include this shot (or one like it) into Optics (or Physics) text books... thumb.gif
    "May the f/stop be with you!"
  • gusgus Registered Users Posts: 16,209 Major grins
    edited June 24, 2006
    Nikolai wrote:
    It's really funny how such a simple thing as a digicam on a tripod at night can efectively tell you which stars fly away from the Earth (red ones) and which one come towards it (bluish ones)..
    They should include this shot (or one like it) into Optics (or Physics) text books... thumb.gif

    Sure its not the temp nik headscratch.gif

    http://zebu.uoregon.edu/~soper/Stars/color.html
  • NikolaiNikolai Registered Users Posts: 19,035 Major grins
    edited June 24, 2006
    Could be that, too:-)
    gus wrote:

    .. and I tend to forget the simplest facts before thinking of some way more complicated ones...rolleyes1.gif

    But my point remains: this IS a physics textbook shot, be it doppler or simply solor=temperature effect :):
    "May the f/stop be with you!"
  • luke_churchluke_church Registered Users Posts: 507 Major grins
    edited June 24, 2006
    gus wrote:

    I suspect it will be almost exclusively temperature + mank in the atmosphere + some of them may be planets.

    The majority of stars appear red-shifted which is supposed to be a justification for the expanding universe. There are also apparently funkier reasons to do with physics that I don't even pretend to understand.

    Or at least that's what clever people than me have told me.

    Nice photo anyhow, it clearly deserves to be in a text-book for its "cool" factor if nothing else. Pun is intentional.

    All the best,

    Luke
  • gusgus Registered Users Posts: 16,209 Major grins
    edited June 24, 2006
    Nikolai wrote:
    .. and I tend to forget the simplest facts before thinking of some way more complicated ones...rolleyes1.gif

    But my point remains: this IS a physics textbook shot, be it doppler or simply solor=temperature effect :):

    headscratch.gif Simplest of facts ...i think its a bit more than that mate.
  • luke_churchluke_church Registered Users Posts: 507 Major grins
    edited June 24, 2006
    gus wrote:
    headscratch.gif Simplest of facts ...i think its a bit more than that mate.

    Nah mate, relativistic stellar physics is what Nik does when he's too tired to do something that actually challenges him mwink.gif):
  • NikolaiNikolai Registered Users Posts: 19,035 Major grins
    edited June 24, 2006
    I know you're pulling Gus' leg
    Nah mate, relativistic stellar physics is what Nik does when he's too tired to do something that actually challenges him mwink.gif):
    .. but it was one of my projects in my senior years...:-)
    "May the f/stop be with you!"
  • erich6erich6 Registered Users Posts: 1,638 Major grins
    edited June 24, 2006
    OK. 15-sec exposure, lens cap on, no noise reduction:

    77465748-M.jpg

    mean: 42.85 st.dev.: 33.87 median: 38

    15-sec exposure, lens cap on, noise reduction ON:

    77465755-M.jpg

    mean: 50.51 st.dev: 39.59 median: 46 (note bottom right corner stray light goes away...also seems to do well getting rid of vertical structure but not so much the horizontal)

    80-sec exposure, lens cap on, noise reduction OFF:

    77465753-M.jpg

    mean: 59.82 st.dev: 45.48 median: 55

    80-sec exposure, lens cap on, noise reduction ON:

    77465751-M.jpg

    mean: 69.83 std.dev: 51.29 median: 65 (same effect as 15-sec exposure).

    Erich
  • NikolaiNikolai Registered Users Posts: 19,035 Major grins
    edited June 25, 2006
    Erich,
    Thanks for the samples!

    Interesting - deviation gets higher (worse) with the noise reduction ON. yet grid pattern partially disappears..
    "May the f/stop be with you!"
  • RichardRichard Administrators, Vanilla Admin Posts: 19,893 moderator
    edited June 25, 2006
    I had no idea this was so complex, but I think I understand the big picture here. Let me try to replay this to see that I have it right:

    1) In-camera noise reduction does affect the RAW file.
    2) In-camera noise reduction addresses a problem inherent in long-exposure, low-light shots. This problem is distinct from the random noise that typical PP software addresses.
    3) The longer the exposure, the more useful in-camera reduction is likely to be.

    Based on all that, I think I would probably have been better off without the in-camera processing when I was shooting fireworks. I could have taken twice as many shots, and a slight increase in noise wouldn't really matter much. Or I could have taken a single black frame with fn-02 off, then done some work in post to approximate the in-camera processing.

    Thanks so much, guys, for all the useful information. So little time and so much to know ...

    Regards,
  • Red BullRed Bull Registered Users Posts: 719 Major grins
    edited June 25, 2006
    Wow, I've actually learned a lot about this noise reduction from reading this thread. Thanks for all the info!
    -Steven

    http://redbull.smugmug.com

    "Money can't buy happiness...But it can buy expensive posessions that make other people envious, and that feels just as good.":D

    Canon 20D, Canon 50 1.8 II, Canon 70-200 f/4L, Canon 17-40 f/4 L, Canon 100mm 2.8 Macro, Canon 430ex.
  • NikolaiNikolai Registered Users Posts: 19,035 Major grins
    edited June 25, 2006
    Exactement, mon ami!
    rsinmadrid wrote:
    Let me try to replay this to see that I have it right:

    1) In-camera noise reduction does affect the RAW file.
    2) In-camera noise reduction addresses a problem inherent in long-exposure, low-light shots. This problem is distinct from the random noise that typical PP software addresses.
    3) The longer the exposure, the more useful in-camera reduction is likely to be.

    Oui:-)
    rsinmadrid wrote:
    Based on all that, I think I would probably have been better off without the in-camera processing when I was shooting fireworks. I could have taken twice as many shots, and a slight increase in noise wouldn't really matter much. Or I could have taken a single black frame with fn-02 off, then done some work in post to approximate the in-camera processing.

    Thanks so much, guys, for all the useful information. So little time and so much to know ...

    Regards,

    Yes, probably. With the fireworks (and lightings, etc) you do want to get more shots, so I'd turn it off (which I usuaaly do, as I mentioned before:-). When working with stars (which do not go anywhere as rapidly as fireworks) it might be useful to turn ON - provided you're ready to pay with 200% of your time.

    Good luck! thumb.gif
    "May the f/stop be with you!"
  • Antonio CorreiaAntonio Correia Registered Users Posts: 6,241 Major grins
    edited June 25, 2006
    Red Bull wrote:
    Wow, I've actually learned a lot about this noise reduction from reading this thread. Thanks for all the info!
    So have I.
    Thank you all.
    thumb.gif
    All the best ! ... António Correia - Facebook
Sign In or Register to comment.