Tutorial: Basic Layer Mask Mechanics

If you have successfully used layer masks in Photoshop, move along--this tutorial is not for you. Rather, it is directed at people who have never been able to get started at all because they have stumbled on the basic mechanics of creating and modifying a mask. Some highly accomplished photographers have had difficulty understanding these concepts, so I thought a Chapter 0 primer might be helpful. My only goal is to clarify the differences between the layer content, the layer mask and the layer mask display.

A mask makes part of a layer transparent. In a portrait, you might want to sharpen only certain portions of the subject. By using a sharpened layer with a mask on top of an unsharpened layer, you can sharpen only once then experiment with changing the mask to control which areas appear sharpened. While you can accomplish the same thing using a combination of the eraser and history brush, masks have many intermediate and advanced applications that the eraser cannot do at all.

To begin, I created an image consisting of two layers, one yellow and one blue:



The blue layer is active (as shown by the highlight in the layers palette). By clicking on the icon circled in red, I added a layer mask to the blue layer.



Notice that a mask icon now appears in the blue layer. It is all white, which means that nothing is currently being masked (that is, hidden) in the blue layer. It also has a black rectangular frame in it, which means that the mask is currently selected for editing, rather than the blue pixels themselves. Finally, notice that a Layer 2 Mask channel has been added automatically, but is currently not checked to be visible. By making it visible and making the RGB channels invisible, the mask itself can be displayed. Note that you can do this quickly by simply alt-clicking on the mask icon in the layers palette. Currently, it is simply all white:



After making the RGB channels visible again by checking the little RGB box in the channels palette, I took a black brush and painted an M on the mask. Notice that the mask icon is selected in the layers palette and the mask channel is visible. You can see the M on the mask icon, but the blue image icon remains solid blue.



The M looks orange, not yellow because the layer mask channel is set to be visible. It displays as red at 80% opacity in this example. Having the layer mask visible allows you to see the content of the layer (all blue) simultaneously with the mask. To just see the effect of the mask, click the little box in the Layer 2 Mask channel to suppress the mask channel display:



The black part of the mask blocks the pixels in that layer, thus allowing the pixels from the layer below to show. It is almost like erasing them, though they are not really gone. If you turn off the visibility of the yellow layer, this is what the blue layer looks like by itself:



Notice that the blue image icon in the layers palette is still solid blue. The pixels of the M are still there, but are being blocked by the mask. If you want to see the mask itself, click off the RGB channel visibility and click on the Layer 2 Mask channel (or alt-click on the mask icon in the layers palette).



The mask is always a gray scale image. Anything in black is completely blocked, that is, becomes transparent in that layer. Anything in white is completely visible. Anything in between is visible in proportion to the gray level. Note that since I used an airbrush to paint the M, the edges are not sharply defined, but have a gray halo. You can see the effects of this by looking closely at the yellow edges above. You can paint directly on the mask if you like, but you will not be able to see what the mask is covering. Sometimes this is useful for cleaning up areas of the mask that have imperfections of various sorts.

You can add more areas to be blocked by simply painting in black with the layer mask selected in the layers palette. Likewise, you can paint in white on the layer mask to reveal an area that has been blocked. Here, I used a white brush and painted over the middle part of the M.




The blue pixels reappear in the part I painted the mask white. Note that I did not have to use a blue brush. The blue pixels were always there; they were just made invisible by the mask. By turning off the yellow layer, you can see the current state of the masked blue layer:



I hope that by now the difference between the mask and the layer image is starting to become clear. If not, you should play around with the example above until it starts to make sense. There are two other areas that seem to cause a lot of confusion that I want to touch on.

The first is the color of the mask display when it is made visible. Take the image above and make the mask visible by checking its box in the channels palette. Now double click the Layer 2 Mask channel:



The layer mask display is not the layer mask itself, but rather a colorized, partially transparent version of the mask. Remember, the mask itself is always black, white and gray, and the level of grayness controls the visibility of the image in that layer. The mask display, on the other hand, lets you visualize the areas that are masked while looking at the image itself. In the above case, the actual black of the mask is displayed as red at 80% opacity. The actual white of the mask is transparent. Thus, the masked vertical shapes appear as orange-yellow from the underlying layer and 80% red from the mask. This color is for your convenience only and does not affect the final image in any way. You can control both the color and opacity of the mask display to suit your needs. 50% red is the PS default, but if you are working, say, on an image of a sunset, the red of the mask display is going to get lost in the colors of the sky. If you double-click on the color in the popup box, the PS color selector opens and you can select any color you want for the mask display. I chose green:



The mask itself has not changed at all. Only the display of the mask is affected. The same is true of the opacity of the mask display. It just makes it easier (or harder) to see the mask and the underlying image depending on the setting. The image itself is not affected. If you turn off the mask display by clearing the Layer 2 Mask box in the channels palette, you see that the image is still only blue and yellow.



OK, one final point. The layer mask alone does not prevent changes from being made to the image in that layer. If I click on the layer 2 image icon (thus making the image active, not the mask) and fill the layer with red, the entire layer becomes red, but the mask still prevents the masked area from showing:



If you right click on the mask icon in the layers palette, a popup menu lets you delete or disable the mask, among other things.



I deleted the mask, and you can see that even though I did the red fill with the mask present, the entire layer became red.



If you want to make changes to a layer that affect (or exclude) only the masked area, you can create selections from the mask using this same popup menu, then make your changes. That's beyond the scope of this primer, though.

Masks are powerful tools that help you make changes to very specific parts of your images while leaving the rest untouched. What I have covered here is like the part of the computer manual that tells you how to plug it in and turn it on. To see how you can use masks in real-world editing, check out the fine tutorials that Andy and Pathfinder posted some time ago.

Hope this helps somebody.