10
The Keying Effects
The term keying is used to describe the process of removing part of a layer to reveal the layers beneath it. This is usually used to refer to the removal of a blue or green screen background for compositing. In case you were curious, blue and green are the colors most often used because they contrast the most with the colors in skin tones, no matter how light or dark the skin. You would think that after all these years of having this work-flow, that you could now just click a button and everything blue or green would be gone. Unfortunately, that’s not the case. Keying is a craft, sometimes requiring many tools to do the job. Thus, we have many effects in this category, all with the same intention of removing parts of a layer to isolate objects to be composited.
The third-party keying tool Keylight is far and away the most powerful keying tool of the bunch. Unless you’re not shooting against a green screen or a blue screen, you should always start with Keylight (covered later in this chapter).
Of course, some of the rest of these effects are more important than others. And not all of them work with blue or green screens. Sometimes, you might want to use keying effects—such as the Luma Key effect—to remove very bright or very dark areas from a layer. Or, you may want to remove a multi-colored background, in which case you might want to check out the powerful Inner/Outer Key effect. Additionally, you can use the very powerful (and automated) Rotobrush tool, which isn’t technically a keying effect at all. My point is that while not all workflows require the removal of blue or green screens, most workflows still can benefit from some degree of keying.
Here’s an example of a recent project I did using these keying tools. I started with some green screen footage of myself reading a newspaper.
I started by removing the green screen using some of the methods discussed in this chapter. I then composited this footage on top of a 3D model of a subway scene, courtesy of
Figure 10.2
The green screen footage composited with other elements with the green screen removed.
Kymnbel Bywater of spilledinkanimation.com. Kymnbel created the subway scene with transparency in the window. This allowed me to add a video that my wife took whilst hanging a camera outside of the window of our car on a drive up the Oregon Coast in the background. I also masked out the stool that I was sitting on, added a reflection in the window, and played with the color using the effects we covered back in Chapter 6. As you can imagine, the proper use of keying techniques can allow you to create a plethora of virtual situations.
Universal Keying Concepts
Before we get started into the individual keying effects, I want to cover a few concepts that are important to keep in mind while keying. A few of these concepts are just a good rule of thumb, while others will be issues you will be dealing with on most keying jobs.
About Spill and Spill Suppression
With most keying effects, even after keying out the blue or green screen, you are still left with a halo around your subject that is the color of the blue or green screen. This is caused by the lights in the scene bouncing some of the color of the screen back on to the subject. This extra bit of color on the subject is referred to as spill. Getting rid of spill is referred to as spill suppression. If you have any control over production, you can usually eliminate most spill with the clever placement of lighting on your subject, and by keeping your subject away from the blue or green screen.
And in many studios, this is just impossible. But if you are shooting in a large space and you have some say in the production, you might also want to insist on using daylight-balanced lighting (as opposed to the much warmer tungsten lights). From my experience, daylight-balanced lights really make the green screen pop, whereas tungsten lights shift the screen color more towards skin tones and diminish separation between subject and background. I’ve even experimented and had great success using green gels on greenscreen lights, but that only works if you have a really deep greenscreen and can put your talent in another zip code (or about 10 feet away). Of course, if a greenscreen shot calls for tungsten lighting, a DP will probably overrule you. But these are just a few extra tips to try out.
Keep a Clean Screen
It’s a really good idea to make sure that your blue or green screen is completely uniform. In other words, you want the entire screen to be the exact same shade of blue or green. Oftentimes, these screens are created with hanging pieces of large fabric. Like most fabrics, they wrinkle easily. Wrinkles cause variations in color, causing keying to become a more challenging process. Many green screen studios even use extra material to create rounded corners where the walls and floor meet. This is so that there are no shadows. Shadows and wrinkles (or any variations in color or luminance) in the blue or green screen add a great deal of headaches into any keying job. Uneven lighting can also create a non-uniform color on a background.
Of course, like many rules that pertain to how to shoot on set, you might not have any control over how this is shot. As is also the case with many of these guidelines, they are much easier to say than to do. I’m frank to admit that I’ve been VFX Supervisor on many sets, and it just wasn’t possible (for a variety of reasons) to set things up the proper way. Be prepared in post to be flexible.
Previewing Your Key
When I first started learning about keying in After Effects, I would import footage with a blue or green screen, and then practice removing the screen. The problem was that I usually practiced this without a layer underneath the layer I was keying. Using a black background is usually not the best way to see problems in your key. Instead, use a color that contrasts strongly with the color of the green screen so that you can easily see any problems. You can also enable the transparency grid at the bottom of the Composition panel to check your key. It’s a good idea to preview your key against a variety of backgrounds, just in case you missed some stray pixels that don’t belong.
Creating a Garbage Matte
Most of the time when keying a subject to remove a background, you don’t need the entire layer. So, the typical process is to create a loose mask, called a garbage matte, around the subject. It should be close enough to the subject to remove junk in the frame, such as boom mics and any other equipment in the shot, leaving only the blue or green screen. But it should not be so close that the subject moves outside of the matte at any point in time. Or, if they do move outside of the garbage matte, then you must animate the mask to follow the subject so that it is always completely encompasses the subject.
Let’s say that I wanted to key out the background, and isolate myself in the shot seen in Figure 10.3.
Figure 10.3
The shot I want to pull a key from.
My first step would be to create a mask that would get rid of all of the background stuff—the stuff I know that I don’t want in the final result. This makes it so much easier for the keying effect to remove the background. Even if the entire background was made up of only the green screen (without extra equipment showing), it still helps the keying process to create a garbage matte.
Figure 10.4
The same footage, with a garbage matte created. As you can see, it doesn’t demonstrate the highest level of sophisticated mask making, but it’s only supposed to be a rough outline like this.
Using Other Effects
We talked at the beginning of the book about how effects need to be used together to reach their potential, and we have an entire chapter on that very subject at the end of the book. But I just wanted to make a quick note about how especially important that is with the keying effects. With the plethora of controls that these effects have, you may be tempted to think that they stand alone and need no others. But often times, stubborn keys can only be perfected with the help of other effects, such as Channel Blur, the Color Correction effects, or the Matte effects (discussed in the next chapter) for example. You can also use multiple keying effects on the same layer, or duplicates of the same effect on the same layer.
Use the Highest Quality Footage Possible
It kinda goes without saying that you should always use the highest quality footage possible. But this is even more important when it comes to keying. Sometimes, when a video with a blue or green screen is shot perfectly, it can still be difficult— or impossible—to key well if it has been overly compressed. I know that in many cases, you probably don’t get to choose how the footage you work with is created. But if you have any say in the process, try to get as little noise, as little compression, and the largest color gamut possible in the creation of blue or green screen footage. And if possible avoid shooting chromakey footage with a DSLR like the plague. DSLR’s create beautiful footage, but it’s extremely compressed, making it a real challenge (read: annoyance) to pull a good key from.
Layer Your Keys
Humans are complex animals, speaking from a keying perspective. We have hard edges clothes and soft edged hair. We have areas of complete opacity and areas of semi-transparency. Most of the time, you’ll never get the “perfect” key by using one pass. No, it’s way better in most cases to get a general key where the core/body of the subject looks good, and then make a mask to isolate that. Then, duplicate that layer and invert the mask to focus on the edges of the character, especially if there is something semitransparent there, either with their hair or clothing. Depending on if your subject moves around a lot, you might need to do some rotoscoping, but this is a small price to pay for a great key.
The Color Difference Key Effect
If the concept of keying intimidates you, then you may want to skip ahead and come back later to the Color Difference Key effect. This effect works really well, and can even key out tough jobs (like smoke or wispy hair) better than most. But it can be a little daunting at first.
To follow along with me, open up the Color Difference Key.aep project from the Chapter 10 folder of the exercise files. For this project, we’ll be using a photo, rather than a video. This effect does much better with lightly-compressed or non-compressed video. And since those are more challenging to distribute, and we’re only using one frame anyway, we’re using a photo. Apply the Color Difference Key effect to the Chad Key layer in the Chad Key comp.
Figure 10.5
The parameters of the Color Difference Key effect.
I realize that these parameters look somewhat intimidating at first glance. They’re certainly more intimidating than the parameters of the other keying effects. But they make a lot more sense when you realize what the effect is doing.
Using Figure 10.5 as a guide, you’ll see that there are actually three main sets of properties here: Partial A, Partial B, and Matte. The Color Difference Key effect creates two mattes, and then combines them together to create the final alpha. Partial A is the first matte. Partial B is the second matte. And the Matte controls affect the final result (the alpha channel). Essentially, we have controls similar to those seen in the Levels effect for each of the three groups of properties.
To use this effect with this example, we first need to change the Key Color value, which is set to a default blue color. Click the eyedropper next to the Key Color swatch and then click on the green background of the photo. From my experience with this
Figure 10.6
The color difference key results. Notice how the subject is also partially erased, showing the magenta solid beneath it.
effect, the initial results are always terrible. But don’t worry about that. We’ll get this key near perfect in just a moment.
The next step to perfect our results is to adjust each of the two partial mattes individually. In this effect, you can do this in two ways. You can use the matte selector and eyedropper tools at the top of the effect in the Effect Controls panel, or you can use the regular controls at the bottom of the effect. We need practice at both, so we’ll use the controls at the top of the effect to adjust Partial A, and then the controls at the bottom to adjust Partial B.
First, select the A matte by clicking the A button underneath the layer thumbnail in the upper right hand corner of the effect. Now we’re going to adjust this matte by using the eyedroppers. The top eyedropper is the equivalent of the Key Color property, which we’ve already adjusted. The second (black) eyedropper is used to click on the areas of the matte that should be black (completely transparent). Partial A’s background looks pretty good, but just to be safe you can select the black eyedropper and click in the area of the image that has the green screen.
Next, you use the bottom (white) eyedropper to select areas that should be completely opaque, like your subject. So, I’m going to select the white eyedropper and click on my face somewhere.
From here, you could select the B underneath this thumbnail to adjust the B matte, or even the greek alpha symbol (looks like a little goldfish to me) to adjust the final alpha. But we’re going to go down and adjust the Partial B matte by using the regular slider parameters. Most of the properties in this effect refer to some form of black or white. Before getting into adjusting these properties, remember that black areas will be transparent in the final matte, and white areas will be opaque. So, if you’re wanting to remove some left over pixels caused by wrinkles in your green screen, adjust the parameters that refer to black. If you eat away at your subject too much, restore it by using the parameters that refer to white.
Tweaking the Color Difference Key Effect
Here are a couple tips when working with this effect. First, use the View drop down to see the results of individual mattes, or the final matte. Second, I usually get the best (and quickest) results from the Color Difference Key effect by getting a rough key using the thumbnails and eyedroppers at the top, and then by fine tuning my results with the regular parameters at the bottom.
In my key, I have too much of the background leftover, and I’m still partially invisible. So, to further remove the background, I’m going to increase the Partial B In Black value. This is like dragging the far left arrow under the histogram in the Levels effect to the right. I then want to restore some of the pixels making up the subject. I can do this by reducing the Partial B In White value. This is the equivalent of dragging the far right arrow under the histogram in the Levels effect to the left.
To double check your final matte, it’s always a good idea (in all keying effects and applications) to view the matte in grayscale. This way, you can easily tell exactly how well you did at keying. In most instances, you will want a pure black background (which indicates complete transparency), a pure white subject (white indicates full opacity), and gray along the edges (which indicates partial transparency). In our matte here, we can see that we have a little bit of gray on the fringes of my hair, which is great. This will help those fringes of hair composite better into their new background. In the Color Difference Key effect, you can view this grayscale matte by changing the View value to Matte Corrected.
Figure 10.7
The matte corrected option in the view drop down shows us our key as grayscale values. These values indicate the degree of transparency in each pixel.
Things are looking pretty good here, so let’s now take the View drop down to Final Output to see the result of our key. You might notice that we’re still getting a little bit of green in the feathered areas of my hair. Sometimes, it’s just impossible to keep semi-transparent edges (which is usually desirable) from picking up some of the colors of the background. We could adjust the Partial A and Partial B In White values to remove the green screen from these hairs, but that would also create hard edges, which would be bad for compositing. To keep the semi-transparency in the edges and remove the color, use the Spill Suppressor effect, discussed later in this chapter.
Figure 10.8
The final result of our key with the Color Difference Key effect.
The Color Key Effect
The Color Key effect is probably the polar opposite of the Color Difference Key effect. The Color Key effect is extremely simple and straightforward, but the results are usually, well … bad. Even with really clean footage that has subjects with solid edges, the Color Key effect has a tough time.
Figure 10.9
The Artbeats footage of leaves falling.
I’m going to import the Artbeats video clip UM230.mov from the Artbeats folder in the Media folder of the exercise files. This is professionally shot footage of autumn leaves falling. Low noise, good lighting, perfect green screen—the whole bit.
You would think that this would be easy footage to key, right? Well, not for the Color Key effect. Let’s see how the results turn out. Click the eyedropper next to the Key Color swatch and click on the green screen in the footage in the Composition panel. Only a handful of pixels are removed, so increase the Color Tolerance value to remove more of the green screen background. Chances are, the results will look awful no matter how much you play with this setting. You can use Edge Feather to soften the edges. You can also use Edge Thin to fix the edges, but be careful here. Increasing the value just a little bit will eat away at the edges of the subject. Taking this to a negative value will expand the edges of the subject. It’s best to use this property sparingly, if at all.
All in all, it’s almost impossible to pull a good key with this effect. Even if you manage to get a frame looking good, usually after previewing the video, the edges of the subject will “dance” around, looking jittery. I would only use this effect if a client was standing over my shoulder, or in some other instance where I needed an instant key and quality was not important. Even then, I would probably rather choose the Linear Color Key effect, discussed later in this chapter.
Figure 10.10
The Color Key effect usually leaves a little bit of the green screen around the edge of the subject (as shown here), or erodes its edges.
The Color Range Effect
For those of you familiar with Photoshop, the Color Range effect works very much like the Color Range selection feature in Photoshop. The big difference here is that you first have to select a color space from the Color Space drop down—either Lab, YUV, or RGB. If you don’t get good results in one color space, you can try pulling a key in a different one. But it’s very important to do this first because changing the Color Space will completely alter your key, and you will have to start all over again.
Once you’ve chosen a color space, click the eyedropper tool at the top of the Color Range effect in the Effect Controls panel, and then click the color you want to key out in the Preview area, or in the Composition panel. Invariably, this will leave large amounts of the key color left behind. So, then select the Plus eye-dropper to select additional colors to add to the key. If you accidentally select colors that you don’t want to remove, you can click on them with the Minus eyedropper to remove them.
As with Color Range in Photoshop, increasing the Fuzziness value is like increasing a Tolerance value; more colors will be selected. The Fuzziness value usually needs to be increased quite a bit. You can then tweak the levels of individual color channels with the Min and Max properties. Note that the cryptic letters (L, Y, R) refer to the first channel of Lab, YUV, and RGB, respectively. The properties that contain (A, U, G) refer to the second channel of Lab, YUV, and RGB, respectively. After increasing the Fuzziness value, our key of the Artbeats autumn leaves footage is looking pretty good, especially considering the lack of help from other effects. Also note that the edges of these leaves are smooth when played back.
Figure 10.11
After pulling a key with the Color Range effect.
The Difference Matte Effect
The Difference Matte effect is a really great idea—in theory. But in practice, it just doesn’t always work out that great. But perhaps you’ll have better luck with it than I have. The purpose of the effect is to use two layers that are otherwise identical, and isolate the part(s) of them that are different. Let’s say you had a video of a dancer dancing around. And you also had a segment of the video (or perhaps a still camera shot) of the same spot where the dancer was dancing, but without the dancer. In theory, you could use the Difference Matte effect to compare the footage with the dancer, and the footage without the dancer, and then extract the dancer.
Let’s look at an example of this effect in action. If you’d like to follow along, open the Difference Matte.aep project in the Chapter 10 folder. I have here some footage of my buddy Paavo from malachyte.net doing some crazy unicycle work.
I also have a still (actually an extracted frame from the same video shot), that contains just the benches.
The idea is that we can isolate Paavo using the Difference Matte effect because we have these two pieces of footage. But we’ll quickly
Figure 10.13
The same shot from Figure 10.12, but without the unicyclist.
see why this effect often falls apart. Apply the Difference Matte effect to the unicycle on bench.mov layer (the top layer in the bench comp). By default, the effect uses the layer it was applied to as the Difference Layer, which completely hides the entire layer. Change the Difference Layer drop down to the Benches alone PRECOMP layer. Selecting more pixels by increasing the Matching Tolerance value. Feather the edges with the Matching Softness value. The problem is that this shot (as with most shots) has a lot of small, moving parts—shadows, moving sunlight, leaves blowing in the wind, and so on. All of these things cause problems in the matte. The final results are less than stellar, but could be used for a cool, stylized effect.
The Extract Effect
Before tackling the Extract effect, you might want to brush up on the Levels effect, covered in Chapter 6. The Extract effect works best (works only?) with footage that has extreme light values or extreme dark values to key out. If the luminance of your subject is similar to the luminance of your background, you’ll have a difficult time getting the Extract effect to work well. If you’d like to follow along, you can open the Extract.aep project in the Chapter 10 folder.
So, we have here a photo of some fireworks, courtesy of Angela McInroe.
Figure 10.15
The fireworks comp in the Extract.aep project.
Apply the Extract effect to the fireworks layer. There is a histogram with a graphic interface, but I much prefer to use the regular sliders/hot text adjusters with this effect. With the Channel drop down set to Luminance, increasing the Black Point value will remove dark areas of the layer. Increasing the Black Softness value will soften the edges of removed dark pixels. Decreasing the White Point value will get rid of bright values in the layer, while White Softness will soften the edges of removed bright pixels. If you’re interested in using the graphic interface (the bar underneath the histogram), the top points represent the Black Point and White Point, while the bottom points indicate the Black Softness and White Softness values.
Note that you can also change the Channel drop down to things like the blue channel or the green channel, but that doesn’t help with this effect unless the blue or green screen is exceedingly dark or light.
The results of the Extract effect are similar to what you would get if you simply used a blend mode like Multiply or Screen, but it does allow you more control over what gets removed and softness.
Figure 10.16
After removing the dark pixels of the fireworks layer with the extract effect, you can see the rad rockets. mov layer behind it.
The Inner/Outer Key Effect
Many of the effects in this chapter aren’t really helpful all that much. And if they are helpful, they usually aren’t very interesting. But the Inner/Outer Key effect is both helpful and interesting, and it’s helpful for far more than just blue and green screen work.
You’ll probably want to follow along with this one, especially seeing how different it is from the other effects in this chapter. I’ll be using the Inner Outer Key.aep project from the Chapter 10 folder of the exercise files. This project contains a comp that has in it some footage of Seattle. The difference here is that I have created two masks. You see, what we want to do here is to isolate the city by removing the sky and mountains in the background. The Inner/Outer Key effect allows us to use these two masks to help us in this otherwise challenging task.
Figure 10.17
The Seattle footage with two masks applied.
The Inner/Outer Key effect functions similarly to the Extract plugin in Photoshop. You first create two masks on your layer—one to specify areas which should absolutely completely opaque, and then another mask that tells the Inner/Outer Key effect where the border is between the stuff we want and the stuff we don’t want. So, in this case, we have one mask around the city, and there’s nothing in that mask that we don’t want to keep. With the other mask, we encompass everything we want to keep, but we also encompass some of the skyline. This mask covers ALL of the stuff we want (the city), which includes some of the stuff we don’t want (the sky and background).
More Than Two Masks
The Inner/Outer Key effect actually allows you to use several masks if you need to. The additional masks can be chosen in the Additional Foreground and Additional Background areas. These extra masks can be used to cut holes in the masks, if needed.
Apply the Inner/Outer Key effect to the Seattle layer. Instantly, it uses the mask of the city (Mask 1) as the Foreground (Inside) mask, and the mask of the city, with its edges and some of the sky (Mask 2) as the Background (Outside) mask. Even the default settings remove the background fairly well. If you were to use another keying effect (such as Extract or Luma Key), the white pyramid in the foreground (which, coincidentally, is actually called the Key Arena) would also have been removed.
Figure 10.18
After applying the Inner/Outer Key effect, the background is instantly removed, even before making adjustments to settings.
There are the standard keying tweaks here, such as Edge Thin, Edge Feather, and Edge Threshold. But there is also an Invert Extraction checkbox. Clicking that checkbox in this instance would remove the city, and keep the background. Remember this as you use this effect in the real world, in case you find an instance where it’s easier to mask out a background than a subject.
Figure 10.19
Selecting the Invert Extraction checkbox will mask out the subject, leaving the background. This also allows you to see the imperfections in my initial matte.
Another thing that makes this effect unique is that you can actually use masks to restore lost foreground or remove more background. And it does this by creating a stroke around the mask. I know—weird, right? To show you how this works, I’m going to create yet another mask—a rectangular one this time— on this layer. I’ll then take the Mode value for this mask to None in the Timeline panel so that it doesn’t affect the layer directly. To restore more of the subject (i.e., foreground), open the Cleanup Foreground area and select Mask 3 from the Path drop down in the Cleanup 1 area.
Using Open Masks
If there’s one tiny little spot that’s bothering you, an entire closed path that has a stroke around all edges might be overkill. You can also use open paths for cleaning up our matte in the Inner/Outer Key effect, which are great for those tiny problems.
Figure 10.20
Choose the mask that you created (Mask 3, in my case) from cleanup foreground>cleanup 1>path.
Because we chose this mask in the Cleanup Foreground area, it will cleanup, or restore, pixels. Again, this comes in the form of a stroke around the mask. You can control the size of the stroke with the Brush Radius value, or the Hardness of the stroke with the Brush Pressure value.
Figure 10.21
Using a mask to cleanup the foreground, or in other words, to restore pixels.
If we wanted to remove more of the pixels in this matte, we would go into the Cleanup Background area, and go through the same process there. Figure 10.22 shows the same mask as seen in Figure 10.21, but used to cleanup the background. The masks have been deactivated in this screenshot so that the removed pixels will be more obvious.
And so, in summary, the Inner/Outer Key effect can seem like Grand Central Station for masks. It allows you to input up to 36 simultaneous masks for various purposes (yes, I counted). But whether you use this effect to its fullest, or on a more simplistic level, the Inner/Outer Key effect can easily extract portions of a layer that other keying tools would have a difficult time with.
The Keylight (1.2) Effect
Keylight is a keying tool created by The Foundry and has been included as a bonus with After Effects for the last several versions. It used to retail for $1000 as a standalone plugin, if that’s any gauge of its quality.
Keylight is designed to remove blue and green screen backgrounds, and it also has pretty decent (and automatic) color correction tools that helps footage get all the green or blue off.
Let’s look at some of the basic tools in Keylight. I have some footage of a model sneaking some footage with an 8mm camera. I’ve already created a simple garbage matte.
Figure 10.23
The original layer.
This is 4k raw footage, so it’s extremely flat. So, before I apply Keylight, I want to get this footage to a good base level. I also want to make sure that the green screen actually looks green. Keylight really responds to saturation, so it might be a good idea to apply Hue/Saturation, set the Channel Control to Greens and boost as needed/desired.
In the Keylight.aep project in the Chapter 10 folder of the exercise files, you’ll find the Keylight START comp. The vintage camera.R3D file is what we’re going to be keying. To get us to a good starting point, I’ve applied Levels and Hue/Saturation already.
Figure 10.24
The original layer after doing some basic color correction and enhancing the green in the background.
Now apply the Keylight effect to this layer. The very first thing we want to do is to choose the Screen Color, which is the color (or “color” if that’s the way you do things) of the background. So click the Screen Color eyedropper and click on the green background of this footage.
Figure 10.25
The status view is a better representation of what’s happening with your key.
When you first select a Screen Color, Keylight jumps into action and creates an initial key, but also tries to do some color correction to remove spill. Because this is such a radical shift, there is a temptation among new users to be satisfied with the initial results. Don’t fall into that trap. Change the View drop down from Final Result to Status to see what’s really happening here.
Color Picking Tips!
Choosing the best color is absolutely essential to getting good results with Keylight. So, there are a couple of tips here. First, if you hold down Alt(Win)/Opt(Mac) while moving your eyedropper around, Keylight gives you a real time preview of what the initial results of that key will be. You aren’t looking for perfection here, just the best results. This is also helpful for avoiding a stray pixel (such as those resulting from noise) and getting a wacky result. Also, if you choose a color and decide it’s not the best, do not just select the eyedropper and try to choose another. This won’t yield good results for several reasons. What you’ll want to do first is to take the View Drop down to Source and then choose a new color. Alternatively, you could also create a new viewer in After Effects that had this footage in the Layer panel, so you always have it around.
The white areas are the areas that are going to be completely opaque. The black areas are completely transparent. The gray areas represent pixels that will be semi-transparent. Our initial results here are actually pretty good (better than usual). But we still have some work to do.
The end goal is to have the core of our subject completely white, the background completely black, and some of the edges and softer details (like hair and reflections) gray.
Let’s first start out by making the background black. There are two contorls to do this: Screen Gain (at the top) and Clip Black (inside the Screen Matte group). The Foundry recommends staying away from Screen Gain because overdoing it can ruin the delicate edges of your footage, which is a critical aspect of a good key. However, I’ve gotten great results by using the Screen Gain parameter that I couldn’t get with Clip Black. So use it to taste.
To get rid of the holes in your subject, you’ll want to decrease the Clip White value in the Screen Matte section. And you’ll want to adjust all of these settings with the intent to preserve a little gray on the edges and the softer portions of the image.
As a general rule, with all of these settings, there’s never a formula for getting a good key. Unless you’re shooting with the same camera, same lights, and the same talent, in the same position, and in the same studio, don’t worry about creating Animation Presets or writing down settings. They’ll always be different. And I’ve found that there are some controls that mess up most of my footage, and then one day, that parameter saves my life when nothing else could. So it’s a balancing act, rather than a recipe that we’re going for here.
Another general rule is that you only want to use the adjustments in the Keylight effect as much as you absolutely need to. With every adjustment, there is a trade-off in the final quality of the key. So be temperate, and don’t push things to white or black (or soften the key or contract the key, etc.) any further than you absolutely have to.
So here are the settings I used in this case. To make the background black, I increased Screen Gain to 108 and Clip Black to 25. I didn’t find that I needed any Clip White adjustment, which is actually quite rare. It’s great if you can get away with that, but I find that I usually have to take this value to about 80 to make sure that my subject is intact.
That got my black and white where it needed to be, but when I took the view drop down back to Final Result, the edge is ugly. It’s too rough and there’s some semi-transparency on the top of her head that doesn’t look good.
So let’s look at some other settings to help this out. I took the Screen Pre-blur value to 2. This is great for smoothing keys in many cases. Note that this doesn’t actually blur anything that you see. This is just a behind-the-scenes blur that Keylight applies to the key before it does its magic. This can help to get rid of noise, which can cause noisy or “sizzling” edges when the footage is played back.
In the Screen Matte section, I also contracted the matte a bit by taking Screen Shrink/Grow to –3.5. That’s a lot more than I usually use, but it works in this instance. Note that a positive value here will expand the matte, while a negative value like this will shrink it. This is starting to look good.
At this point, I think I’m going to turn on the background layer (IMG_3452.CR2) to see how our composite is coming along. Looks all right, but her edge is really strong, which is a dead giveaway of a composite. So let’s soften that. I’m going to take Screen Matte>Screen Softness to 2. Be careful not to take it too much further than that, as an edge that is too soft is just as much of a dead giveaway of fakery as an edge that’s too hard.
Figure 10.26
After fiddling with keylight and enabling our background layer, we have a fairly decent composite.
The thing that’s missing here is a little extra color correction to our subject. She’s a bit too dark and red (and a little too blue) and saturated for this background. The Keylight FINAL comp has my final color correction and composite. I didn’t spend too much time with it, but we’re more in the ballpark.
Figure 10.27
The final composite in the keylight FINAL comp.
One other thing about Keylight that I wanted to mention was its automatic color correction. This is usually an amazing feat of wonder and miracles. But sometimes, it causes more problems than it cures. I was once working on a project and couldn’t figure out why my footage was so terribly noisy, when it looked great before I brought it into After Effects. The problem was that Keylight was adding tons of noise as a result of its color correction. If this is happening to you, or for whatever other reason, you just want to turn off the Keylight color correction, take the View drop down from Final Result to Intermediate Result.
My Favorite Keying Tool(s)
For what it’s worth, my favorite keying tool of all time is the Primatte Keyer from Red Giant Software. I find that it usually gets me a better key faster than Keylight, and it’s very intuitive. It also comes in a suite of tools called the Keying Suite that has many tools for refining keys and matching background color for better composites. It’s an extra purchase, but I wouldn’t be able to survive without it.
The Linear Color Key Effect
The Linear Color Key effect is a quick and simple keying tool. It’s slightly more complex than the simple Color Key, but the results are usually much better. For this example, I’m going to be using the Linear Color Key.aep project from the Chapter 10 folder. This contains a video clip of me looking like a total doofus in front of a green screen. As much as this footage humiliates me, it’s also extremely compressed. If you can pull a good key with this footage, you can probably pull a good key with most footage.
First, let’s apply the Linear Color Key effect to the Chad with Hand Dot.mov layer. These settings should look familiar to you if you’ve been working with the other effects in this chapter, so I won’t mention them again here. The two notable differences are that you can choose to match color using RGB, Hue, or Chroma (color—which refers to all attributes of color, not just the hue). You can also choose to adjust the Key Operation value to Key Colors (remove them) or Keep Colors. This property is helpful if the Linear Color Key effect accidentally removes too much of a color in your subject, you can apply another instance of the Linear Color Key effect and keep (or restore) the missing colors.
Click on the Key Color eyedropper and select the green in the green screen that is closest to our subject, as there is some darker vignetting happening in the corners of this shot. I found the best results with this clip with my Match Colors drop down set to Using Chroma. I also used a Matching Tolerance of 5%, and a Matching Softness value of 4%. The results came out pretty good, especially considering how badly this footage is compressed, and the fact that I’ve only applied this effect once, and I’m not using a Spill Suppressor or any other effect here. Remember that for the best results, it’s best to constantly change the view to the grayscale matte so that you know exactly how transparent your pixels are.
Figure 10.28
The quite impressive (and quick) results with the Linear Color Key effect.
The Luma Key Effect
The Luma Key effect is similar to the Extract effect in that it attempts to pull a key by looking at a layer’s brightness, and keying out either light or dark values. The Luma Key effect doesn’t give you as much control (as the Extract effect also allows you to adjust individual color channels), but it is easier to use, and the results are comparable.
For this look at the Luma Key effect, we’ll be using the Luma Key.aep project from the Chapter 10 folder, which is identical to the project we looked at with the Extract effect. That way, you can compare and contrast the results with what we saw of the Extract effect a little earlier.
Apply the Luma Key effect to the fireworks layer. The default Key Type is Key Out Darker, which removes dark pixels. You can change this value to Key Out Brighter to eliminate bright areas. To remove any pixels, you need to increase the Threshold value. The Tolerance parameter functions similarly to Threshold, but only works when Key Out Similar or Key Out Dissimilar is chosen as the Key Type. When Key Out Similar or Key Out Dissimilar is chosen as the Key Type, you can use both the Tolerance and the Threshold properties simultaneously to key out a wider range of tonal values. Key Out Similar is very much like Key Out Darker, and Key Out Dissimilar is very much like Key Out Brighter. The other properties (Edge Thin and Edge Feather) have been covered already in this chapter.
Figure 10.29
The results of keying with the Luma Key effect. These results were achieved with all properties at their default values, except for threshold, which was at 60.
The Spill Suppressor Effect
As we’ve been playing around with keying effects in this chapter, you might have noticed that some keying effects leave a small trace of green around the edges of the subject. Sometimes, there’s just a green reflection from the screen on your subject. The purpose of the Spill Suppressor effect is to remove the colors of the background that are left around the edges of your subject. It is the only effect in this category that doesn’t remove pixels; it only desaturates them.
For this example, open the Spill Suppressor.aep project from the Chapter 10 folder. This contains a more realistic use of some of the footage we’ve been keying. We have two clips from Artbeats—one with falling autumn leaves against a green screen, and the other depicting running children. We’re going to composite these leaves over the footage of the children, to make it look like these leaves were actually in the shot with the children.
In this project, I’ve already keyed out the green screen using the Color Key effect. Because I’m using the Color Key effect, you know that the results aren’t going to be spectacular. Notice how the green outline around the leaves makes this look especially fake.
Figure 10.30
The comp in the Spill Suppressor project, with a bad key courtesy of the Color Key effect.
Figure 10.31
The final keying result, after removing the green from the edges with the Spill Suppressor.
Now, add the Spill Suppressor effect to the UM230.mov layer (after the Color Key effect). The first order of business is to inform the Spill Suppressor effect which color needs to be desaturated. You could turn off the visibility of the Color Key effect to see the green screen again, and then select it with the Color to Suppress color swatch eyedropper. Or if you have good aim, you could just click the Color to Suppress color swatch eyedropper, and then click the green Key Color color swatch in the Color Key effect. Then, you don’t have to fiddle with effect visibility.
Then, just increase the Suppression value until the green edges of the leaves are gone. For better results, take the Color Accuracy value from Faster to Better. The Spill Suppressor effect will desaturate the color in the Color to Suppress value, making the final result much more believable.