|
|
|
Tips and Tricks for Optimizing the 3D Compositing Process
|
By Johnathan Banta, May 1, 2001
Welcome to the digital future of visual effects. For those of you producing the next Star Wars or Jurassic Park in your backyards on shoestring budgets, I'd like to point out that some of us were doing much the same thing with decidedly more primitive tools-clay and Super 8mm movie cameras. Clay and Super8 weren't exactly state-of-the-art, nor were they easy to work with. And integrating my miniature clay animation with live action was nearly impossible, so I consider today the "good old days."
But I do miss the one advantage of clay puppets: They are real physical objects that come ready-made with physical properties. They cast shadows, they have color and texture, and you can reach out and move them without having to go near a mouse or graphics tablet. By comparison, computer-generated (CG) objects require nearly every aspect to be specified and built in digital bits. So when these CG elements get composited into live-action footage, the digital bits tend to look like digital bits if they aren't prepared properly. The illusion that they are real physical objects doesn't come across to the audience. The following tips and tricks will help your CG 3D images look like they belong in the same reality as the live-action footage they're composited into.
Good compositing requires good planning. Loosely defined, a composite is a collage of different elements that are layered on top of each other to form a single image. Every visual effect is a composite, regardless of how it is produced. Whether you assemble the image in a layered 3D scene or in a compositing program designed for mixing 2D images, every element is an addition to the final image. Compositing is the fundamental process of visual effects.
Your visual plan should include procedures that exploit the strengths of 2D and 3D. Considerable amounts of 3D work can be produced in the 2D realm, so you don't need to waste countless hours making your 3D elements perfect. In some cases, decisions normally relegated to 3D can be put off until later. Conversely, greater attention to certain areas of 3D can quickly facilitate a realistic composite.
There are a lot of things to keep in mind when you're executing a clear visual plan, and an exhaustive analysis of the approach would be ponderous, so I'll go over a list of guidelines and procedures that I hope will quickly get you 90 percent of the way to a solution.
The 3D process is nothing without good models, and if you're matching a real-world object, it's always useful to gather as much photographic reference material as you can-especially reference materials that document how an object reacts to different lighting conditions. Without such references, you'll be left guessing about such things as detail, reflective properties, and even shape. Many times getting photos isn't possible, but try to include them in your planning because what you think an object looks like is not always what an object actually looks like.
Keep in mind that it's not always necessary to model everything perfectly. Take your final output resolution into account and note how close to the camera the final object will be rendered. The closer the model is, the more detail it will require.
Game developers have experience working with limited polygon counts imposed by realtime game engines, which require them to reduce the complexity of their models' geometry to improve playback speed. To compensate, they apply robust textures that supply details missing in the actual geometry. I'm not suggesting that all of your 3D objects should be low-rez, but they should have the required resolution and no more.
Here's a prime example: The Statue of Liberty was recently featured in a film, and one visual effects company built an exhaustive model of the statue with NURBS geometry. It included detail of every rivet and band on the statue. When it came time to render the image for film, they threw away a significant amount of modeled detail and improved the texture mapping to get render times fast enough to complete the project by the very tight deadline. Sometimes building every rivet on an object is nothing more than a great exercise in building rivets.
You can't always anticipate the final proximity of your 3D model to the camera-which was the case in the aforementioned movie-because the scene's demands may change. In such cases, you're probably better off starting with more detail and paring back where possible. But be cautious. Whether you're building realtime games or high-end visual effects, the same rule applies: More geometry equals more rendering time.
Also consider that modeling an entire environment is not always in your best interest. Anything that is outside the camera's field of view (FOV), and will never be seen, isn't something you need to waste time modeling. Having a clear visual plan from the outset is of paramount importance to optimizing the process. There's no sense in wasting time and money building things that don't need to be built.
Your 3D objects must seem to exist in the real world and obey the laws of perspective that govern your live footage. Therefore it's important to match the perspective of your camera and the lens FOV-especially when you're creating virtual environments for live-action insertion. Without matching these parameters, realistic integration is nearly impossible. (Be aware that certain lenses change their FOV properties as they rack focus, so good records on lens performance are very helpful. For a chart comparing various lenses' focal lengths, see www.gregssandbox.com/gtech/filmfacts/focallen.htm)
Try to record as much information as you can about the on-set camera setup (a digital camera is useful for taking stills of the camera setup just before filming). Good information is not always available because of the pressures of filming on the set, or because conditions didn't allow or require it.
If you don't have the information when it's your turn to work on the scene, you'll be left guessing. Sometimes you just have to suffer without this information, and repeatedly test your choices.
Automated 2D tracking is available in many software packages, and there are also many commercial 3D tracking packages available, but they're not always successful in deriving a viable 3D scene file.
Major effects facilities have entire departments dedicated to nothing but matchmoving, and much is done by hand. An automated solution isn't always possible, so manual tracking is something you will eventually have to face.
Let's explore how to control certain aspects of your 3D scene within the compositing package and go over a few guidelines for integrating the final 3D image into the shot.
What is multilayer rendering? Try to visualize your 3D scene in 2D layers, rather than visualize it in its fully dimensional form. Separate the scene into planes of action that can be easily isolated and modified as necessary. The background action can be rendered out separately from the midground, foreground, and so on
. Further separate effects elements such as lasers, engines, headlights, explosions, sun flares, and the like.
By separating layers of action or effects, you have more options for post-processing them in the compositing package, which lets you concentrate on fixing problem areas without the overhead of all of the objects that are already successful.
What's particularly beneficial about multilayer rendering is that it increases efficiency and permits you to create effects that are generally difficult in 3D. You can add noise, defocus certain layers for a simple depth of field, or motion blur certain sections.
All of the 2D tricks are now available in the depth of your 3D layered action, giving you maximum flexibility when you're asked to make changes.
Changes from the client come down quickly, and the ability to react to them in the compositing process, while running up against the deadline, can be what makes or breaks the project.
The next technique we'll explore is called multipass rendering. It's an extension of the last procedure, but it involves isolating specific properties (diffuse, specular, shadows, and so on) of a 3D scene or object. It allows the ultimate amount of control in materials management and eliminates rendering thousands of test frames to get the exact mix of reflection, specular, and diffuse color that you need.
Why spend immeasurable amounts of time rendering test frames when you can mix properties in near realtime within a compositing package?
To execute multipass rendering, render out each property of a desired object by itself-one pass each for diffuse, specular, reflection, luminance, and so on. For diffuse color, increase the ambient amount to full strengthso that all you get is the texturing information and no lighting. You should render each property at full strength, turning off all other properties (but remember to keep the bump calculations on so that light interacts properly).
The lighting can be rendered separately on a perfectly white object, and specular and reflection rendered over a perfectly black surface. For further control, render out certain texture maps or objects as black-and-white images to act as mattes, which will be useful when you need to isolate skin tones on your render for that oh-so-perfect match.
In the compositing stage, you can mix all of the separated properties together and adjust their overall contributions to the final image by varying opacity.
Why limit your mixing to properties alone? What about the lighting of your 3D objects? With proper planning, lighting control for any 3D object can be brought directly into the 2D compositing package to allow maximum flexibility on the 3D process all the way through the pipeline.
All of the procedures for rendering out diffuse and reflection passes still apply, but specular and lighting pass procedures will change. The major difference is that you will be rendering out saturated colored light rather than standard white lights.
Start by treating every three lights as a group when it's time to render. Color one light pure red, another pure blue, and another pure green. Then render out the lighting and specular passes separately as before at full strength. Repeat this procedure for each of the three lights and for their specular passes (if you want more control).
What that process does is to isolate each light into a specific color channel within a single image, improving disk storage. The red, green, or blue channel can be pulled out of the full-color rendering, yielding a grayscale lighting intensity.
Those channels can be laid down at different opacities over the other properties already mixed. In this manner, you can vary lighting as per client request or create multiple variations for evaluation purposes-all without having to tweak and rerender your 3D project many times. You can blur the sources, add noise, and change color and contrast levels to get the desired effect. Very nifty.
Layering images in a compositing program is straightforward. Most DV readers are familiar with using alpha channels, but when it comes to mixing the multipass techniques, using alpha channels falls short. Sometimes an effect needs to darken, brighten, or somehow modify the elements beneath it. The solution for this is transfer modes, and they are slowly standardizing across packages and platforms. Many compositing programs use the standard Photoshop-style transfer modes (Discreet packages just added them), so it's likely that your compositing app supports them.
Using transfer modes in multipass mixing is relatively simple. Use the transfer mode that best suits the type of element. Specular and reflection passes work best when added down, and lighting passes use multiply-most of the time. Each situation can vary, so use the mode best suited to the image you're creating. Here's a general procedure for mixing multipass elements.
Typically the diffuse pass is laid down first and then each successive pass-such as specular, reflection, and luminance-is added (or screened on) afterward. Because these are rendered out at full intensity, varying opacity is the best way to control an attribute's contribution to the final image.
If multilighting passes were rendered out, the diffuse channel should be rendered out fully luminant so that it contains no lighting (see the earlier notes about multipass rendering). All lighting should be added together first in a precomposite, and then laid on as a group over the diffuse pass. The lighting precomp is then multiplied down so that it darkens the diffuse surface. The other passes, such as reflection and so on, are then added over this one.
I can't stress color matching enough. The color value of black in a background plate is as black as your CG image should go. If it is a little blue, then shift the black levels of your CG to match. Adjust the levels of your element to match white and midtones as well. If they are wrong, the CG will immediately call attention to itself.
Is there any type of grain moving in the original footage? Try to replicate this on each one of your CG objects with randomized noise.
In all of this work, you need to be familiar with seeing. Take time to look at the frames in the footage or in the rendering that you're matching. Observe closely and do your best to match the pixels of your rendering to the pixels in the image.
As a 3D and compositing artist, it's your job to ensure that the final CG elements are properly integrated and correctly color balanced to match the original footage.
Whether you separately render out different properties of the CG object or merely layer separate action and effects, proper planning in 3D saves you a lot of time dealing with the inevitable changes that will come down the pipeline. If you base at least some element, whether it's textures or models, on real objects, a sense of that reality may just sneak through to the final composite.
Every visual effects shot is a composite, and how professionally and consistently you assemble it is what feeds you the next job and delivers the greatest satisfaction.
Johnathan Banta is lead 3D animator/compositor at Sassoon Film Design in Santa Monica, CA; director of Agrapha Productions; and the programmer of AG_Shaders for Electric Image.
|
|
|
|
|
|