In the world of professional video production, the HDV format has gone from novelty to viability in record time. The rapid changes began over a year ago when Sony introduced the HVR-FX1, and DV Contributing Editor Adam Wilt pronounced its compressed, MPEG-2 picture to be "better than it has any right to be." The release of the Sony HVR-Z1U upped the professional-quality ante, and HDV cameras from Canon, JVC, and Panasonic have streamed into the market. With slick editing solutions available from all of the major players-Apple, Avid, Canopus, CineForm, Lumiere HD, and others-the HDV format has arrived in a big way.

Despite the sales success and broad acceptance of HDV, there is still an information gap in terms of how well it really performs. Although a lot of HDV units have been sold, a large number of DV readers are still on the fence about whether to buy into the format or not.

So I decided to put the latest HDV techniques and technology to the test. I produced a low-budget short from beginning to end in HDV, with the intent of creating the best possible product in standard definition for DVD distribution, HD for digital broadcast, and, because the promise of low-budget filmmaking is driving the intense interest in HDV, I decided to take the piece all the way to a 35 mm film blowup.

The end result is here: An overview of the decisions and discoveries I made while producing a real-world project in HDV, along with the reactions of some seasoned DPs to the final film result. I found out a lot during the process-more than can fit in the pages of the magazine. More details about the project are available online at DV.com.

Early decisions: Camera, frame rate, and gamma

Armed with the script for Windsor Knot (a historical period drama about the abdication of England's King Edward VIII that I've been wanting to shoot for some time), I set out to choose a camera. At the time I was shooting (May 2005), I felt that there was only one viable camera for the project-the Sony HVR-Z1U (Reviews, May '05 DV). The Z1 has several appealing features, the most important of which are the abilities to use pro audio connections and shoot in 1080i-50. Even though creating a 24 fps version for the film blowup was critical, I found that creating true 24p was easier and more effective in my tests starting with 50i than with 60i (more about this reasoning can be found online at DV.com).

I didn't consider Sony's CineFrame mode, which, though it creates a 24 fps video file, has unaccept-able motion artifacts and image degradation (for more about CineFrame, see Adam Wilt's analysis at www.adamwilt.com/ HDV/cineframe.html).

I experimented with the Z1U's CineGamma settings in an effort to achieve a film-style gamma curve. The Z1U has two CineGamma settings, with CineGamma 2 being more pronounced. I liked the look of CineGamma 2 and decided to use that mode in shooting. On inexpensive cameras, these film-like gamma settings are accomplished not with the true S-curve that I would apply in post, but rather by tilting the straight gamma curve to a more extreme position Although this accomplishes almost the same thing as the gentle S-curve, it's not quite the same. It doesn't supply the more gentle roll-off of highlights and tends to crush dark grays to black sooner.

Click here to see a larger image Video has a straight gamma curve. Film stocks often have a gamma curve shaped like a shallow S. (right) Inexpensive cameras with "cine" gamma adjustments usually simulate this by tipping the straight video gamma.

After testing color tweaks in the camera menus, I left most of the settings on factory defaults. I turned the sharpness control down considerably, though not to its lowest setting, which is a tweak I recommend for nearly all cameras if you want a film-like finish. Reducing the artificial sharpening (technically known as aperture correction) tones down harsh edges and avoids compression artifacts.

On location: Decisions and discoveries

I was dealing with a microbudget for this project (well under $3,000 for all expenses and salaries), so I scheduled a single day of shooting on location.
The day came, we showed up with a truckload of equipment and people, and Windsor Knot was in production.

The decision to shoot in 50i necessitated a PAL-compatible monitoring solution. I brought the JVC BYH13Y production monitor from my edit suite.
We monitored composition, exposure, and color on SD via the down-converted Y/C output of the camera. But we discovered that neither an SD monitor nor the camera's viewfinder, nor its LCD screen were up to the job of representing HDV's details-we ended up with slightly soft focus no matter what we used. To deal with this issue, Sony provides a zoom function on the viewfinder, as well as a peaking function, which flags high frequencies (such as in-focus sharp edges) with an accent color (red, yellow, or white). This works well to visually cue the camera operator when focus is right on. Unfortunately, peaking can't be used at the same time as zebra display, so the cameraman had to keep switching between the modes.

The HVR-Z1U also has a basic zebra display that defaults to 100 IRE. We set the zebra at 90 IRE to avoid the "video look" of overexposure and clipped whites. I think clipping in HDV is really nasty, even worse than in DV. So an essential rule for shooting in HDV is don't overexpose! It's much better to be a stop under rather than a stop over while shooting. If necessary, you can stretch the contrast back out in post.

Postproduction: Quality control

If I was going to edit on a Mac with Final Cut Pro, my logical choice would have been Lumiere HD, a plug-in that provides machine control and on-the-fly transcoding to the editing codec of your choice. But my shop is PC based, so the strongest contender was CineForm's Aspect HD plug-in (Reviews, July '05 DV), which works in Adobe Premiere Pro 1.5.1.

Click here to see a larger image Editing was done in Premiere Pro 1.5.1 in 1080-50i using CineForm Aspect HD Codec.

Editing native in HDV with no quality loss is now touted by some NLE vendors, but it was not an option for Windsor Knot. Because HDV's MPEG-2 compression retains only one frame (the I-frame) every half-second or so and interpolates the others, native HDV editing must cope with a major challenge: Even a simple cut, unless executed at an existing I-frame, causes massive recompression and artifacting in the subsequent frames. I chose the CineForm codec, which is wavelet based and doesn't interpolate frames, to avoid this problem and gain some elbow room for the intense process of compositing and color-correcting.

I used the Sony HVR-M10U (Reviews, Sept. '05 DV) as my VTR for capture and recording. The editing system was built around a Windows XP CPU with a 3.2 GHz Intel Prescott HT processor, 2 GB of memory, and dual-200 GB SATA drives set up in a RAID-0 configuration for video storage.

I upgraded to version 3.1 of Aspect HD. The 3.1 version enabled batch capturing, and the process went smoothly. Immediately after each capture, the software automatically transcoded the original MPEG-2 file into the Aspect HD codec. This process was roughly equivalent to real time on the 3.2 GHz machine, which means that, when you use this method, you need to budget for about double the time needed to capture your footage.

Once capturing and transcoding was completed, managing files and editing the HDV material worked just like in any other format. Editing with the Aspect HD codec was a satisfying experience. The response was snappy, and CineForm's real-time engine really worked and allowed me to play back more layers and effects from the timeline than some higher-end hardware-based systems.

Just one thing was missing-live playback to a real monitor. This can't be done in real time via 1394 with Aspect HD because the software must transcode back to an MPEG transport stream for display. A couple of options enable playout to a color-accurate video CRT. If your computer has a motherboard with the new PCI-Express slots, the Nvidia Quadro FX 540 allows a dual-screen editing setup plus built-in HD monitor output. My machine didn't have a PCI-Express bus, so I used the Matrox Parhelia board, which allows a dual-screen setup plus program output to an SD monitor. Although I prefer true HD monitoring, I was able to check scene-to-scene color with this setup.

The color-correction tools in Aspect HD, while not high end, are comprehensive enough to allow pretty decent manipulation of the color in shadows, midtones, and highlights. I tried to stick with the Aspect HD real-time plug-ins to give them a thorough test, and found that the combination of the two (Color Balance and Color Corrector) generally allowed me to do what was necessary.

Click here to see a larger image The Aspect HD real-time color-correction tools provided enough control for the basic scene-to-scene color correction.

I ran into two problems from the initial shoot that normally would have prompted a reshoot, but since that wasn't an option, I decided to fix them in post and test the limits of special effects in the HDV format. Remember when I said an essential rule for shooting in HDV was, "Don't overexpose"? Well, the first shot we did was a rack focus with a brightly sunlit window that was completely blown out. The second problem was a shot where the camera crossed the line, meaning that the character in the shot seemed to be facing the wrong direction when intercut with the other camera angles used in the scene. I decided to use After Effects to fix both these issues. I planned to motion-track a fix for the blown-out window and use a greenscreen shot of one character facing the correct direction to replace the other shot.

Click here to see a larger image To correct and over-exposed window a new exterior was motion-tracked in After Effects. Luma Key and Matte Choker filters were used to make the original clipped window transparent.

The Aspect HD footage imported correctly into After Effects, though it was necessary to manually create a 1.33 anamorphic aspect ratio preset for the footage to display properly. To correct the overexposed window, I prepared and blurred a graphic of exterior foliage to seem out of focus. I placed it behind the video footage (below it on the composition line). I applied the luminance keyer with the matte choker to make the offending window transparent, and the new exterior was motion tracked to match the foreground. Because the shot wasn't designed to be tracked, it required some extensive hand-tweaking to get the motion just right, along with the look of the rack focus. The end result works well, however, and doesn't call attention to itself.

A Click here to see a larger image B Click here to see a larger image (a) The original footage with clipped exterior; (b) the finished effect is less distracting.

The greenscreen shot was an interesting test of how far you can push this highly compressed 4:2:0 format in the special-effects realm. We had high-res still photos of every corner of the opulent living room set, and I simply used a portion of one of those as the background plate. We shot the character (Stanley Baldwin) delivering his line in front of a green fabric backdrop with similar lighting to the live shoot.

Click here to see a larger image I created a greenscreen composite using the Color Range keyer in After Effects. I used the matte Choker filter to smooth the jagged edges of the matte and choke (contract) it to eliminate green edges.

In post, I tested several keyers, including Ultimatte AdvantEdge and Zmatte, but got the best results on our footage with the Color Range Keyer and Matte Choker in After Effects. The After Effects matte choker has improved in the last few upgrades, and now can smooth out the jagged edges of low color sampling schemes (such as DV 4:1:1 or HDV 4:2:0) effectively. Considerable choking was necessary to achieve a clean edge. This worked nicely with Stanley Baldwin, whose 1936-era slicked-back hair provided a fairly defined edge to work with. On a subject with wisps of escaping hair or similar fine lines, those details would have been lost.

I also needed to blur the edge of the matte considerably to match the other hard edges in the natural picture. The initial matte had a crisp, well-defined edge that looked artificial-too sharp-when composited into the HDV picture. This is true to a lesser degree in less-compressed HD formats as well, but mainly showed the limits of the Z1U lens. The HDV format can hold edges that are much sharper than the standard lens can transmit.

Deinterlacing and conversion

Shooting in 50i was a decision I made in the beginning to preserve video quality through to the end. Prepping the final sequence for DVD and film output required a two-step process: First, I had to deinterlace the footage (from 50i to 25p), and then change the frame rate from 25p to 24p. (After I wrapped up postproduction on Windsor Knot, a new version of Aspect HD was released that would have made it a one-step process; see the "An Even Better Workflow" sidebar.)

There are numerous techniques for deinterlacing footage, some of which are terrible (field duplica-tion), others of which are a little better (field blending methods), but the goal is to get good results without sacrificing vertical resolution. After all, what's the point of shooting in a high-definition format if you just go and toss away half the vertical detail in post?

The approach I used for Windsor Knot was a "smart" deinterlace in After Effects, which involves identifying those areas of a picture that have motion between the fields and those that don't, and selectively applying field blending only to the areas that have interfield motion. Another good option for smart deinterlacing is DV Filmmaker from DV Film in Austin, TX (www.dvfilm.com). DV Filmmaker is available for both Windows and Mac, and has a batch mode that can process individual capture files. For all of the details on what constitutes a "smart" deinterlace, along with detailed instructions for setting it up in After Effects, go to DV.com.

Once the material was deinterlaced into a 25p AVI movie, conversion to 24p was done on a frame-to-frame basis by interpreting it as 24 fps in After Effects. To stay in sync, the audio was slowed down 4 percent in Sound Forge separately. The resulting 1440 x 1080 24p AVI file was ready for output to film. For DVD output, the file was scaled in After Effects down to 720 x 480 with 3:2 pulldown applied. The end result-viewed on a 720p projector- looked very film like. Surround-sound encoding was provided by the SurCode for Dolby Digital 5.1 Encoder plug-in.

If you'd like to see Windsor Knot, you can learn more about the short and get a DVD copy online at www.windsorknot.org.

Film blowup discoveries

The most intriguing test of the HDV format was the final stage-the output to 35 mm film. I sent identical 2-minute 24p CineForm HD codec files to DV Film in Texas and Heavy Light Digital in New York. The file included a test shot with high color and detail, and the greenscreen composite shot from the finished short, as well as interior and exterior shots from the piece. Both labs printed the file to Fuji 8522 negative stock and printed to Fuji print stock.

Because HDV-to-35 mm is still fairly new territory, I used two different labs to discover what, if any, differences would show up in the final print. The results point out the importance of running a test before choosing a lab.

We screened the test prints at the School of Filmmaking, North Carolina School of the Arts, in a full-size theater. A panel of experienced film DPs was in attendance, including Arledge Armenaki (Dennis the Menace, The Howling V), Richard Clabaugh (Children of the Corn IV, Plato's Run, The Prophecy II), and D. A. Oldis (Non-Abductees Anonymous). Armenaki and Clabaugh had both taken part in a similar screening of DV-originated footage blown up to 35 mm a couple of years ago.

The first print from DV Film was a bit disappointing. The DPs complained about low black density (blacks looked dark gray instead of black), graininess, pixelization, color blocking, and low saturation. The resolution of fine detail in the test shot was lacking.

The print from Heavy Light Digital was much better. The print had good black density, yet still had visible detail in dark areas. The color was better, and the resolution was much sharper. Wide shots of the manor house exterior showed detail in individual stones.

DV Film asked for another shot at the footage. The second print was better and had higher overall detail. The blacks were denser-too dense, in fact, with a subsequent loss of detail, and the color was artificially intense.

We watched the samples twice, pausing in between to talk about them. Afterward, I asked the group to sum up their reactions to the best example, and to HDV as a low-budget filmmaking format. Overall, they agreed that the HDV blowup was substantially better than DV-originated blow-ups we had seen previously, though the sample HDV clearly remains a step below true HD in effective resolution. They thought it was likely that the inexpensive lenses used on the lower-priced prosumer units contributed to the softer image when compared with footage from a Panasonic VariCam or Sony FDW-900H; those cameras are used with lenses that cost many times more than the total price of the Z1U.

After some discussion, Armenaki and Clabaugh agreed that the blowup from Heavy Light looked very much like a blowup from 16 mm. They felt that origination on Super 16 would look somewhat sharper, but the parallel to 16 mm was strong. Armenaki went so far as to say that if he had been told it was a blowup from 16 mm before screening the footage, he might not have noticed the subtle cues that the footage had originated as video. All agreed that the result was watchable, and that an average audience wouldn't notice the differences if the other aspects of story and filmmaking craft were right.

It was interesting that none of the DPs noticed the greenscreen composite shot until I pointed it out. D. A. Oldis was impressed that the composite was so clean when originating from a 4:2:0 format.

Budget considerations

Then came the difficult question: At what budget level should a filmmaker consider HDV? This produced several answers and discussion about local low-budget projects being shot on film. Armenaki and Clabaugh finally suggested a feature project under $200,000. Both felt if they had higher budgets, they would opt for renting a higher-end HD camera or shooting on Super 16.

To answer this question, of course, requires a judgment call. There are many other factors that would enter into a producer's decision to use HDV for a film with potential for theatrical release. Many types of documentaries would be better if they were acquired on video because it's more economical, allowing for many hours of lengthy subject interviews. While a DP might want the higher resolution and exposure latitude of a VariCam or CineAlta on a $500,000 project, the producer and director might opt to pay for higher quality talent in front of the lens-or hire a larger number of high-quality actors. The producer might even prefer to actually pay actors who otherwise would be working on deferral (fat chance!).

Conclusions

My overall reaction to this trial project was real excitement about the potential of the HDV format.I'll be even more excited when the manufacturers stop squabbling and get HD DVD or Blu-Ray players shipping, so there is a viable HD distribution and playback method. Editing was a far better experience than I expected, and the end result was more filmic than I expected.

As people who have attended one of my workshops know, I pay a lot of attention to compression artifacting. One of the very interesting discoveries I made was that the MPEG-2 artifacting in most scenes seemed very similar to film grain- very different from the obvious mosquito noise artifacting of DCT compression in DV footage. Although the pseudofilm grain was more pronounced than I prefer, it wasn't unpleasant or intrusive, and wasn't more noticeable than the grain on some film stocks.

In retrospect, I would probably opt to use the less-aggressive CineGamma 1 setting on the Z1U or to change the gamma in post.

The format wasn't as delicate as I predicted, contrary to early fears about dropouts. I ran initial tests using ordinary pro DV tape rather than Sony's more expensive HDV tape, and saw no noticeable dropouts. Of course, for a more extensive project, I'd probably want to use the HDV tape just to have the extra bit of insurance against dropouts.

I'll probably buy one of the HDV cameras soon, but before I make my decision, I want to undertake a similar test all the way to 35 mm film blowup. And I plan on reporting the discoveries I make along the way, so stay tuned.

HDV to Film: A Real-World Test (Bonus Tracks)
  
Now that you've read through the main article, here are some extra details and information about the HDV-to-film process that just didn't fit into available space for the feature.

  
Bonus Track 1: Reality Check

So, you're going to do a pizza-budget indie film with your pals in HDV, and dream of doing a 35mm film blowup for distribution. That's great, and exactly why I tested the production process with Windsor Knot. However, let's be realistic.  A quality digital blowup from HDV of a feature-length film is going to run anywhere from $40,000 - $60,000 to start with.  And then there's distribution, without which only friends and relatives will see your film.  In my workshops on digital filmmaking, I make a pretty big deal about raining on this parade -- because I've seen too many people hock everything and end up bankrupt.  Did you know that less than five per cent (that's 5%, or 1/20th) of fully funded, completed feature films are picked up for theatrical distribution?  The percentage of films in the dreaming stage that will find distribution is infinitesimal.
 
Don’t let this stop you from pursuing your dream movie.  However, I recommend that most filmmakers plan for a film blowup -- but don't even think of actually doing one until distribution is on the immediate horizon.
 
 
Bonus Track 2:  Anatomy of An HDV Stream

HDV is a format agreed on by a consortium of manufacturers including Canon, Sharp, Sony, and JVC.  Panasonic is noticeably absent from the consortium and appears to have no interest developing HDV products. 
 
The format has been defined as a constant data rate, highly compressed long-GOP MPEG2 stream that can be recorded on stock DV cassette tapes.  The overall datarate is about 19Mbps for 720p and 25Mbps for 1080i.  The color sampling is 4:2:0, similar to that of DVDs and unlike the 4:1:1 used in NTSC DV.  A 48kHz, 16 bit stereo audio stream is compressed with MPEG1 and is incorporated in the HDV transport stream.  The resulting file can be captured via IEEE1394 interface just like DV footage.
 
The MPEG2 compression used for HDV includes an IBP structure, including both Predicted and Bidirectional frames.  While this approach dramatically reduces the bitrate for a given quality, it also means that editing is quite challenging.  The entire GOP, or Group of Pictures, must be decoded and held in memory on the fly.  When working a transition or multiple layers, the computer may have to decode and hold as many as fifteen or twenty decompressed frames in memory.  The Sony Z1U uses a 15-frame GOP while the JVC HD100 uses a shorter, 6-frame GOP.

Click here to see a larger image An illustration of the MPEG-2 format, with its IBP encoding structure. Only the I-frames have all the information necessary to recreate the captured image. The P and B frames have less information, and the correct cadence of the I, P, B frames is required to play the footage correctly.

Deinterlacing techniques are a favorite topic of self-declared "experts" in web forums, but unfortunately there's lots more smoke and verbiage in those threads than actual illumination.  So let's demythologize the subject here. 
 
The basic deinterlacing provided on the timeline in most NLE packages is simply field duplication.  One of the fields (perhaps the even field) is thrown away and the "kept" field (in this case, the odd field) is duplicated.  In this approach, lines 1 and 2 are duplicates, line 6 is a duplicate of line 5, etc.  This approach is quick-and-dirty and produces the least satisfactory result, since the vertical resolution is instantly halved.
 
A slightly better result can be obtained with a technique sometimes called "instant sex" (honest, I'm not making this up!) which is the deinterlacing version of a color enhancement technique from a decade back.  The video track is duplicated and laid over itself on an NLE’s timeline.  The bottom track is deinterlaced using the odd field.  The top track is deinterlaced using the even field, and then the opacity of the top track set to 50%.  Various techniques touted online call for different variations of boosted chroma, a touch of gaussian blur, or some other combination of effects.  The result is what is known as field blending.  It is an improvement over field duplication because it doesn't have the jagged edges of the first method, but it still reduces vertical resolution by half.
 
The best deinterlacing (which some have called a "smart" deinterlace) involves identifying areas of the picture that contain motion between the fields, and creating a mask so that only those areas are affected.  Inside that mask, the fields are blended (resulting in convincing motion blur) but areas where there is no motion are left unchanged -- and hence retain their full vertical resolution.  This method can be achieved in After Effects (as described in the article), but is also used by DV Filmmaker and Magic Bullet for the 30P mode.
 
The method I used in the actual post for Windsor Knot was a "smart" deinterlace in After Effects.  After the final edit was completed, the entire file was exported as an AVI and then brought into AE.  There are two excellent methods of field blend deinterlacing in AE, depending on the effect you want.  The first results in a sharper picture with less motion blur, the second results in slightly more blur for objects in motion.  In both, areas of the picture that do not move between fields are unaffected and remain at full resolution.
 
Method 1: In the File menu, select Interpret Footage, and in the resulting  dialog box,  select the proper field order..  In the case of HDV footage, choose Upper Field First in the Separate Fields pulldown menu.   Then  check the Motion Detect box immediately below the Separate Fields pulldown, and click OK. Then, from the Effect menu, choose Video and apply the Reduce Interlace Flicker filter to the entire program.  Set the Softness slider to 0.7-0.9.  Do not exceed 1.0.
 
Method 2: In the Interpret Footage dialog,   set Separate Fields to Off.  The Motion Detect box will be disabled.  Then, apply the Video/Reduce Interlace Flicker filter to the entire program.  Set the Softness slider to 0.7-0.9.  Do not exceed 1.0.
 
Both of these methods work quite well; the second does begin to affect non-motion pixels if Softness values of 1.0 or above are used.  It does give stronger apparent motion blur, however, and that was desirable for the look of Windsor Knot, so I used method #2.
 
As mentioned in the article, aAnother good option for "smart" deinterlacing is DV Filmmaker from DV Film in Austin, Texas www.dvfilm.com, and is available .  DV Filmmaker is available for both Windows and Mac., and has abatch mode that can process individual capture files.
 
 
Bonus track 4: What about the CineFrame mode?

The Sony HVR-Z1U has a "CineFrame" mode which the manufacturer claims creates a film-like motion.  While the mode has some fans, most aficionados of the “film look” decry the mode.  CineFrame mode takes the interlaced video and applies a 3:2 pulldown to it; indeed, this is similar to the manner in which 24 FPS film is transferred to video.  However, since the Sony's CCDs operate only in interlaced mode, the motion blur is the same as 60i, not like 24p at all.  And since the faked pulldown is achieved by tossing away fields periodically, the resulting motion is jerky and irregular.  The motion artifact shows up most dramatically on camera motion that is usually avoided in 24p camera work; but the irregular jerkiness is evident enough to make me avoid the mode.  In addition, when CineFrame mode footage is deinterlaced to create true 24p, the result has lower vertical resolution.  In his tests, Adam Wilt also identified an artifact that would show up occasionally where a field would be pulling color samples from a previous field, resulting in an odd color flash in some scene transitions.
 
Bonus track 5: What About 60i to 24p?

While there are some amazingly good methods of creating 24p footage out of 60i, they are very complex and involve sophisticated motion interpolation.  Examples would be the Snell & Wilcox Alchemist standards converter or the Twixtor plugin.  Any software that does not use motion interpolation (such as Magic Bullet or CineLook) will accomplish the conversion by "throwing away" a field periodically to make the frame conversion math work.  The result is often an irregular jerky motion that would not occur in real film, and which can be quite distracting.  This is why shooting in PAL 50i has always been preferred for 24p conversion, since the frame rate is so close that a frame-to-frame conversion can be done without losing any fields or frames in the process.