Back in "Lookin' Sharp," (Aug. '00 DV), I looked at sharpness, resolution, and aliasing (www.adamwilt.com). With the advent of low-cost, undersampled HD camcorders, it's time to revisit the topic and reconsider how we measure camera resolution. Undersampling and its companion artifact, aliasing, aren't just a problem in HD, but the demands of packing too many pixels into too tiny of a chip make them more common in small HD cameras.
Some definitions
Resolution measures how much spatial detail a system can reproduce: The finer the detail (the higher the detail's spatial frequency), the higher the resolution. In video, horizontal resolution is measured in TV lines per picture height, TVl/ph, where a "TV line" is a line of dark or a line of light. The measurement is defined across an area of the image as wide as the picture is high, making it independent of aspect ratio and easier to compare with vertical resolution, traditionally measured in scanlines.
Resolution is traditionally specified as the point where increasingly fine detail blurs out to an undifferentiated gray—where its contrast diminishes to zero, or very close to it.
Modern cameras resolve solid black and solid white accurately: their contrast ratio is 100 percent. As detail frequencies increase, the contrast of that detail tends to diminish, resulting in a modulation transfer function, or MTF, that gradually goes to zero as the limiting resolution is reached. (Strictly speaking, MTF is measured using targets with sinusoidally varying contrast, whereas the square-wave targets of most video test patterns yield a measurement of contrast transfer function, CTF. For our purposes, this distinction is unimportant.)
Figure 1:
Two MTF curves: The red curve resolves 100 lines more than the blue curve, but the blue curve is perceived as being sharper. |
Image sharpness is a psychophysical reaction proportional to the square of the area under the MTF curve. An image with high contrast most of the way out to its limiting resolution may well be perceived as sharper than another image with degraded contrast but a higher limiting resolution (Figure 1).
Sampling is the process of describing a signal (such as an image) by taking small periodic snapshots, rather than recording it continuously. Tubes (and film) are continuous image sensors, but CMOS and CCD sensors have discrete light-sensitive photosites (pixels) that sample an image. When you have more samples than you need to represent scene detail, you're oversampling. When you have more detail than you have samples to record it, you're undersampling.
Aliasing is an artifact of undersampling, and we'll get into much more detail about it in a minute.
Measurements
Measuring resolution for continuous imaging systems like tubes is a simple matter of focusing on a test image portraying areas of differing detail frequencies such as resolution trumpets or frequency sweeps and observing the monitor to see where detail vanishes (or, more precisely, looking at the video waveform and measuring the contrast as represented by the amplitude of the luma signal).
Measuring resolution for discrete-photosite CCD or CMOS sensors is a more uncertain proposition. As long as an image is sampled more finely than its detail changes, the sampled image will faithfully reflect reality. The sampled image will show increasing detail frequency up to the point where the detail frequency equals the sampling rate (Nyquist's sampling theorem tells us the highest image frequency we can sample is half of the sampling frequency, but "TV lines" consist of black and white lines indi-vidually—it takes two TV lines to make a complete cycle usually measured by the theory).
Figure 2:
Alignment between test pattern and photosites is critical at the limiting resolution. |
This holds when the lines exactly match the photosites on the sensor. Each photosite will be "viewing" a solid black line or a solid white line, and the image of the lines will have high contrast. But if one offsets the lines by half of their spacing, one photosite will see half a white line and half a black one, the next photosite will see half a black line and half a white one, and so on; each photosite's average picture level will be a neutral gray (Figure 2). If you slowly pan across the resolution trumpets on a test chart, you'll see the trumpets varying between contrasty and grayed out at the point where the sampling frequency equals the detail frequency, with this dynamic change repeating as the pan continues (see the two samples, mere frames apart, from the HVR-Z1 in Figure 3). Thus we have the peculiar situation that the apparent resolution varies as the camera moves.
Click To Enlarge
Figure 3: (left)
Two frames from the Sony HVR-Z1.
Figure 4: (right)
Images from three cameras:
at 800 TVI/ph, the HVX and Z1 show roughly 300-line aliased detail, while the CineAlta is still accurate. |
Once we start imaging even finer detail, we undersample and aliasing results. Aliasing is an artifact of detail too fine to be accurately reproduced in the imaging system used. It shows up as spurious detail that shimmers and moves as the camera moves, as crawling moire on repetitive patterns and stair-stepped edges along near-vertical lines. To a first approximation, aliased detail falls in frequency as scene detail increases (up to a point where scene detail hits two times the sample rate). If your sensor samples at a rate of 550 TV lines, then 600-line scene detail will appear as 500 lines in the image, 700 lines shows as 400 lines, and so on (Figure 4). Thus aliasing doesn't contribute to the right-hand side of the MTF curve, it "folds back" as a reflected artifact, adding spurious detail below the Nyquist limit. That detail is inconstant and mobile and therefore calls unwanted attention to itself. It appears as a defect, not an enhancement (Figure 5).
Figure 5:
A lens resolves almost 800 lines, but its CCDs sample 550 TVI/ph. The shaded area determines sharpness, while the reflected detail adds alias artifacts. |
Is it really that bad?
My narrative assumes point sampling, where the value of a sample is determined by measuring an infinitesimally tiny area. In reality, the photosites on a typical sensor cover 20 to 80 percent of each pixel. (The remaining area is filled with other circuitry. Frame-transfer CCDs offer 100 percent fill ratios but require mechanical shutters, and no sub-$10,000 camera uses FT chips.) The larger the fill ratio, the closer the chip approximates a continuous sensor—and the less aliasing you'll see. Aliasing is still present, but with lower contrast so its impact is lessened. Unfortunately, high fill ratios are difficult to achieve.
Aliasing is reduced through the use of optical low-pass filters just in front of the sensors to blur fine detail. Ideally all detail below the limiting frequency would be allowed through and all higher-frequency detail would be blocked, but it's impossible to craft such a "brick wall" filter. Manufacturers must choose between a strong filter that blurs too much useful detail and a weak one that preserves scene detail at the expense of excessive aliasing. Most of them choose more detail and more aliasing.
What about pixel shift? Many manufacturers horizontally offset red and blue sensors half of a pixel's width from the green sensor, effectively doubling the number of horizontal samples. Panasonic also uses vertical pixel shift in some of its cameras, most recently the HVX200. However, the Y (luma, or brightness) signal is primarily derived from the green channel (about 59 percent in SDTV colorimetry and 70-71 percent in HDTV), with red and blue combined contributing only about a third of luma, so most vendors claim "only" a 50 percent increase in effective resolution.
Pixel shift improves image quality, but it's apparent from looking at the test charts ("Four Affordable HD Camcorders Compared," May '06 DV) from which this article's images were taken that aliasing isn't affected. What we saw on the charts closely correlates with the raw CCD pixel counts used by the various cameras. The Sony Z1 and HVX200, with 960 pixels per line, both resolve about 550 TVl/ph (960 ÷ 16 x 9 = 540, close enough by our best-guess measurements). The 1440-pixel Canon gets up to almost 800 TVl/ph (1440 ÷ 16 x 9 = 810). Vertically the 540-line-high chips on the HVX200 result in 540 lines before aliasing occurs. So how does pixel shift help? It may indeed enhance and add information to the signal, so that the MTF curve is boosted. Perceived sharpness may be increased, even if limiting resolution isn't.
The presence of high-frequency aliasing indicates that the lens is capable of supplying finer detail than the chips can handle, but a few moments contemplating the resolution trumpets should convince you that such detail goes to waste: It doesn't enhance image sharpness, but instead degrades its accuracy. For that reason, simply looking for the point on the MTF curve where contrast goes to zero isn't good enough—you also have to consider the sampling limit. Whichever comes first defines the true resolution limit of a discrete-sampling camera, and aliasing may introduce unwelcome artifacts to the image below that limit.
Aliasing is much less noticeable in real-world images than on test charts. As with vertical smear or mosquito noise, it's just an artifact of current technology that may affect how a given scene is rendered. If you're aware of it, you're better prepared to look for it and take steps to reduce its impact when it appears. Avoid fine repetitive patterns that show moiré, defocus offending details, or use a camera with a higher limiting resolution.