Where would the HD revolution be without affordable digital storage? No matter what flavor of HD you're shooting, editing, massaging, or playing out to air, you need big hard drives to hold all that high-def footage. If you're working with material that's longer than a few minutes, a big hard drive will likely translate to lots of hard drives bundled together into a storage system known as a RAID.
First conceived in 1987 as a redundant array of inexpensive disks and later renamed redundant array of independent disks, RAID storage systems have become an indispensable component of data-intensive workflows such as the aforementioned HD shooting, editing, massaging, and play out. Thanks to RAID, editors can power through hi-def video streams, multiple users can collaborate on projects simultaneously, and facilities can avoid production meltdowns in the near-inevitable event of hard drive failure.
Click To Enlarge
Avid found its RAID partner in Medea, which it acquired in January 2006.
|
Amid ever-tighter project deadlines and the constant need for greater bandwidth, RAID technology keeps getting faster, smarter, more scalable, and perhaps best of all, more affordable. In the following primer, we'll look at some RAID basics and potential gotchas.
Click To Enlarge
LaCie's Biggest FW800 sports RAID capacities of up to 2 TB and data transfer rates of up to 55 MBps.
|
What level are you on?
Disks in RAID systems can be configured to various levels to yield a combination of greater data throughput (reading and writing speeds) and protection than a single drive could provide on its own.
At the most basic technology and budget level, you can take two off-the-shelf hard drives and stripe them together using software tools that are built into Mac OS X or Windows XP. Striping refers to how drives divvy up data (say, elements of a video stream) among one another. Several RAID levels approach striping differently.
Click To Enlarge
A RAID-0 setup breaks data into blocks (shown here as A1, A2, etc.) and divvies up the blocks among the drives in the array.
|
RAID level 0 (RAID-0), the simplest configuration, stripes data across two or more drives to keep editing sessions running smooth and fast. This basic level offers all speed and no security; the harder you push a RAID-0 system, the greater the risk for irrevocable data loss.
RAID-1 introduces the concept of data redundancy, as one disk mirrors (i.e., is a complete copy of) another. If a drive crashes, users still have all their work. RAID-3 includes parity information--a reference copy of data--on a dedicated disk to safeguard against errors. The trade-off for such protection is a loss of an entire drive's worth of capacity.
In RAID-5--currently the level of choice for video editors--parity information is distributed in blocks across multiple disks. This way, if one drive fails, a user can swap in a new drive without shutting down the system--and the disk can automatically rebuild itself and assume its place in the array. (Video systems will sustain a hit on performance during the rebuild; depending on the gear, operations could slow, frames could drop, or screens could go dark.)
Click To Enlarge
Click To Enlarge
RAID-3 and RAID-5 configurations contain parity information, or duplicates of data blocks, to protect data in the event of a drive failure.
|
High-end houses with big bandwidth requirements also employ nested RAID levels that have arrays layered on top of one another. RAID-50, for example, is a set of RAID-0 arrays grouped together in a RAID-5 configuration.
When choosing one level of RAID over another, individuals and facilities must prioritize what they want out of their setup--be it lightning work with video files, a safety net for projects, capacity that leaves room to build, or best overall bang for the buck.
LaCie spokesperson Melissa Logan notes that while RAID-5 gives editors recourse if a drive failure occurs, it generally reduces the speed of production by one-quarter compared with RAID-0. "Large production studios have IT managers who handle the backup portion, so they're buying RAID for speed," she says. "We always stress the importance of keeping a backup, whether it's to a separate external [drive] or in a RAID box."
"Everyone gets burned at some point by the loss of a drive," adds Pete Schlatter, director of marketing for RAID systems maker G-Technology (G-Tech). "That's when they say, "This isn't going to happen again'--and they choose a RAID-protected system."
Editors working at a specific resolution and number of video streams will want to consult with their NLE manufacturer for recommended RAID configurations. Users of Canopus' Edius, for example, must attach a RAID array for multiple HD stream output. Avid users pairing the Liquid (XP) platform with the company's VideoRaid boxes need to set up four arrays to work with uncompressed HD streams.
Performance versus protection
At the dawn of nonlinear editing, companies such as Avid offered only minimal RAID-0 functionality to enable capture, editing, and playback of digital video. Just a few years ago, protections such as parity posed a bottleneck for NLEs. "We had a really hard time finding a fault-tolerant RAID solution that would match our NLE's resolutions and real-time performance, at a price that the market would bear," says Lesley Glorioso, Avid's senior product marketing manager.
Avid ultimately found its RAID partner in storage specialist MedŽa Corp., which it acquired in January 2006. MedŽa's rebranded fail-safe line of VideoRaid storage ranges from a desktop SCSI system that offers transfer rates of 160 MBps and five removable drive modules, to a tricked-out, rackmountable Fibre Channel model that offers an ultra-fast 300 MBps transfer rate and configuration for as many as 10 drives. Through MedŽa's patented technology, Avid has been able to manipulate the distributed parity of the RAID-5 scheme to wring even more performance out of the array, Glorioso adds.
For the entry-level editor who just needs a jolt in throughput, MedŽa offers several RAID-0 arrays, including the VideoRaid HDV--a SCSI system that supports four simultaneous streams of HDV video, starting at $1,479 for 640 GB of storage and a 65 MBps transfer rate.
Fibre Channel drives dominated Avid's storage business a decade ago, Glorioso points out, but now the high-end hardware interface is limited to the most bandwidth-hungry operations working in uncompressed HD video. The majority of today's video pros employ a RAID system that's based on drives using either SCSI or the Serial ATA (SATA) interface; the latter underwent dramatic improvement in recent years in a bid to become truly enterprise-class.
Apple optimized its fourth-generation, SATA-drive-based Xserve RAID system for video editing. The 7 TB Xserve RAID ($5,999), which comes set to RAID-5 as a default, boasts separate hardware controllers for each hot-swappable drive module--all of which must pass Apple's own quality-control test beyond general original equipment manufacturer (OEM) standards. As a result, says Alex Grossman, senior director of server and storage hardware at Apple, the company can guarantee levels of performance--including an 85-percent level of performance during a drive rebuild.
Other companies have joined the RAID fray for video apps. LaCie's new Biggest FW800 RAID system ($1,699 for a 1 TB model) features a FireWire and USB 2.0 dual-interface for its four internal SATA drives, which can be configured to RAID-0, 0+1, 5, or 5+ levels. Archion Technologies' just-announced Synergy HD, certified for use with Avid Unity networks ($19,995 for 8 TB of RAID storage), comes ready for intensive high-definition work, featuring two 2 Gb Fibre Channel ports for data transport between the storage unit and host computers.
Click To Enlarge
Weighing just 1.15 pounds--and just longer than an open cell phone--G-Tech's G-RAID mini offers as much as 320 GB of storage.
|
Extra perks on state-of-the-art RAID systems include remote management capabilities, where the system controller e-mails the user a faulty drive alert, and empty hot spare disks that automatically begin copying data in response to drive failure.
Behind the multitude of RAID vendors, however, are varying levels of video experience; that too-good-to-be-true Internet offer could be for a system designed to serve Excel spreadsheets. And with so many performance variables--SCSI versus SATA, software versus hardware drive controllers, 7,200 rpm disk platters versus 15,000 rpm models, the number of bad data blocks already on disks from the factory--there's a lot of lingo for the video professional to mull over before taking the RAID plunge (see "An Array of Jargon"). In short, the caveat emptor rule is in full effect.
Inferior RAID systems, says one industry insider, "fail when they're under the greatest load, and at the most inopportune times." Editors and studios "will lose business because they were using Brand-X RAID. They couldn't recover the data, and they missed their deadline. It happens a lot more often than you would imagine."
Apple's Grossman urges RAID buyers to keep in mind scalability options, for everything from storage capacity to data protection levels. "Today you're doing work in standard definition," he says. However, "you might want to do work in high-definition, or you might bring someone else into the organization to work with you. The worst thing you can do is have a bunch of disparate systems all running with different arrays and drives." Often times, Grossman posits, the price differential between simply meeting today's needs and preparing for the future can be made up in a few billable hours of work.
Hard to beat hard-disk drives
RAID was originally designed for magnetic hard-disk drives (HDDs), although most any type of storage device can be fashioned into an array. One group of geeks last year published on the Internet how to build a RAID out of four iPod shuffle devices.
But HDDs still form the backbone of most professional RAID systems, outperforming other storage media such as data tapes, optical discs, or solid-state flash chips on storage capacity, access times, and data transfer rates. With the constant flow of cheaper, faster, higher-capacity, and smaller form-factor HDDs to the market, the medium's role in RAID is unlikely to be challenged anytime soon.
Storage industry consultancy MMIS notes that HDD makers have long been able to increase capacity in their products by 40 percent annually--and they'll have little problem keeping that pace through the next 10 years. That means by 2009, while 1-inch drives for mobile consumer devices will be capable of storing 50 to 75 GB, 2.5-inch drives (the current standard for RAID systems) will tout some 550 GB of capacity.
What's more, MMIS president Larry Lueck says, disk drives recently passed the coveted milestone of transferring data at a rate of 1 billion bits per second. Seek times of 3 to 4 milliseconds and data transfer rates of 125 MBps (the equivalent of 1 Gbps) are now readily available in several enterprise-level HDDs.
And with well-understood science and mature manufacturing processes, the technology's only getting cheaper. "Most of these drives," Lueck predicts, "will carry OEM prices not all that different from those current today, reducing costs per gigabyte to the point where the cost of storage is no longer much of a factor."
Indeed, new drives are creating entirely new subcategories of RAID products, such as portable systems that editors can take into the field. G-Tech, for example, introduced in September a mini version of its RAID solution for mobile multistream video editing. Weighing just 1.15 pounds, the FireWire bus-powered unit offers as much as 320 GB of storage with latest-generation 2.5-inch SATA II drives. Now shipping, the G-RAID mini carries a $499 sticker.
While individual HDDs may get flagged for a rebuild by a RAID system, drive reliability and longevity keeps improving. G-Tech's Schlatter notes his company currently uses drives from Hitachi that have a 3-year warranty, but G-Tech will soon start offering HDDs from Seagate that sport a 5-year warranty. "We've got thousands and thousands of units in the field," he says, "and we don't see very many of them back."
|
Tim Sassoon
DV contributing editor and head of Sassoon Film Design in Santa Monica.
The one thing he wished he knew before installing his first RAID: "That settings to force cache writes were very important if power was fragile, because you can lose the volume in the event of a power outage."
Current RAIDs of choice: Xserve RAIDs and one Huge Systems U320 used with Adobe After Effects and Apple Final Cut Pro running on Mac G5s.
|
|
Terence Curren
DV contributor and founder of AlphaDogs, a production and postproduction facility in Hollywood.
The one thing he wished he knew before installing his first RAID: "That losing power while writing to the drives will still kill them. I lost power on an overnight render many years ago and it killed two drives in the JBOD (just a bunch of disks), which was more than parity RAID could fix."
Current RAIDs of choice: MedŽa, Avid Media Array, Terrablock, and "a lot" of G-RAIDs. Software/hardware AlphaDogs' RAIDs are connected to: Avid Symphony Nitris systems, Avid Symphony Meridiens, Avid Adrenaline HD, Avid Xpress Pro HDs, Apple Final Cut Pro HDs, Digidesign ProTools HDs, plus workstations running Adobe After Effects and Photoshop, Autodesk Combustion, and Avid Commotion.
|
|
Chris and Trish Meyer
DV contributing editors and Motion Graphics columnists; principals of CyberMotion, a motion graphics house in Los Angeles.
The one thing they wished they knew before installing their first RAID: "That supposedly hot-swappable drives really weren't for quite a few years."
Current RAIDs of choice: "We do far less video capture here than we used to, so we've dropped back to mostly LaCie two-drive FireWire 800 RAIDs for faster file access and uncompressed SD capture/proofing on each workstation."
|
|
An Array of Jargon
You know the basic difference between RAID levels. To get maximum speed, reliability, and workflow compatibility out of your storage system, here's more spec-sheet-speak you should consider before buying a RAID:
Hardware versus software controllers. RAID utilities are like any other data-intensive function you perform on your computer: they can be total CPU hogs. Windows XP and Mac OS X come with software that lets you stripe two hard drives together and build a basic RAID array. Alternatively, some external systems, such as Apple's Xserve RAID, have dedicated hardware controllers (read: processing power) for each storage unit. Basically, software RAID controllers are fine for the simpler, speedier configurations. But if asked to perform RAID-5 data protection--or even worse, a data rebuild--software can buckle and burden the CPU.
Interfaces. All paths that data travels along within the RAID system itself and to/from the host computer are not created equal. For system interfaces, SCSI, FireWire, and USB all have their maximum data transfer speeds, measured in megabytes (MB) per second. With drive interface types come reliability considerations as well.
"All drives require housekeeping," says Apple's Alex Grossman. "You want to be able to manage how a drive writes the data." SCSI and Fibre Channel drives attach to a computer through a host bus adapter--a dedicated hardware card that manages the storage units. An ATA drive, Grossman points out, "has half its brain on the drive, and half its brain on the board. So you have a (software) ATA controller, not a host bus adapter, you attach them to. If you have the right command set and the right software managing the ATA drive, you can have that same level of reliability. But natively, you need to have the right tools around it."
Drive speeds. Hard-disk drives are basically miniature turntables spinning a thin magnetic record. The speed at which the disk rotates affects the speed at which it's able to record and retrieve data. Today's entry-level drives spin at 5,400 rpm, while higher-end models top 7,200 rpm, and certain enterprise-class drives hit 15,000 rpm. If you plan on working with HD video, you'll want 7,200 rpm or faster drives.
MTBF. Drive-maker acronym for mean time between failures, which these days averages around 300,000 hours. That's contingent, of course, on the level of usage and system components.
Drive failure, notes LaCie's Melissa Logan, is an inevitable result of physics. There are ways, though, to prolong drive life. LaCie employs "a thermal foam and piping inside the drive to help draw heat away from the drives; the case itself is made of aluminum alloy, a natural heat sink, which again draws heat away from the drive." Logan continues, "We [also] employ a smart fan that turns on when needed and off when not needed." Another basic but important key to drive longevity is to not skimp on system components such as chips and boards--and to make sure these components all can work at the same pace with one another.
|