HD editing and compositing workstation

The technical specifications imposed by the highly demanding HD format on editing and compositing workstations make choosing the right system an absolute minefield for most editors. Because HD requires data rates and processing power that are up to 46 times higher than less demanding formats such as SD or DV for example, the system requirements for HD editing and compositing workstations are entirely different from any other systems to date. In this 2 part article, we will look at the differences between HD systems and those designed to handle lower format. In next month's issue we will see how these differences go beyond the purely technical as we go into ergonomic, usability and financial issues.

There are three basic approaches to handle the extraordinary data throughput and processing requirements of HD: dedicated hardware, general purpose computers and a third that can be used with either of the first two, video compression. Three years ago, when the efforts to develop workstations for HD were started, it was not clear which approach would be feasible. Indeed the conservative approach was to choose dedicated hardware because it was unclear that Moore's law would prevail and allow the general purpose computer to become fast enough to handle such data speeds. Certainly at the time they could not. An even more conservative route was to choose an easier combination of dedicated hardware and video compression. This is even more guaranteed to work but with quality drawbacks. Today, it is clear that Moore's Law has again prevailed and computer development at the high-end has met the challenge of the data rate requirements necessary for processing uncompressed HD.

The 1 Beyond HD Pro™ has made the most of advances in computing power and uses an open system, general purpose computer approach. It is therefore much more cost-effective and unlike using dedicated hardware, it has the future flexibility to add new capabilities just by adding new software. Perhaps most importantly, it works in the highest possible quality, 10 bit uncompressed HD. There is no substitute for the speed, quality and efficiencies of HD editing on-line all the time. If you have an extremely large amount of raw footage, you may use reduced quality to initially scan it and create a batch capture list. This could cut 50 hours of video, for example, down to 5. Then capture it in HD and do all the editing on-line. By this I am not recommending off-line editing. This is video selection only and does not preclude the advantages of doing all editing on-line i.e. viewing the full HD quality throughout the editing process.

Another significant challenge for the HD system designer is disk speed. The transition from SD to HD is very significant. For disks, this is not only a challenge for data rate but it also impacts response time. These massive amounts of data are not only required to be retrieved, but they are time sensitive. Unlike almost any other computer application, video editing is unique in its requirement for constant sustained data rate combined with fast response time. The difference of a few milliseconds response is the difference between a useable system and a frame drop, a worthless system. This is also combined with the requirement for completely random access. If you have a clean disk and are inputting data from a tape deck, the writing is fairly contiguous. When you edit and playback the time line which contains portions of clips in completely random order, smooth responsive playback is much more demanding.

The traditional approach to this problem is to use SCSI disks. SCSI disks in a RAID configuration can just barely do the job, but they can do it. However as you will note in the following sections, there are many disadvantages to this approach.

The first disadvantage of traditional SCSI is disk bus design. Disk speed in a SCSI raid configuration is sufficient for HD if you use enough disks but you can not have too many disks on a given bus. The speed curve goes up with the number of spindles (disks) but with SCSI you reach the bus bottleneck point. The SCSI bus design allows as many as 16 drives on the same SCSI bus. This bus has a high momentary bus speed (which is what you see quoted, 320 MB/s), but this is not sustained for even a small amount of time. To make matters worse, because the bus design allows up to 16 drives on a single bus, as the number of disks increases, you reach a peak and then the sustained data rate actually decreases due to too much bus traffic. Therefore with a limited number of disks the maximum sustained data rate for SCSI is typically over 200 MB/s. Again this will sustain a single stream of HD, which can require 185 MB/s, but it's not sufficient for the instant, smooth response you are accustomed to with SD.

The other major disadvantage of SCSI disks is available storage size. Given the fact that HD can require as much as 10 GB/min, you can see the problem of limited storage size. The typical SCSI storage configuration is 7 drives per bus with 2 buses for a total of 14 disks. This means you are limited to 2 TBs of storage. This limitation takes us back 10 years to a time when we had to do SD projects in sections due to storage limitations. Worse yet, if you apply Raid 3 protection, 2 TBs is reduced to 1768 GBs. Allowing for temporary files and other project components this leaves a little over 100 minutes of raw video storage. Editors today are spoiled because disks have become so large that when working in SD you can have the luxury of many hours of raw footage instantly at your disposal.

With the unique 1 Beyond HotRock™ storage, the 1 Beyond HD Pro system can have as much as 7.5 TBs of internal, fast, raid 3 protected storage. 1 Beyond HotRock storage (Hot performance / Rock Solid) has a unique solution to the disk speed and bus speed problems. HotRock storage has a separate bus for each disk. This eliminates the bus traffic problem and combined with the fast and separate PCI buses discussed above, yields over 400 MB/s of sustained data rate. Roughly twice the sustained data rate of traditional SCSI disks. Some of the advantages of this storage are obvious but many are much more subtle. The discussion below elaborates on these advantages.

An important part of the editing experience is the responsiveness of the system. When editors or artists use an HD system for the first time, they are often surprised by the lack of system responsiveness. Smooth response is very dependent on the data rate of the system. Straight playback requires 185 MB/s. When you scrub the timeline or search at 2X, 4X or 10X for example, you are requesting much faster data throughput than 1X playback. Editors today are accustomed to virtually instant response because the data rates of typical SD systems are many times faster than the required data rate for 1X SD playback. When you work on a system with barely enough data rate to play the HD timeline, there is a definite sluggishness to operations like scrubbing and searching for an edit point. To test this, try the "J K L" keys test. For those unfamiliar with the search keys, successive "L" keys move the timeline playback faster: first twice, then four, then ten times the playback speed. The "K" key instantly stops play back and the "J" key moves the playback in reverse faster and faster. A professional editor who is used to using these keys is often disappointed with the response of an HD system.

Professional editors who have tested the 1 Beyond HD Pro have been very pleased to find it so responsive. Again, this smooth and instant responsiveness is due to the fast internal buses combined with the unique 1 Beyond HotRock storage.

Audio lag is another area of frustration to a professional editor who is first trying a less responsive HD system. When searching for an edit point, an editor is often dependent on a point (word gap) in the audio. It can be very frustrating if there is even a split second audio lag when starting playback or scrubbing. Again, test this as explained above with the "J K L" keys. With the 1 Beyond HD Pro, you can quickly find the edit point you want without having to do several iterations of "cut and play". This may sound insignificant but it is another example of why system responsiveness can spell the difference between a pleasant, efficient editing experience and frustration.

Achieving a pleasant and efficient editing experience will be the subject of next month's article when we look at ergonomics, usability, and compare the tradeoffs of compressed and uncompressed HD all put in context with the financial considerations. For additional details consult www.1Beyond.com and read the White Paper "Design Considerations of HD Workstations".

Terry Cullen - President, Founder and CEO: A computer industry veteran, Cullen has more than 30 years of computer industry expertise, including senior positions at Westinghouse, as well as at Digital Equipment Corporation where he was product manager for Ethernet responsible for establishing it as a worldwide standard. Cullen has used his computer science and Harvard Business school background to build a career as a successful serial entrepreneur. Frustrated with the cost of video editing for his own companies, he founded 1 Beyond in 1996 because he was convinced he could manufacture PC-based editing systems that offer the features of high-end systems at a fraction of the cost. In 2003 he repeated this accomplishment for High Definition editing and compositing.