Multimedia
Blogs
Forum
Affordable HD
Whitepapers
Newsletters
Events
Advertisers
Blogcast
MillimeterThe Visual Effects Industry Needs a New Production Model
Nov 1, 1998 12:00 PM, Ray Feeney, Chuck Spaulding, and Kevin Mullican
The use of digital technology to produce visual effects for feature films is at an all time high. Today, virtually every Hollywood movie employs digital effects in one form or another while certain blockbuster titles feature hundreds of incredibly complex digital scenes. Whereas Jurassic Park involved digital effects encompassing several gigabytes of data (an unprecedented number for 1993), Independence Day, Starship Troopers, and Titanic featured data sets measured in terabytes-a staggering increase in just a few years.
This surge in demand has spurred significant growth in the visual effects industry and has created high paying jobs for hundreds of digital artists and other technical specialists. Today, a dozen or more suppliers take on the effects load while, just a few years ago, single companies usually provided most or all of the visual effects for a film.
As fast as Hollywood's need for visual effects services has risen, the cost of the hardware used to produce effects has dropped. Visual effects houses that paid $80,000 for a computer workstation three or four years ago can get more speed and power for far less money. A visual effect that might have required a Cray a decade ago can now be executed on a desktop workstation.
Software prices reflect the same trend. Companies that 10 years ago employed teams of programmers to write effects software from scratch can today license off-the-shelf software with more features and better reliability for just a few thousand dollars.
While these trends are well known, what does surprise some people is the diminishing return for effects houses. With mushrooming demand and plummeting costs, one might reasonably expect to find visual effects houses carting money to the bank in bushel baskets. But they are not. If you could get a peek at their books, you would find that many visual effects suppliers are making scant profits or are operating in the red. For many it is a struggle to survive.
Not too long ago, the failure of two high profile companies, Boss Films and Warner Digital, rocked the industry. One popular hypothesis for their demise cited the companies' huge staffs and high salaries for digital artists. These enormous overhead costs became unbearable during down periods between big projects. Another theory was that low-ball bidding by competitors was at the root of the financial problems. There is some validity to these claims, but they tell only part of the story. High overheads and low bids do exist, but they are not the cause of the industry's woes. They are symptoms of a deeper and more fundamental problem that lies at the heart of the way production houses go about producing visual effects.
The true source of the problem starts to become clear when you examine the per-frame cost of producing visual effects. Because computers have been getting cheaper and faster, costs per element have been going down. However, complexity is rapidly increasing. In the time that machine speed has doubled, data sets have increased tenfold. Producers may be running faster, but they are falling further behind. This is a scalebility problem. Although the cost of individual workstations has fallen, the total cost companies are spending on hardware, software, and salaries continues to rise. Because the per-frame complexity is not dropping, producers have to buy more workstations, license more software, and hire more artists just to finish existing projects. Faced with this spiraling overhead, production companies find themselves in a position where they cannot afford to be without work. This inevitably leads to underbidding-a problem the studios exacerbate by pressuring suppliers to cut costs. It is a vicious cycle, and it will continue to get worse so long as the gap between production demand and production efficiency continues to widen.
This problem affects large companies more than small companies because large effects producers do not enjoy significant economies of scale. In other industries, large companies produce goods more efficiently than their smaller competitors. General Motors can build a car more cheaply than a single artisan can working in his garage. That is why GM can support the cost of huge factories. But, in the visual effects industry, large operations are not necessarily more efficient than small operations-they are only bigger. Why? Because large and small effects houses follow the same production model, one that is based on the individual workstation.
When effects houses grow by adding more workstations, those workstations continue to operate, more or less, like self-contained production units. It's as if General Motors hired 10,000 workers to each build cars by hand. As a result, big visual effects companies acquire large company drawbacks in the form of high overhead without gaining much advantage in efficiency. This explains why Boss Films and Warner Digital, two relatively large companies, were among the first to go.
Why, after 10 years, are these disadvantages of scale now becoming an insurmountable problem? Because production companies have come to depend on computer prices falling at least $20,000 with every production cycle, a phenomenon that is clearly not sustainable. Machine speed increases do not equal cost reductions. The declining cost of computers may reflect the same percentage decreases as before, but the real effect of a $2,000 savings does not impact the bottom line the same way that a $20,000 to $40,000 cost reduction used to. Even if machine speed could rise ad infinitum, the future effect of increases in individual workstation speed will not produce the same benefit.
If data sets continue to grow at the present rate-and there is no reason to doubt they will-then the yawning gap between scaleability and production demand will grow wider. Focusing on the speed of individual workstations also misses the point. The inefficiency inherent in today's production model is not primarily due to the slowness of individual computers-it is the result of slowness between applications. The real issue is not machine speed or application speed, it is machine-to-machine latency and application-to-application latency.
1989 *Approximately 600 animators in the industry make a full-time living in digital film effects. *The industry spends less than $100 million for the development of effects. *The average number of shots is less than 30. *Eighty percent of content creation software is proprietary. *Ninety percent of computing is done on SGI graphic workstations.
In the current production model, most components exist as small computer islands, networked to one another in a uniform fashion. In this model, the only differentiating factor is the number of islands. The system as a whole is no more efficient than the sum of all the parts. No driving force exists to create a large-scale facility. In order to create a cost-effective solution that scales in efficiency, it is necessary to re-evaluate the system as a whole. The core of the problem lies not in the number and speed of processors, but in how the processors interact with one another without a common memory space. A cost-effective solution demands a hierarchical approach and much finer granularity in the delineation of data across the system as a whole.
Production also bogs down because houses can not arrange work into a chain of individual tasks: modeling, followed by animation, then texturing, then compositing, and so on. The trouble is, in the real world, effects companies can not work this way-production schedules are too short. All of these tasks must be ongoing, more or less, simultaneously. Problems then arise when someone involved in one of these tasks makes a change, for that change affects everyone else in the chain. Distributing updates to 100 separate workstations takes an enormous amount of time and leads to further bottlenecks.
Hardware and software manufacturers have given little attention to the problem of system-wide latency. They are concerned with making incremental improvements to individual pieces of hardware and individual programs. Chip manufacturers focus on making faster chips. Drive manufacturers on making faster drives. Software developers want to make their software run faster and provide more features. Everyone is busy making their piece of the system work better, but no one, it seems, focuses on making the production model work better as a whole.
We need a new production model that will enable large visual effects operations to take advantage of their size, work more efficiently, and dramatically reduce the per-frame cost of producing visual effects. Kodak's Cinesite offers one of the only recent attempts to make a fundamental change in the visual effects production model. Cinesite did not try to improve efficiency by lowering equipment costs-it used more expensive computers to significantly reduce the per-frame cost of producing visual effects. Their concept of reducing the overall cost of producing visual effects by using more expensive equipment was sound and necessary-although it came with too much baggage.
1991 *Approximately 2,000 animators in the industry make a full-time living in digital film effects. *The industry spends less than $200 million for the development of effects. *The average number of shots is less than 75. *Fifty percent of content creation software is proprietary. *Ninety percent of computing is done on SGI graphic workstations.
The new production model must enable large visual effects companies to enjoy economies commensurate with their size. A dramatic improvement in production efficiency would enable them to operate at a profit. Ideally, the new model would organize workstations not as a network of individual boutiques, but as components of a unified production machine. In order to be cost effective, the extensible units must be relatively inexpensive (perhaps NT). Because of cost constraints, the extensible pieces probably attach to the whole system through relatively slow links. Components with higher data demand can integrate through faster (more costly) links with a core computer cluster built from more expensive and the most efficient components.
Most production companies believe they are already doing this now. However, for the outermost components to be useful in this modified star topology, the system as a whole must be aware of the latencies inherent in all the levels of memory into which each component can see. In addition, the periphery devices must retrieve only subsets of data. To use the slower links effectively, data should migrate from the central location to the individual stations. In this model, the longer operators work, the closer the data resides and the faster they can work without adversely affecting the rest of the system.
1993 *Approximately 4,500 animators in the industry make a full-time living in digital film effects. *The industry spends less than $600 million for the development of effects. *The average number of shots is less than 150. *Thirty percent of content creation software is proprietary. *Ninety percent of computing is done on SGI graphic workstations.
This proposed workflow demands a finer breakdown of available memory. The jump between local and remote memory is too great and requires the movement of entire contiguous data pieces before any work can begin. Some intelligent non-standard changes in data sharing at the memory and file system levels would produce large strides in increasing the overall efficiency of visual effects production.
Such changes in data sharing would require multiple delineations between local memory and remote memory with all gradients capable of sharing at the page level (as opposed to the whole file level). The outermost machines need to become page granular memory caches for the inner layers of the system. Of course, each distinct machine would maintain some amount of private memory for local non-shared processes and operating system kernel. Already existent directory-based cache coherency is capable of addressing shared information problems inherent in such a distributed memory model.
We must address similar issues at the file system level. Fortunately, modern technology allows for file system sharing to occur independent of a server. Distributed file systems allow data sharing in much the same way as distributed memory. Furthermore, a page granular file system cache allows discrete data to map directly to local memoryand removes the need to copy the data within memory. This, coupled with some form of shared memory between workstations, would allow for the creation of a true virtual machine that we could apply to the production process as one cohesive system.
This would solve the latency problem, as each workstation would have direct and immediate access to the same data. No more downloading scenes from a server. As artists work, the data migrates throughout the facility. There would no longer be a need to provide updates to everyone in the production chain, as changes would be immediately available on a system-wide basis. Multiple workstations could also work on the same scene simultaneously.A much finer breakdown in the representation of the data in memory, removal of the file system server (and all the overhead associated with sharing at that level), and direct mapping of file system cache data to shared workstation memory could accomplish this memory migration.
Some will argue that visual effects operations cannot be scaled because creating visual effects is an "art," but that is simply not true. As things stand today, visual effects companies have little time for artistic experimentation-they are too busy cranking out effects. Only when their workload is made manageable again will visual effects houses enjoy the luxury of time necessary to experiment for art. Only if the effects business is made economically sustainable will artists get the chance to create the kind of breakthroughs that inspired them in the first place.
Star Wars and Terminator II changed the face of movie making and fueled a decade or more of intense invention, excitement, profits, and growth. Artistic and technological investment at the production company level created the innovations of these films. Today, neither production companies nor software developers can develop a new production model on their own. The problems that must be solved involve how networked computers share memory and access data, and these issues are imbedded in the operating system. Only a manufacturer has the ability and resources to make fundamental changes to the operating system architecture.
When the digital entertainment industry needed to complete tens of shots, it required the utility of a single integrated compute environment-a workstation. Silicon Graphics stepped up to the plate to provide the solution. As shot complexity grew in both size and number, the industry could always count on SGI to deliver the breakthrough technologies necessary.
Most often, these breakthroughs grew outside the world of standards only to become standards themselves. Hardware advances such as processor cache, symmetric multi-processing, and non-sequential instruction execution coupled with operating system advances such as a full suite of TCP/IP services, Sun RPC services, XFS, 64-bit address space, OpenGL, and runtime linking have increased the efficiency of the individual machine.
1995 *Approximately 7,500 animators in the industry make a full-time living in digital film effects. *The industry spends less than $1 billion for the development of effects. *The average number of shots is less than 225. *Ten percent of content creation software is proprietary. *Eighty percent of computing is done on SGI graphic workstations.
Unfortunately, though advances have made inter-operability possible, little has been done to address multi-workstation system-wide efficiency. Today all the inter-operability software is buried under multiple levels of standardization; it is impossible to extract any appreciable amount of efficiency. Non-standard innovation again is probably the only way to solve the problem at hand. The next logical step is for someone to step back and address the industry's need for a single integrated compute environment, no longer at the individual machine level, but at the systems level. It is not at all clear that there is a manufacturer ready and able to take on this challenge since the effort to develop a new production model could cost tens of millions of dollars. Silicon Graphics, the most likely candidate, currently lacks the resources to undertake such an ambitious project and, caught up in trying to broaden its market, seems also to lack the will. Microsoft has no interest in developing a niche market in the film industry. IBM, Sun, and Apple, too, appear unlikely to step forward with a solution.
If no manufacturer accepts responsibility for addressing this problem, the consequences for the film industry could be severe. Large effects operations will continue to find it impossible to operate profitably, and that could lead to a shakeout similar to that of the 1980s when several leading CGI companies-Robert Abel & Associates, Digital Productions, Cranston/Csuri, and Omnibus-went bankrupt, leaving only one principal supplier, PDI, standing. If the flaws in today's production model are not solved soon, many more companies could go bankrupt. As production companies close, hardware and software vendors will be forced to move to greener pastures, leaving a much smaller visual effects industry to face a slow and painful recovery.
In the field of visual effects, some of the most difficult times have come on the heels of significant films that have epitomized the technical processes of the day-taken to a new level. Usually the result of one individual's vision against incredible odds, such films take their place as crowning achievements in the art of visual effects. Those knowledgeable in the art marvel that films such as 2001 and Blade Runner could get made at any price. But all too often, these achievements are also box office failures, and it is only years later that they clearly stand alone as a measure of a particular era in visual effects. It is a tribute to Jim Cameron that Titanic is both a commercial success and a watershed visual effects statement. But, like 2001, it too marks the end of an era in visual effects, not the launch of a new model.
1997 *Approximately 10,000 animators in the industry make a full-time living in digital film effects. *The industry spends less than $2 billion for the development of effects. *The average number of shots is less than 300. *Ten percent of content creation software is proprietary. *Seventy-five percent of computing is done on SGI graphic workstations.
There is no surpassing such motion pictures without significant changes and advances to the art and technology of visual effects. The manufacturers who could provide the next-generation solutions have turned their backs on production, and it seems unlikely that the movie industry can generate the needed innovations alone. It was a long nine years between 2001 and Star Wars, and once again the barbarians are at the gate. The Renaissance is over and a new Dark Age has begun.
Ray Feeney, the founder of Silicon Grail and RFX, developed many visual effects technologies that are now industry standards. Formerly of Robert Abel & Associates, Feeney has earned Academy Awards for his work on motion-control camera systems, the Solitaire film recorder, film scanning technology, and software used in blue screen matte extraction.
Kevin Mullican is a co-founder of Look!, a provider of visual effects and digital production services for motion pictures, television, and commercials. Previously the head of research and development at RFX, Mullican has provided consulting services on digital technologies to effects houses including DreamQuest Images, Rhythm & Hues, and VIFX.
Chuck Spaulding, Silicon Grail's director of marketing and business development, worked as a documentary filmmaker and a television editor before becoming involved in the visual effects industry. In 1992, he co-founded the computer animation company Procrastination Animation. Two years later, he joined RFX and has been working with its sister company, Silicon Grail, since its founding in 1995.
Continue the discussion on “Crosstalk” the Millimeter Forum.