Behind Avatar's Fancy Special Visual Effect
Behind Avatar's Fancy Special Visual Effect
Since 12/18/2009, Avatar has attracted all people's eyes on this
planet. I watched a 3D IMAX version of this film some time ago. After
seeing those fancy special visual effect, I can't help wondering how
they accomplish that and what computer hardware/software they had used.
The special effect of Avatar is mainly made by Weta Digital(alongside with ILM), who had also made Lord Of The Ring, King Kong etc. Data Center Knowledge and Information Management both have some great articles([01] - [07]) about the IT infrastructure that supports the making of Avatar.
Here,
I try to summarize core information gathered from these posts and focus
on the IT related aspects. For the making of special effect itself, see
"Avatar - The Game Changer" please.
Part I - Hardware Configuration[3][7]:
Cluster
- HP Cluster Platform 3000BL
- 34 Water Cooled Racks(from Rittal), each with 4 HP BladeSystem Chassis
- Each Chassis consists of 16 Blade Servers
Blade Server
- Type: BL 2x220c
- 2 Nodes, each with 2 4-core processors
- totally 34 * 4 * 32 * 2 * 4 = 34, 816 cores
RAM
- 104 T
Network
- 10G Ethernet Network from Foundry Networks(now acquired by Brocade)
Storage System
- 3P Network Area Storage
- FAS6000 and FlexCache from NetApp
- Titan Server from BlueArc
Space Size
- 10,000 square feet
Cooling System
The industry standard, raised floors and forced-air cooling, doesn't work because the blade server is very dense. Weta's innovative solution consists of two parts:
- Enclosed, Water Cooled Racks where the hot air is sucked into a radiator and cycled back through the front of the machines.
- Run the Machine Warm (~25 C), which modern computer doesn't mind.
Part II - Movie Rendering
"The main activity in a visual effects data center is called rendering,the process of turning the digital description of an image into an actual image that can be saved to disk and eventually written to film or another media. And the banks of computer is called render wall."[01]
Software
- OS: Linux
- Job Management: Pixar Alfred
Job System
-
In HPC or MPP community, each job is huge. But when doing move
rendering, you faces with large amount of relatively small tasks. In
any single job, there might be thousands of interdependent tasks. As
soon as CPUs on the render wall are freed up, new tasks are fired at
idle processors.
- The queuing system is a Pixar product called
Alfred, which creates a hierarchical job structure or tree of multiple
tasks that have to run in a certain order.
Work Flow
"The artist shows their work, the director wants more of this or that and it goes back and forth many times,” says Wilkie. “It’s the review cycle
that drives the way things look and that drives our job work. In that
way each tiny bit gets better and better, then perfect, and we move on
to the next item.” [01]
Processing Scale
- Job Scale : 10K jobs and 1.3~1.4 M tasks per day
- Data Scale : 12M/Frame, 288M/Second and 17.28G/Minute ...
- Speed : 7~8 G/Second for about one Month
[Reference]
- Data Center Aspects
01. Processing Avatar
02. Data Center Lessons from Avatar
03. Data Crunching Powerhouse behind Avatar
04. One Data Center to Rule Them All
05. Overheard: Avatar's Data Center
06. Data Center Plays Supporting Role in Avatar
07. Avatar, Hollywood and the Data Center
08. Weta Digital's Cluster Ranking among Top500 systems
- Special Effect Aspects
11. Avatar, the Game Changer
12. ILM Steps Into Making Avatar Visual Effect
- Partner's News Report
21. HP - Behind the Magic of Avatar
22. NetApp - Weta Digital and NetApp Bring Avatar to Life
23. BlueArc - BlueArc Storage Solution Powers Visual Effects Behind Avatar
- HP Blade System
31. HP - What is a Blade?
32. HP Cluster Platform 3000BL DataSheet
33. HP Official Page on BL2x220c
34. BL2×220c Video Introduction
35. Blog on Inside BL2X220c