Sony Pictures Imageworks
"Autodesk Maya is terrific! It gave us the ability to believably animate the faces and bodies of all these characters and the ability to write plug-ins and custom tools with MEL scripting. It was just amazing."
—Kenn McDonald, Animation Supervisor, Sony Pictures Imageworks
Sony Pictures Imageworks Animation Supervisor Kenn McDonald and Technical Animation Supervisor Corey Turner talk about the challenges of accurately replicating old men and young women. They detail how their teams benefited from Autodesk® Maya® and Autodesk® MotionBuilder® software, and from Autodesk Consulting, to bring the 1,300-year-old epic tale to life in breathtaking 3D.
© 2007 by Paramount Pictures and Shangri-La Entertainment, LLC.
Charged with creating fully accurate human and mythical characters for every frame of Beowulf, the Sony Pictures Imageworks animation teams took nothing lightly. Using a combination of Autodesk Maya and MotionBuilder, more than 250 artists contributed to the project. Maya was used on all animation shots, while MotionBuilder was used for all motion capture editing, blending, and camera layout. The Imageworks teams also relied on Maya to previsualize and build the film’s strikingly realistic 3D imagery and characters. With help from Autodesk Consulting, Imageworks customized Maya to enable its animators to view a virtual world in stereo.
The team created elaborate facial rigs to handle the painstaking and unforgiving detail required to replicate increasingly complex expressions of male, female, and monster faces. Muscle systems were developed to give Beowulf a realistic physique. Using MotionBuilder, the technical animation team provided director Robert Zemeckis with fully textured and lit scenes from which he chose the most appropriate camera angle or shot sequence.
The Future Is History
When Zemeckis, the man behind such award-winning, effects-laden blockbusters as Who Framed Roger Rabbit? (1988), Forrest Gump (1994), and The Polar Express (2004), decided to take on Beowulf, the 1,300-year-old epic Scandinavian poem of heroes and monsters, moviegoers knew to expect the extraordinary. When the animation team at Sony Pictures Imageworks got into the act, however, “extraordinary” became an understatement.
Over an intensive two-and-a-half years, Animation Supervisor Kenn McDonald and his technical counterpart, Corey Turner, made unique, imaginative, and innovative use of Autodesk Maya and Autodesk MotionBuilder. Not only did they push both systems to their limits, they went beyond those limits with product customization and scripting help from Autodesk Consulting.
“This version of Beowulf began with an involved conversation about characters and faces,” says McDonald. “We started out talking with Bob Zemeckis about how the characters in Beowulf were going to take shape, what facial system we were going to use, and how to fully integrate both Maya and MotionBuilder into our production pipeline. Having perfected our work with bodies and motion on The Polar Express and Monster House, we knew from the start that faces were going to be our big challenge on this show. We spent the first six months or so just figuring out how to best build our pipeline and toolset for moving animation, motion capture data, and rigs back and forth between the packages.”
Just in case you are unfamiliar with the story, Beowulf is full of interesting characters, not least its title character, played by Ray Winstone. Providing a virtual template for action heroes ever after, Beowulf engages in three epic battles through the course of the story. He begins by battling Grendel, a 12-foot-tall wrecking machine, portrayed in the film by actor Crispin Glover, who attacks the local meadhall and Beowulf’s soldiers. After Grendel is vanquished, his even more murderous mother (Angelina Jolie) comes looking for revenge, proving to be a formidable foe. Crowned a king for his heroic efforts, Beowulf ultimately battles a mighty dragon.
Beyond the monsters and mothers, however, Beowulf provides a wealth of epic characters, all of whom had to be realistically and painstakingly portrayed by the Imageworks team—a team that would eventually grow to about 60 animators, in addition to the specialists handling performance capture and facial rigging. In the end, Imageworks would handle some 700 effects shots on Beowulf, all of which went through Maya and MotionBuilder. In so doing, Sony Pictures Imageworks took the worlds of performance capture and 3D stereoscopic imaging to a whole new level.
Under the direction of stereo specialist Rob Engle, the Imageworks team requested specific enhancement to the Maya API (application programming interface) from the Autodesk Consulting team and extended the Maya interface to support their “stereo sweatbox.” Zemeckis and the rest of the Beowulf team were then able to view scenes in context, experiment with 3D in real time, and adjust settings to provide an unparalleled 3D experience for the audience.
© 2007 by Paramount Pictures and Shangri-La Entertainment, LLC.
“All of our shots needed to be set up in Maya,” says McDonald. “The Dragon was completely keyframed, while Grendel was based on performance capture, but also required a lot of keyframing in Maya. In order to have Crispin resemble a 12-foot-tall beast, we had to scale a lot of curves, as well as add weight and adjust timing to create the feeling of a huge monster. Crispin did a lot of jumping around on wires within a scaled set, and we imported that motion from MotionBuilder to Maya to retime it. Through a combination of tools, we attached our new character to the performance capture character to duplicate certain poses and keyframes.”
With help from an on-site team from Autodesk Consulting, the Imageworks teams were able to customize their Autodesk systems and use Maya Embedded Language (MEL) scripting to write the tools of which McDonald speaks:
“A great thing about Maya on this project was that it enabled our technical directors and pipeline people to write plug-ins and custom MEL scripts for the pipeline,” says McDonald. “Maya facilitated moving data between all our packages. We did all of our rigging for the characters and faces within Maya, and we used the system tools for deformations and weighting.”
Facing the Legend
Even more challenging than Grendel’s size, however, was the meticulous creation of the characters’ faces. Says McDonald:
“Faces were the make it or break it part for us. We started out with a generic facial rig for our initial set of controls for each character. As the production went on, we found more and more nuances within each actor’s face. We used the facial performance capture data together with our muscle system to create a matrix of poses. We then created combinations of these poses to create the facial performances. We knew we had to add shapes and deformers to get those expressions right. We wrote custom tools for our facial controls, and each of our characters eventually had as many as 300 facial controls that we could animate. Creating an interface with MEL scripting let us access and organize all those controls really quickly. Right up until the end, we were constantly adding new facial controls. That is another huge benefit to using Maya.”
Interestingly, the high level of detail necessary to create the “less perfect” faces of the male characters actually made those faces easier to create:
“We found the male faces, though intense, to be much more forgiving,” says McDonald. “To create Grendel’s mother, for instance, we had to pay a great deal of attention to Angelina Jolie’s face and make sure it was just as beautiful and flawless as that of the real-life actress. With more wrinkles and imperfections, the faces of King Hrothgar (Anthony Hopkins), Unferth (John Malkovich), and Beowulf himself read more clearly. In the end, our job was all about getting the best realization of the actor’s performance. Maya gave us the ability to animate these characters, and the ability to write plug-ins and custom tools was a crucial part of that process.”
MotionBuilder, Nitty, and Dilo
“MotionBuilder has been a part of our pipeline for some years now,” says Corey Turner, who served as technical animation supervisor on the film. “With every project we work on, we keep building on what we can do. On Beowulf, we pushed the system harder than we ever have.”
Pushing MotionBuilder past its previous capabilities was accomplished, according to Turner, through “Nitty” and “Dilo.” No, those are not two more murderous characters in our grand tale; they are stages in an intricate and ultimately invaluable process.
What Turner and his team familiarly refer to as Nitty in fact stands for near-time integration or NTI. As Turner explains:
“NTI is the relatively early process of getting motion data onto characters, and getting those characters into their respective positions in the digital set,” he says. “We began NTI while the director was still capturing scenes, but he wanted to see all he could before setting up his shots. We’d never really pulled any textures or lighting into MotionBuilder before, but that was all part of the job on Beowulf. All characters were textured as they would be in a high-end video game. We lit all our scenes, used particle effects for flames, shadow maps, normal maps, anything we could to help the director develop the mood of the scene. The scene was then saved and passed on to DLO.”
Dilo is the colloquial term for Director’s Layout or DLO, the phase of the project that enables the director to review, select, and combine created scenes into his perfect shot. Through MotionBuilder and NTI, Turner and his team provided Zemeckis with completely lit, textured, and character-populated scenes. Turner’s team used an InterSense Motion Tracker system to track a sensor placed on a handheld video camera that would later provide the scene in Zemeckis’ own handheld camera. When the director lays out his shots, they are recorded in real time.
Explaining the value of this system, Turner doesn’t mince words:
“Basically, if you’re a director using motion capture, your physical actor’s performance changes with each and every take,” he says. “Rather than wait for the ‘perfect take,’ the beauty of this system is that you can take the best parts of each live take, combine them, and edit them using the MotionBuilder Story tool, giving the director one perfect take. All of the layout camera work is done postcapture, and the director can experiment until he loves it. And with MotionBuilder, he can do it in real time, viewing and changing his camera completely on the fly.”
Zemeckis Versus Grendel
Asked for an example, Turner goes back to Grendel’s attack on the meadhall:
“When we laid out the scenes for Grendel’s destruction of the meadhall, we really had no idea what the director’s shot would eventually look like,” he explains. “The request was for a scene with from 10 to 100 characters inside the hall, all of whom needed to be edited to have the ability to shift their timing according to the director’s instructions. We would have a file with 150 characters in the scene. We don’t know where the camera is going to go, but the director needs to view all the action in real time before dropping a camera in. We created a perspective camera which allowed the director a bird’s-eye view of the hall. At that point, he’s able to adjust anything that doesn’t work for his camera framing, and all of his camera layout is done in real time.”