The Avengers – Hulk

Marvel’s The Avengers was a highly anticipated blockbuster and no doubt it was big hit in cinemas when it was released last year. A main character in the movie was the Hulk, played by Mark Ruffalo. In the movie, he turns from a normal human into a buff green shirtless killing machine. Industrial Light and Magic (ILM) was responsible for this CGI and they did such a great job that they were just recently nominated for an Academy Award for Best Visual Effects.

Here’s kinda how they did the digital double. ILM used motion capture to catch the emotions Mark Ruffalo potrayed on screen. Every bit of Hulk stems directly from Mark, from the pores on his skin, to the grey hair of his temples, right down to using a dental mold of Mark’s teeth as a basis for Hulk’s teeth. Their strategy was to work to out rendering and texture issues on the Banner (Hulk before he turns human) digital double until it looked indistinguishable from Mark Ruffalo.

The realism of this digital double is fucking awesome!

OMG WOWZERS!

As the Banner and Hulk shares the same topology, they were able to transfer textures, material settings and the facial library for animation. This gave them a decent base to start working from but with their significantly different proportions, there was a lot of retargeting work that need to be done. They tried to be economical with their poly counts but with Hulk they made a conscious decision that he was going to be extremely dense in his resolution for a better mesh. By working ike this, they never came up short on resolution for all of the close-ups and detailed shape work that was required to represent the anatomy under Hulk’s skin. They then incested in a robust multi-resolution pipeline so that the model was manageable for the artists to work with.

WIP

Here’s an interesting behind the scenes video!

Pirates of the Carribean: Dead Man’s Chest

Davy Jones stars as the protagonist in the second installment of the Pirates series. He is completely CGI and everything about him is so believable it’s crazy! Of course the team responsible for this had to be none other than Industrial Light and Magic.

The production shot real actors on set and digitally replaced them. In order to do this, each actor was scanned and modelled. They wore a motion capture suit which enable them to be replaced in post production. ILM was unable to rely on traditional MoCap or hand animation as there were multiple issues. It had to be done in special studios with multiple cameras and the cameras and tracking markers are special expensive equipment used only in a calibrated environment. Also, the data needed to be cleaned up tremendously as the data stream has both noise and errors. The whole process is complex to set up, and it’s also expensive and highly specialized therefore it wasn’t used. ILM created an innovative new system called Imocap and that allow onset and on location motion capture to elicit the most believable look and performance possible out of actor Bill Nighy.

He wore a pair of gray ‘pajamas’ with reference dots placed around the suit and his face, and his performance was captured entirely on set as he interacted with other actors. This improves the performance of the other actors as they would have someone ‘real’ to interact with, and it also gave the animators a highly detailed reference.

Being ILM, they made a breakthrough with Imocap when they only had to film with a single onset film camera instead of multiple cameras when using MoCap. A single camera removes the many restrictions motion capture process gives. With Imocap, motion capture could be done on set. The approach is to model the actor’s range of motion and then they used an elaborate system to fit the range of possible motions the actor could do, to the data from the single camera source.

Besides Imocap, the other challenges ILM faced with the character of Davy Jones was his 46 flopping tentacles. ILM wanted the tentacles’ curling and movement to reflect Davy Jones’ mood, not just lifelessly bob around, but they didn’t want an animator to have to manually manipulate each and every one frame-by-frame so to solve this, their programmers added a sort of inter-tentacle motor to automatically move them around. Mathematical expressions and/or keyframe motion fed to motors in the joints between the cylinders making up Davy Jones’ 46 tentacles caused them to bend, curl, writhe, and perform in life-like ways. “Stiction” kept the tentacles from sliding.

As the computer knows what the actor’s limbs could do from any one frame to the next, it can ignore a lot of mathematical possibilities and add to the solution. Once the solution is constrained by this virtual range of possible motion, a single camera can produce a very powerful motion capture data stream. While the motion capture system worked extremely well, the lip sync was not done this way and instead hand animated.

For the tentacles, an articulated rigid body dynamics engine was utilized to achieve the desired look. Each tentacle was built as a chain of rigid bodies, and the articulated point joints served as a connection between the rigid bodies. This simulation was performed independently of all other simulations, and the results were placed back on an animation rig that would eventually drive a separate flesh simulation.