Davy Jones stars as the protagonist in the second installment of the Pirates series. He is completely CGI and everything about him is so believable it’s crazy! Of course the team responsible for this had to be none other than Industrial Light and Magic.
The production shot real actors on set and digitally replaced them. In order to do this, each actor was scanned and modelled. They wore a motion capture suit which enable them to be replaced in post production. ILM was unable to rely on traditional MoCap or hand animation as there were multiple issues. It had to be done in special studios with multiple cameras and the cameras and tracking markers are special expensive equipment used only in a calibrated environment. Also, the data needed to be cleaned up tremendously as the data stream has both noise and errors. The whole process is complex to set up, and it’s also expensive and highly specialized therefore it wasn’t used. ILM created an innovative new system called Imocap and that allow onset and on location motion capture to elicit the most believable look and performance possible out of actor Bill Nighy.
He wore a pair of gray ‘pajamas’ with reference dots placed around the suit and his face, and his performance was captured entirely on set as he interacted with other actors. This improves the performance of the other actors as they would have someone ‘real’ to interact with, and it also gave the animators a highly detailed reference.
Being ILM, they made a breakthrough with Imocap when they only had to film with a single onset film camera instead of multiple cameras when using MoCap. A single camera removes the many restrictions motion capture process gives. With Imocap, motion capture could be done on set. The approach is to model the actor’s range of motion and then they used an elaborate system to fit the range of possible motions the actor could do, to the data from the single camera source.
Besides Imocap, the other challenges ILM faced with the character of Davy Jones was his 46 flopping tentacles. ILM wanted the tentacles’ curling and movement to reflect Davy Jones’ mood, not just lifelessly bob around, but they didn’t want an animator to have to manually manipulate each and every one frame-by-frame so to solve this, their programmers added a sort of inter-tentacle motor to automatically move them around. Mathematical expressions and/or keyframe motion fed to motors in the joints between the cylinders making up Davy Jones’ 46 tentacles caused them to bend, curl, writhe, and perform in life-like ways. “Stiction” kept the tentacles from sliding.
As the computer knows what the actor’s limbs could do from any one frame to the next, it can ignore a lot of mathematical possibilities and add to the solution. Once the solution is constrained by this virtual range of possible motion, a single camera can produce a very powerful motion capture data stream. While the motion capture system worked extremely well, the lip sync was not done this way and instead hand animated.
For the tentacles, an articulated rigid body dynamics engine was utilized to achieve the desired look. Each tentacle was built as a chain of rigid bodies, and the articulated point joints served as a connection between the rigid bodies. This simulation was performed independently of all other simulations, and the results were placed back on an animation rig that would eventually drive a separate flesh simulation.