While it may seem extremely cool to don a helmet with a head-mounted camera and shoot, motion capture is a technique that involves an entirely different level of complexity. James Cameron’s Avatar beautifully used the technology for the CGI created characters of the Na’vi tribe; and the Planet of the Apes series exploited the technique successfully with Andy Serkis’ Ceaser becoming the most memorable character in the film.
With the amount of research, training and skilled execution that comes with using this technology, it is an equal amount of dare to execute it for a TV show. And this dare has been taken up by Abhimanyu Singh’s Contiloe Pictures for their upcoming mythological TV show Vighnaharta Ganesh, making it the first TV show in India to use motion capture.
Why Motion Capture (mocap)
“Whenever Ganpati has been shown on screen, you will see makeup done to convert his human face to an elephant face,” said Contiloe Pictures CEO Abhimanyu Singh. “It looks odd,” he added, and argued that the form of an elephant’s face cannot be done just by applying makeup on human’s face. “That is why we have used mocap.”
The technique is used for the elephant head of Ganesh in the show, and will be seen from the third or fourth episode.
Inception of the Show
“When we were discussing the show with Sony TV, Danish Khan, the EVP and business head said that the unique part about the show should be to replicate Ganpati exactly the way we have seen him,” which got them thinking on how to achieve that. Singh then got the VFX and make up team involved and carried out various tests.
Then began the search for vendors that offered the technology. After a detailed research, Contiloe zeroed in on Faceware Technologies, an innovator in markerless 3D facial motion capture solutions. “We discussed with them, sent the model out, used the data that they had, did a couple of tests and saw that yes, this was happening,” shared Singh, explaining that the R&D procedure took them nearly six months to be able to determine what the technology will offer and how they will leverage it.
Training and Getting Used to the Technique
Contiloe has purchased the equipment and software needed for mocap from Faceware. In addition, experts from Faceware had come down to India to train the team of Vighnaharta Ganesh for three days, on the usage of software and hardware.
The teams hold multiple meetings before the shoot to discuss how to go about the technique for the shots.
Procedure and Execution
The boy who plays Ganesh has to first act at the shoot after which the footage of his final shot is sent to Harsh and Hari, the two men that have been assigned with the execution of the technique. They have a sample script to give them an idea of the expressions they need from the actor.
The actor wears the facial motion capture helmet with the head mounted camera system. A wireless transmitter in the belt connects to a wireless receiver. The receiver is connected to the recorder and monitor present with Harsh and Hari. Through the monitor, they can see the performance as well as get an idea of the framing. When all is checked, they go ahead with the recording.
The final take of the actor is edited and the data is extracted using the Faceware proprietary software for transferring it to 3D. This data is then put on the CG face of Ganesh.
For a 20-minute episode, one full day is needed for recording in mocap. And according to Harsh and Hari, per shot processing takes longer than that. In case of issues faced with the software or additional doubts, Faceware’s online centre comes to aid.
Integrating Mocap Data to the CG Model
The VFX team have the CG model of the elephant head created. This model is rigged and textured to make it look like a human face. “The VFX artists have a 3D CG model of Ganesh ready, to which they apply the mocap data,” mentioned VFX supervisor Deepak SV.
Through mocap, they only get movements of eyes, lips and cheeks. The trunk and other elements of the face have to be animated manually. They have a library of emotions for the CG model of Ganesh as generating expressions on the model is time consuming.
For wider shots, animatronics and prosthetics are used for the trunk and elephant head. CG head with mocap data is used for close-up shots.Nearly 40 artists are working on the head, which in itself is sufficient to explain the complexity of the task.
Experience and Challenges
“In the earlier TV shows where Ganesh has been shown, there was no movement seen in the trunk and ears. Ganesh eyes have been shown as human eyes and not like elephant eyes. Hence, creating a Ganesh that is never seen before has been the greatest challenge,” expressed series director Vaibhav Muthu, who has also directed Maharana Pratap, Sankatmochan Mahabali Hanuman and Maharaja Ranjit Singh’s pilot for Contiloe.
“Translating human emotions to elephant face is the biggest challenge technically,” highlighted Deepak. “Proportions change with the heads.” Also, Vighnaharta Ganesh being a daily show, the deadlines are strict. Timing constraint is also a major challenge. “The central character of the show is in mocap, so we must deliver that much quantity of work with quality.”
Apart from the challenges, the experience for Muthu and Deepak has been a refreshing one. “It is difficult and time consuming but very interesting,” exclaimed Muthu while Deepak said, “It is complex but wonderful.”
Keeping the budget under wraps, the show-helmer Abhimanyu Singh confirmed that the costs are substantial. With about 40 people working on the head of Ganesh, one can imagine the amount of time, effort and resources needed to breathe life into the character.
The mocap technology is still at a nascent stage of use in India, but with Contiloe Pictures breaking barriers and dodging all odds to set a trend, it will only inspire and encourage Indian animation and VFX houses to experiment with it.
