Award-winning video game developer Supermassive Games has earned a reputation for creating some of the most immersive gaming experiences ever made, in part thanks to a heavy emphasis on building from real performances that bring its digital characters to life. For its latest title, The Quarry, it went further than ever before and turned to Oscar-winning VFX powerhouse Digital Domain to help create one of the most visually immersive, photorealistic games of all time. But to get there, the VFX studio first had to expand and grow its award-winning tools, and invent a new one along the way.
“Right from the start we were blown away by Supermassive Games’ ambition, and we knew our experience with feature films put us in a unique position to help. We started with the same tools we use to create movies and episodics, then adapted them for game development. Basically, we came up with a new, faster way to animate photorealistic digital characters,” said Digital Domain creative director and VFX supervisor Aruna Inversin.
The Quarry is an interactive survival-horror experience fueled by the performances of an award-winning, ensemble cast. Photorealistic digital versions of several well-known Hollywood veterans react throughout the story with the same level of nuance and emotion seen in a live-action film, but with the added bonus of having the players control the circumstances and actions. Will they choose to investigate the sounds in the distance or hide? Will they help out the others or try to survive on their own? There are several outcomes and survival is far from guaranteed.
The Quarry game director Will Byles said, “It was important for us to make every movement and reaction as realistic as possible, to create an immersive experience where players truly feel the weight of their decisions and connect to the characters. We wanted to get as close to the likeness of the actors as possible, both in terms of the super high detail scans and the subtleties and nuances of their performances. With its history of award-winning VFX and pioneering digital human work, Digital Domain was the best choice for this.”
Expanding on the idea of The Quarry as an interactive cinematic experience, there is even a Movie Mode option that allows users to tweak the personality traits and actions of each character, and then sit back and watch it all play out. Additional filters can further enhance the experience by adding an 8mm-style Indie Horror film grain, a retro VHS 80s Horror look and a black-and-white Classic Horror filter.
The process
To help bring photorealistic facial animation to a new level, the VFX studio famous for its work on Spider-Man: No Way Home, Stranger Things, WandaVision and countless others, began by conducting a series of facial scans of every cast member. That gave the team at Digital Domain an accurate template to work from, as well as a comprehensive library of facial shapes and expressions for each performer. The data was then sent on to the artists at Supermassive Games, who created the looks for the in-game characters.
Next, it was time for the performances. Each cast member reported to Digital Domain’s performance capture stage in LA, where they slipped on a full mocap suit and a facial capture rig. They then performed their roles individually or in small groups, acting out several versions of the story to create multiple branching narratives, including dozens of death scenes for each character.
Game developers are no stranger to using mocap and facial rigs to help create digital characters, but with help from Digital Domain, The Quarry went several steps further than most. In total, the cast underwent an incredible 42 days of motion capture, resulting in 32 hours of captured footage.
With the digital characters from Supermassive Games ready and the live-action performances recorded, the combined data was then fed through Digital Domain’s Masquerade facial capture system, a tool initially developed for Thanos in Avengers: Infinity War. Since its critically acclaimed debut, the system has evolved for use in projects beyond just feature films.
Masquerade 2.0 can analyze live-action footage, combine that with the individual and unique library of that performer’s facial shapes and then blend it all together with the CG character. Utilizing machine learning, it can then create a photorealistic CG version of the performance and fully animate it, all in record time. For The Quarry, that ensured that each performance seen in the game includes every facial movement, every emotive reaction and every bit of nuance from the talented cast.
“Masquerade is an incredibly powerful system, but ultimately it’s the performances that really make The Quarry stand out. We wanted to ensure that our tools amplified what the cast brought, so each performance down to the subtlest movement is as true to the live-action performance as possible,” said Digital Domain senior producer Paul Pianezza.
In order to create digital characters that could be edited in real-time, Digital Domain first needed to address two common challenges with facial capture that typically require post-capture clean-up by artists: eye-tracking and helmet stabilization. First, Digital Domain started with an open-source eye-tracking solution known as Gaze ML and heavily modified it over the course of three years. By using machine learning to identify the unique nature of each cast member’s eyes and pairing it with a high frame rate head-mounted camera, Gaze ML was able to produce better accuracy in tracking, and improve the look of the digital eyes.
Next, Digital Domain created a new way to stabilize the data from the headcam. Facial-tracking solutions typically require the wearer to remain relatively immobile to avoid shaking and blurring. But using a new, proprietary technique developed in-house, machine learning algorithms in Masquerade 2.0 were able to analyze the captured footage and compensate for any jostling. That gave the cast the freedom to move around normally and unrestricted, even run and jump while wearing a head rig to elevate their performances.
Introducing Chatterbox
Tying it all together, Digital Domain went one step further and introduced a new technology called Chatterbox, a tool initially developed by its internal Digital Human Group to help lay the groundwork for the future of autonomous humans. Powered by machine learning algorithms, Chatterbox can analyze live-action facial expressions unique to each performer, then determine the best possible options to seamlessly alter facial expressions, while still maintaining quality and lifelike movements. By streamlining the amount of data and focusing on only the best facial options, the digital characters can then be uploaded and edited directly in a game engine.
When paired with Masquerade 2.0 for The Quarry, Chatterbox was able to access the library of facial expressions from each cast member, opening up a host of possibilities for Supermassive Games. Once the data was running in Unreal Engine, the developers were able to see the characters in polished digital environments and make real-time adjustments to the performances as needed, all based on each performer’s actual unique facial movements and mannerisms. That eliminated the need for costly reshoots, while still ensuring the most accurate and faithful performances possible.
Overall, less than 0.5 per cent, only 27 out of a staggering 4,500 shots deemed game-quality, required minor post-production alterations by an artist. In total, Digital Domain rendered over 250 million frames for The Quarry, helping to create a new high water mark for high-fidelity, interactive cinematics.
Digital Domain global president John Fragomeni commented, “Creating high-quality, photorealistic visual effects used to be unique to feature films, but the technology has evolved, and given our decades of history and ongoing success, we are in a unique position to bring professional-quality VFX to any screen. We have some of the best artists in the world, so there’s really no limit to where and how we employ our tools, and we were honoured to work with Supermassive Games to help bring The Quarry to market.”
The Quarry will be available on 10 June in both physical and digital formats for PlayStation 4, PlayStation 5, Xbox One, Xbox Series X|S and digitally on Windows PC via Steam.