DNEG unveils the technological marvels behind ‘Citadel’: VFX supervisor Daniel Baldwin shares insights

International VFX and animation studio DNEG has made waves with its exceptional work on Newton Thomas and Jessica Yu’s highly anticipated spy thriller series, Citadel. Created by Amazon Studios and Russo Brothers’ AGBO, Citadel is Prime Video’s ambitious franchise that promises interconnected stories spanning the globe. 

Leading the charge in creating the essential VFX for Citadel, DNEG has collaborated closely with the series’ unit heads. Their team of talented artists have worked tirelessly to ensure that the action-packed sequences are visually stunning and immersive.

Starring Richard Madden and Priyanka Chopra in the lead roles, Citadel follows the journey of two spies, Mason Kane (Madden) and Nadia Sinh (Chopra). Both characters have had their memories wiped after a near-fatal mission. However, a new and dangerous threat emerges, forcing them to remember their past and save the world from a sinister plot to establish a new world order.

AnimationXpress had the opportunity to speak with Daniel Baldwin, the VFX supervisor at DNEG for Citadel, who provided fascinating insights into the studio’s work on Citadel. In this exclusive Q&A session, Baldwin sheds light on DNEG’s involvement in the series, the challenges they faced, and the innovative techniques used to bring the thrilling visuals to life.

Read on to discover more about the creation of Citadel’s stunning visual effects, the use of virtual production, the challenges of maintaining continuity across different shooting locations and times of day, and the seamless integration of CG assets with live-action performances. 

When did DNEG get involved with the project?
As the main VFX partner, DNEG was involved with Citadel from the very beginning of pre-production in 2020 and worked across the entire six-episode series, offering advice on shooting methodology, providing virtual production support, data capture and on-set supervision, as well as post-production VFX.

How many VFX artists have worked on the series and which locations of DNEG have worked on Citadel?
931 DNEG artists worked on the project, across our London and India studios. 

What was the brief from the director in terms of VFX for the series? 

Our brief was to take what had already been shot and turn up the intensity of the action, adding scope and scale to what’s seen on screen. Without a doubt the number one instruction was realism. They wanted the action to feel grounded in reality, for the fights to be gritty and brutal, and for the physics to always feel believable. We spent many hours gathering and studying vehicle references and understanding the stunt work performed on the show so that our VFX and animation always felt plausible. We bolstered that with extensive reference shoots on location, using helicopters and drones where it was necessary to shoot plates from the required angles and in the relevant lighting conditions, so even when our shots were full CG, our compositing team had a reference to match to. This resulted in the final images being as photoreal as possible.

Was virtual production used in the series? Could you explain in detail?
Yes, virtual production played a key role in the Mountain Fortress sequence in episode three. Kane’s escape from the exploding fortress was shot on the backlot at Bray Studios in the UK, on a relatively small set with multiple blue screens. Our job was to put him on top of the Alborz Mountains, surrounded by a vast snowy landscape and CG burning fortress. To help the DoP and director visualise the scale of the environment on set, DNEG virtual production worked alongside CG supervisor, Tom Hales, to generate a real-time version of the environment that was loaded onto interactive iPads. This gave them the ability to frame shots and select lenses to create the ideal composition of the environment, with the actors on the blue screen stage in front of it. This was a crucial part of the shot design that we were able to commit to on set at the time of the shooting, rather than having to manipulate and experiment in post. It put the ability to frame the CG shots back in the hands of the DoP and camera department on set, which helped to maintain visual consistency throughout the sequence.

Baldwin explained to us the creation and breakdown of the following shots and sequences in the series:

Citadel US Headquarters
DNEG’s art department created the concepts for Citadel HQ. It is constructed into the face of the Utah desert cliffs, which we rebuilt from helicopter photogrammetry photography. We were instructed to design a building that reflected the nature of the Citadel organisation – modern, cutting edge but functional. The space also had to feel big enough to encompass the helicopter landing pad, as well as the interior HQ set seen in numerous shots throughout the series.

The train crash sequence

DNEG took this sequence from previs to final, with work including the full CG environment, camera and vehicle animation, fire, flames, smoke, destruction and water FX. Nirmaljeet Singh oversaw the design and build of the 10-carriage bullet train and our environment team, led by Joakim Flaten and Tom Brown, created over 15 km of full CG landscape that was lit for three different times of day as the sequence progresses – starting with the magic hour and transitioning through sunset into dusk.

The inspiration for the train journey was taken from the Flam Railway in Norway, a similarly picturesque journey through the fjords (a narrow portion of the sea between cliffs or steep slopes) of Myrdal and Aurland. I spent a week there along with a data capture team from DNEG, shooting over 20 km of landscape; photographing, LiDAR scanning and using a drone to fully capture the environment to be rebuilt in CG using scattering tools and rules emulating nature.

We gathered extensive footage and built a library of real train disasters, as well as references from car and truck crashes and explosions. This helped to keep the animation physically plausible and gave our FX team, led by Vladimir Popovic and Markus Drayss, a goal to ensure the results were as realistic as possible.

Alborz mountainous region
The Alborz mountains environment was a combination of the scanned UK set build and various mountains around the ski resort of Bovec on the border of Slovenia and Italy. I was out there supervising the second unit portion of the ski chase sequence and captured the environment for us to rebuild in CG using a combination of LiDAR scanning, drone photogrammetry and a six-camera hydra array rig attached to a helicopter. The mountains were sculpted around a brutalist CG building inspired by concepts from the art department. We added a layer of snow, scattered vegetation and rocks, a surrounding wall, a communication antenna that was scanned from the set build, and multiple CG snow vehicles that were replicas of action vehicles on set. The Mountain Fortress environment was complete with a 360-degree wrap-around DMP of distant mountains and sky, also captured on location in Slovenia.

What was a challenging sequence to work on?
The most demanding task for us was definitely selling the huge scope of the Mountain Fortress and the subsequent ski chase in episode three. The biggest challenge was seamlessly tying together the multiple locations and shooting conditions into one coherent sequence, as well as building on the second unit’s practical action and integrating it with VFX additions and enhancements to move the story on and add to the visual production value but not distract from the in-camera stunts and gritty realism captured on the shoot.

On screen, the whole sequence takes place in the Alborz mountains, but in reality, the sequence was shot over four locations across the UK and Europe. Our environment and comp team, led by Kris Anderson, did a fantastic job of combining full CG environments, aerial plates and DMP to tie the whole sequence together and keep the action on screen flowing from the top of the mountains to the forest down below.

We used Digi-doubles and CG FX to enhance the action and story-telling, but we wanted to be sure the VFX didn’t distract the audience or do a disservice to the amazing stunts and practical SFX directed by Simon Crane on set. We were constantly referencing real second unit photography and elements to build a library of our own animation and CG FX bullet hits and explosions to be placed into shots next to existing stunt performers or SFX, and then balanced in comp to give the same photoreal look.

Were there any new techniques or technologies used for Citadel?
We really pushed our 360-degree involvement in this project. That is to say, we weren’t just brought on after the shoot to deliver the VFX but were involved in helping to design the world of Citadel from the very start. We worked with the showrunners to create vehicles, weapons and gadgets from concept to final, previs whole sequences, plan best practices on shoots, and enhance the storytelling. The previously mentioned virtual production impact on the shoot is definitely where I felt we used the most cutting-edge technologies. Being able to bring the VFX process forward onto the set really helps to keep the visual look consistent and involve the other departments as much as possible in the creative process.

There were digi-doubles used for certain sequences. How did you go about achieving those shots?
We have an advanced digi-double pipeline at DNEG, so the photo-realism of our assets is always at the cutting edge of industry standards, which all starts with controlled and thorough data capture of the actors on set. For the facial animation of our digi-doubles, we collaborated with Di4D to capture action-specific performances from the actors, which were then used to drive our CG facial rig. This removed the need for creative interpretation or adjustment in post, which can often lead to undesirable results, and instead allowed for the actor’s performance to be directed on set and preserved exactly from the shoot. Full body performance was a combination of key framed animation using reference from the stunt performers on set and scene-specific motion capture from our own animation artists, supervised by Nigel Rafter, in the DNEG studio.  

Any particular shot which made you think beyond what’s capable?
There are a number of shots in episode three during the fortress escape and ski-chase that are full CG. Environment, digi-doubles, vehicles, FX and all. It can be so easy for these sorts of shots to jump out of a sequence and distract the audience from the flow of the action if they don’t sit in the same photoreal world. I’m very proud to watch the entire sequence without ever having that feeling. I think the DNEG teams were very successful in achieving our goal of grounding the VFX in the same reality as the principal photography.

What were the things you had to keep in mind to ensure smooth continuity of shots, especially those that took place at different times of the day?
Any typical outdoor shoot throws up different weather and lighting conditions, so sunlight intensity and shadow direction are almost always inconsistent throughout a sequence. However, this is reality and we’re so used to seeing it on screen, without necessarily realising it. The key for us in VFX was to always start by matching the reality available on the plate. Even if we were only keeping the foreground element of Kane running on the ground, our lighting artists, supervised by Adam Baldwin, would begin by matching the sun’s height, direction and intensity to his element. This creates many shot-specific lighting set-ups, but that’s crucial to maintaining photo realism later in the process. Only once the CG is balanced to the plate in each individual shot would we start to balance the comps across the whole sequence. At that point we would agree on the master look for the sequence, and then grade the whole image in individual shots to match the same colour temperature and light intensities.

How was your experience working with the various unit heads in the team?
My personal experience during the shoot was collaborating with Simon Crane as second unit director and then working with Wesley Froud as production VFX supervisor in post. Simon is a very experienced action director and knows exactly what he wants. He sets out to capture as much as possible in camera with stunts and SFX, so my job was to support his vision by adding visual effects in the circumstances where shooting restrictions wouldn’t allow us to capture everything we wanted on the day. We built huge libraries of reference data captured while on location so that our CG additions always matched the visual look that was shot on camera. Wesley and I are old industry friends, so being able to work so closely with him on Citadel was a real pleasure. His briefs were always detailed and clear, but he also allowed room for the DNEG team and me to be creative and explore new ideas, which made for a very collaborative process. I really feel like the result is the most successful combination of ideas possible from the directors, showrunners and the DNEG VFX teams in both London and India, led by Shivesh Gehlot, William Carley, and E S Banu Kiran.

You have worked on plenty of action films. How different was it working on a series that is building a spy universe, compared to the movies that you have worked on over the years?
Many of the films I’ve worked on previously already had an established look from previous movies in the same franchise, or from the comic book or computer game visuals that inspired them. The really exciting thing about Citadel is that it’s totally original and this is the first season and step in creating a whole new universe, so we were responsible for setting that look for the subsequent series to follow. This allowed for a lot more creative exploration of ideas. We would often discuss how sci-fi we should go, and how much we should push the technological advancements in the gadgets, weapons and vehicles they use among other things. I think we found a really successful result, giving the audience the feeling that Citadel is an advanced organisation on the cutting edge of technology and military advancements but still grounded in our present-day reality. They feel like the sort of organisation that could be operating somewhere in the world right now, outside of and above government restrictions, and that’s exactly what they are.

The good viewership of the first season of Citadel has forced the makers to create a second season. It is confirmed that Joe Russo will be directing all the episodes of season two. The audience will have to wait and watch how the makers are going to pan this spy universe.