In 2016, Dan DeLeeuw explained his work on CAPTAIN AMERICA: CIVIL WAR. Today, he tells us about the challenges on the epic new AVENGERS film.
How was this new collaboration with directors Russo Brothers?
Working with the Russo’s is always a pleasure. The amount of collaboration on their films is unprecedented. They always have a saying that “The best idea wins”. This gives us and and VFX houses the opportunity to suggest ideas for shots and animation. For instance, during the Hulk Buster Cull Obsidian fight, Hulkbuster and Cull originally ran at each other in a way that didn’t feel quite real – Cull left himself open for a punch that a trained fighter would never allow. I asked ILM to come up with a new attack for Hulk Buster. What they came up with was an awesome coup de gras with Hulk Buster blasting off into the air and smashing down his fist onto Cull. It was a great animation and ended up in the film.
The movie is really massive. Does that change their approaches and expectations about the visual effects?
By the nature of the story we actually ran towards doing more with VFX than we ever had before. This movie wasn’t grounded like WINTER SOLDIER or CIVIL WAR. It involved eight to ten foot tall aliens and battles on other planets. We knew the movie was going to be massive and had to plan accordingly.
How did you organize the work with your VFX Producer Jen Underdahl?
Our plan was to split the movie into individual sequences that would be awarded to separate VFX houses. Each house would get a sequence that would tell a complete narrative. This way each house could maintain the look and feel of the sequence while minimizing the need for shared shots.
How did you split your work with the Marvel Studios VFX Supervisors?
We knew going in that we couldn’t be everywhere at the same time. In addition to this movie being massive, we where shooting AVENGERS 3 and 4 simultaneously. I took an overall view of the project working with the directors to achieve their vision for the film. Swen Gillberg and Marten Larsson were on set for shooting and transitioned into post production as shooting completed.
Can you tell us more about the previz and postviz work?
Previz was handled by The Third Floor and headed by Gerardo Ramirez. In general we sit down and decide where best to apply our resources between story boards, stunt-viz, and Previz. The general rule was: Anything that involved more complex fighting would be handled by stunts. Anything that involved CG characters or required techviz was completed by Previz. Storyboards would give us awesome ideas for the sequence as well as providing coverage for simpler scenes. The trick with this movie – again because we were shooting two movies together – is that Previz would transition to Postviz but as the second movie ramped up artists would go back to Previz. Essentially we were Previz’ing and Postviz’ing at the same time. We had about 35 artists working at the height of production.
What was your approach with a movie featuring so many important full CG characters?
We wanted to get started on development as early as possible. Framestore started on the Black Order and Digital Domain and Weta Digital started on Thanos. We knew we had to get Thanos going as quickly as possible. So much of the movie depended on him as a character everyone needed to see how much range we could achieve with his acting. The directors needed to have the confidence that Thanos could carry the movie. We shot a test with Josh Brolin early in production where he explored the character while in a mocap suit and a helmet cam. We ended using out takes for the Thanos test where Josh was very introspective. We had seen Thanos as a brooding character in other Marvel films but we hadn’t really pushed how far he could go. The early test show how much we could do with the character. It showed that would could take exactly what Brolin did on stage and reproduce it with the CG character. In addition, we didn’t want our tests to be a sterile demonstration of what we could achieve. Framestore did awesome character studies that were fully rendered vignettes with mini stories. Some of the ideas from the vignettes actually made it into the movie.
How did you work with the art department for the new characters?
Marvel’s Look Dev department headed by Ryan Meinerding designed all the characters. It is truly some of the best artwork you will ever see. We would get illustrations that showed us texture and coloring for the costumes and characters skin. In addition Ryan’s team would sculpt the character in Z-Brush and the VFX house would build their geometry from Look-Dev’s sculpts.
With so many CG characters, how did you collect all the important informations on-set?
We tried to capture as much motion on set as possible. It was something that I and Jeff Ford, our editor, pushed for as strongly as possible. A characters performance grows out of the interaction on set with the other actors. To achieve this we used a back pack rig for eye lines and where possible we would place Josh on a platform so that the other actors could look straight into the eyes of the 8 foot tall Thanos. In addition to mocap, we scanned and photo textured every set, and collect HDR bubbles to help with lighting.
Let’s talk about the bad guys, Thanos and the Black Order. Can you explain in detail about their design and their creation?
All the characters in the film went through the same process. Each character was based on the sculpt we received from the Look Dev department. We would scan every actor’s face to start as a base shape for the facial animation. In addition we would use Medusa which is a high resolution facial capture system created by the Disney Research Group. Medusa would give us pore level detail for all the actors faces. The trick was when selecting key shapes we needed get all the VFX houses to agree on a set of shapes. With so many characters at multiple houses this became a bit of a Tower of Babel problem. Even thought they characters were alien we would merge some of the actor’s skin detail into their alien counterparts. Each actor was equipped with a mocap suit and a helmet cam. Editorial would cut with footage either from witness cameras or the motion picture cameras. Once the cut was signed off, we would order up the data from Profile and the send it to the vendors. It was a lot of moving parts that Swen, Lisa Marra, and Leah Sargent managed beautifully!
What was the main challenge with Thanos?
Making sure we honored Josh Brolin’s performance was our main goal and challenge. We would constantly refer back to the helmet camera footage to make sure we were on point. Even the slightest variation would change the intent of the shot. Weta and Digital Domain did an amazing job with matching Brolin’s performance. There is scene where Thanos has to cry in relation to his daughter Gamora. The scene plays beautifully. We end up sympathizing for a character we should actually hate.
How was updated the Thanos model and design since his first appearance?
Early in production Ryan Meinerding, the head of Marvel’s Look Dev department, asked what they could do to help VFX with the creation of Thanos. I asked if we could make him look more like Josh Brolin. The more similarities to their faces the more of Josh’s performance would read. Thanos still has the out of proportion chin, head, and body but we were able to get just enough Brolin into the model to really push the boundaries of what you could do with a CG character.
Beside Thanos, which CG character did give you the most headache?
The new Iron Man suit presented a unique challenge. His suit was made from liquid metal and was more reflective that what we had done before. You expect an Iron Man suit to be rigid because of the nature of armor. In our case the new suit was so streamlined it didn’t give us the same areas in we would cheat the deformations. In this case we spent months making sure that you couldn’t see the new suit stretch. You buy it with the Iron-Spider but when an Iron Man suit stretches it throws you off. In addition the reflective suit required us to light it more like a car in a car commercial. We ended up having to hang digital silks in our lighting rigs to get the proper look and feel for the armor.
Can you tell us how you choose the various VFX vendors?
For INFINITY WAR, all of the shots were complex. There were only a handful of simple A over B comps. All the other shots were complex CG sets with multiple CG characters. We needed to pick vendors based on their ability to handle shot complexity, number, and animation. As a result we were able to work with some of the best VFX houses in the business.
How did you split the work amongst these vendors?
We tried to keep the shared shots to a minimum. Most vendors shared characters but each where responsible for their own sequences.
Can you tell us more about your collaboration with their VFX supervisors?
We were lucky to work with Supervisors that had worked with us before. We already had a shorthand and the Supervisors were up to speed on the Russo brothers style and tastes. In addition we got to work with a great new group of supervisors we hope to work with in the future. Each vendor supervisor would come to set for the shooting of the their shots. This is always helpful because you get a real insight into the photography. Bringing them to set also allowed us to bring them to the art department and gave us the ability to sit down with our Production Designer Charlie Wood and discuss the look of the film!
The vendors are all around the world. How did you proceed to follow their work?
We would spend very very very long days in the screening room.
How did you handle the challenge for the look and animation continuity of the many CG characters amongst the various VFX vendors?
The motion capture provided the basis for the majority of the characters. When would animate we always had the comic books for the source material for poses and looks.
Where was filmed the different parts of the movie?
Main unit and second unit shot in Georgia here in the US and in Edinburgh, Scotland. The effects plate unit shot in Alaska, Brazil, Argentina, and the Philippines.
Which sequence or shot was the most complicated to create and why?
I think all of Reel 7 was the most complex. It’s a long reel where Captain America, Tony Stark, and Thor are all deeply involved in their plot lines and respective fights. It was a combination of a completely CG Nidavellir with CG Rocket and Groot, CG Set Extension Wakanda with thousands of Outriders, and Wakandans, and a CG Titan with CG Thanos, Iron Man, and Spider-Man. It was a huge amount of work. Of all the shots in the movie we didn’t touch 86!
Is there something specific that gives you some really short nights?
Reel 7.
What is your favorite shot or sequence?
I think as a sequence it’s Reel 7. For shot it’s a very simple shot. There is a shot with Thanos arguing with Gamora, and Josh did this wonderful performance where he tilts his head back in exasperation. It’s such a great character moment where the toughest guy in the Galaxy – that can take on all the Avengers without much trouble – finally gets frustrated when he can’t get his daughter to agree.
What is your best memory on this show?
Showing Josh Brolin the first test of Thanos and seeing how excited his was for the character.
How long have you worked on this show?
Two years!
What’s the VFX shots count?
2680 shots.
What was the size of your on-set team?
It was a combination of Supervisors, Production Managers, Data Wranglers, Texture Photographers, Scanning Booth, Texture Booth, Mocap, Facial Cap – So basically a huge crew!
A big thanks for your time.
© Vincent Frei – The Art of VFX – 2018