What is your background?
My background in VFX is in 2D compositing. Prior to VFX I worked in editorial on factual entertainment for a short time and have always had an interest in all aspects of film making from music and sound design to prop making.
How did you get involved on this show?
I started on the project while production was still shooting. The Knowhere sequence had just been shot and prep work was being done for the Edinburgh sequence. DNEG were awarded work on these sequences and I began liaising with production and planning for data capture for that sequence, before meeting the team when they came to Edinburgh for the shoot.
What was your feeling of being in the MCU?
On the production side it’s a great big family. The team worked on many of the films and the environment is very professional and friendly. They want to make the best film they can and want to encourage everyone to produce their best work. The studios are very supportive and respectful of VFX as it is a very important part of their story telling process.
There is a style in the films that has been built up over the last 10 years that has flexibility within it to incorporate new ideas and progress whilst maintaining a continuity with the Marvel universe. Getting up to speed with this involved analysing and studying the films most relevant to our work. To get an idea of the Russo brothers’ style I also re watched and dissected WINTER SOLIDER and CIVIL WAR.
How was the collaboration with directors Russo Brothers and VFX Supervisor Dan DeLeeuw?
We worked mostly with Dan DeLeeuw who has a very good handle on what the Brothers wanted, having worked with them since WINTER SOLDIER. He has a massive amount of knowledge about the whole Marvel universe and characters, and a fantastic experience of the industry, so reviews with him were always very informative and encouraging.
We had regular feedback from the directors from early on in the post production process. So via Dan the show was guided towards their vision and it always felt like there was a clear path forward.
As the project ramped up Dan’s time got more and more precious and we worked with additional VFX supervisor Mårten Larsson. He was our point of contact through to the end of the show. Again he brought another great voice to the table and helped maintain consistent contact and feedback from the production so we were never left waiting for answers.
How did you organize the work with your VFX Producer?
Melinka Thompson-Godoy and myself split the work up in terms of type of work and sequence so that teams had continuity in the work they were doing.
A build team lead by James Benson created the hard body assets for the show.
An environment team lead by Dan Pastore looked after designing and creating environments across the show including matte paintings, projection work and fully 3D environments
Markus Schmidt was our creature supervisor, supervising the build rigging and deformation work on all of the characters and digi doubles.
Shot finishing work was big enough to be broken into two teams, the Edinburgh sequence was supervised by Dave Lee and Malcolm Neily looking after composting and lighting respectively.
The Knowhere and Wakanda sequences were grouped together and supervised by Milos Milosovic for compositing and Alison Wortman for lighting.
We had a fantastic team in Mumbai lead by producer Asha Jospeh and DFX supervisor Bharat De. They handled a few sequences of their own and a sequence of set extensions inside the Benatar, as well as other sequences and sections of work.
What are the sequences made by DNEG?
DNEG worked on 5 sequences across the film.
The Edinburgh sequence, where Proxima and Corvus attack Vision and Scarlet Witch. This sequence included extensive character work for Proxima and Corvus along with set extensions and CG environments, digi doubles, FX magic and destruction and Vision augmentations.
The scene in the jungle in Wakanda where Vision and Captain America fight Corvus, followed by the scene where Wanda attempts to destroy Vision with her magic.
The scene in Knowhere, where Thanos tricks Gamora, for which we did the environment work including fully CG establishing shots and ship animation for the Benatar.
Avengers compound interior shots, including set extensions hologram displays and Vision augmentations.
Benatar Cockpit interiors including set extension work.
How did you work with the art department for the Black Order?
The process was one of passing the baton from the art department to us. We were handed packages of concept and design work that we then developed further. We received material in relation to the Black Order but also other aspects of design.
We had a great meeting with production designer Charlie Wood early on where he talked through the work his team had done on various environments and showed various concepts we also received CAD models and 3D files that were useful as a starting point for various builds.
Can you explain in detail about the design and the creation of the Black Order?
DNEG’s work covered Proxima Midnight and Corvus Glaive. In terms of design our main contribution was realising Proxima Midnight to completion.
DFX supervisor Aharon Bourland led this process in conjunction with Markus Schmidt. When we started, work had already been done by Framestore who had done initial work in pre-production which had brought the character to a certain place. Some of the key elements were in places such as the broad strokes of the costume design but she was quite far removed from what ended up on screen.
Some beautiful maquettes were designed for the black order characters; these were the ones revealed at D23. This was our starting point to work from and we began building the character from scratch based on these maquettes.
The costume design stayed fairly true to the maquettes with tweaks to the materials and extra details added. We did explore physiques for the character and ended up with a slightly more athletic build than the maquettes, however it was her face that the directors and studio wanted to explore the most.
We attacked this in a number of ways. On a technical level, we ended up doing a mix of 3D sculpts in Z Brush and paint overs to come up with ideas. A few artists gave different takes on the character to explore different ideas. The challenge was to get a character who was relatable but also had some alien features. Some of the decisions were quite small, such as how red her eyes were to be but we also experimented more extreme alien variants of the facial structure although the design ended coming round to being more human.
We also played with the make-up design and discussed whether her horns and black make up were either part of her head or more of a cowl or helmet, which is how it seems to be in some of the comics. We ended up with the logic that the horns are physically part of her but that she has oily war paint on her forehead.
For the human facial features the process followed along with the casting for the character. We used a 3D scan of Monique Ganderton who was the key on set presence for Proxima as a starting point, which we then added more alien features too such as the stronger cheek bones and her horns. Carrie Coon was then cast to give Proxima’s vocal performance and we experimented with combining aspects of Carrie’s face. Although her face maintained most of the features we had built up through the earlier design process Carrie ultimately drove the facial performance and we received a set of ILM’s Medusa data for Carrie that we used for final face modelling and blend shapes for her performance.
Which character was the most challenging to create and why?
Proxima was difficult from a design perspective, incorporating the various requirements of the character, her implied strength, femininity and alien nature definitely provided a challenge.
From a shot execution point of view, Corvus had challenges of his own as he had a very complex costume with a hood and cape with rough strands of cloth this was complex to simulate. FX supervisor Adrian Thomson and cloth lead Christopher Kilshaw did a great job handling this across the shots, in many cases the cloth needed to be art directed so that it didn’t distract from the fighting action that was happening.
Can you tell us more about the rigging and animation?
The main rigging of the characters was fairly straightforward. Facial rigging was more complex and was based on the Medusa data we received for both Carrie Coon and Michael Shaw. The expressions for these actors had to be translated to the characters as they have quite different faces so this was no simple task. Ben Wiggs was DNEG’s animation supervisor and led a team looking after around 200 animation shots. Post animation we had a creature pass team adding additional muscle definition and animation along with the cloth and hair simulations.
Did you receive specific indications and references for the animation?
We received lots of references for animation, although there was a fair amount of exploration involved. The process generally went as follow:
Onset action was filmed almost always with a performer, either the main actor for example Michael Shaw who provided the facial performance and voice for Corvus, or a stunt performer for more extreme actions, as a stand in for the CG character.
In some cases this also provided the facial performance and the actors were equipped with facial capture cameras by TechnoProps to record their onset performance. This provided calibrated stereo footage with consistent infrared lighting and face markers.
Editorial used the onset performance to drive their edit, and postviz was done by The Third Floor to provide editorial with something to cut with. In the most part the postviz was heavily driven by the on set action and was often rotomated based on what was referred to as ‘Faux Cap’ suits. These were not full motion capture suits, just costumes designed to be easy to track.
Once an initial edit was done, additional material was shot on a motion capture stage providing great quality data to be used in production and also facial performances that were used for the dialogue.
DNEG also created additional motion capture for many of the shots using a Moven suit.
The onset performances and the postviz generally provided a guide for the timing and then the motion capture drove the movement. Key frame animation work was then done to clean up the motion capture, tweak timings, adjust eye lines and poses and to tighten up interaction with the actors. Key poses were often tweaked to be more dramatic, comic book poses. And improve the fluidity of the movement.
Can you tell us more about the face animations?
Facial performances were driven by the facial capture footage, shot on set and during the dialogue and motion capture stages. The footage was used as a basis for the performance but often needed tweaking based largely on the difference between the actors playing the characters and the alien characters who were physically different. In some cases subtle performances had to be exaggerated and more extreme ones toned down based upon what read well.
How did you handle their various skins shaders and textures?
To develop the skin textures we used a selection of high res skin scans to make sure the detail was there at a high level. Make-up maps were painted by hand and work was done to have obvious separation between the grease paint like makeup and the fleshy skin.
Creature lead Stefan Mayr and Aharon Bourland worked on the shading. We referenced some generic light stage material to establish a realistic base for the skin shading even though the character was an ‘alien’ we wanted the look to be grounded so we gathered as much real life reference as possible.
We used Isotropix Clarisse for all of the shading a lighting work on the show.
The characters have many different skin colors. How does that affect your lighting work?
This did create problems in lighting as maintaining consistency was a challenge and having the right skin tone was key to getting enough of a fleshy look to the skin for the characters.
We had originally designed Proxima to be wearing powder white makeup over her skin. It had a quite striking look in the night sequences but was less successful in the daylight sequences, so we moved towards a more fleshy skin tone.
Can you tell us more about the eye work?
We built Proxima’s eyes to be physically the same as real human eyes. They were built with separate geometry for the cornea, the lens and the iris and an accurate corneal bulge. Proxima’s eyes are meant to have some technology in them giving them a red glow in the iris. Getting a natural looking balance between a realistic looking iris and a glowing tech eye was quite tricky.
The fight is taking place in different parts of Edinburgh. How did you create these places?
I took a small team from DNEG for the shoot in Edinburgh to capture extensive amounts of on set photography and be able to rebuild any of the shooting locations. We ended up building parts of St Giles Cathedral and the surrounding parts of the Royal Mile and buildings, so that we were able to add destruction and extend blue screen shots. By and large a lot of background plates were shot to put our CG characters into with only a few shots that needed to be created fully digitally. We did end up building a fairly large section of the Edinburgh train station to allow for fully digital shots where Falcon is flying around and the shots could not be achieved on set.
Can you tell us more about Scarlet Witch powers?
Scarlet Witch’s magic is something that has developed over the series, the idea being that when she first gets her powers in AGE OF ULTRON they are fairly weak and she learns to control them more through CIVIL WAR and up until we see her in INFINITY WAR. Visually DNEG had done work on Wanda’s magic on CIVIL WAR and we used this as a starting point with the brief that she should have become ‘full comic book Scarlet Witch’ as Dan put it. So we studied a number of comic book references where her magic has some very strong graphic components of arcs that form around and lead away from her hands. We wanted to try to incorporate these with the established look and maintain hints of the stylised arcs without getting into something that looked too graphic so we incorporated additional FX passes some of which were designed to be camera facing like in the comics. These were then balanced in to not become too overpowering.
How did you handle all the FX work?
Adrian Thompson was our FX supervisor and led a team who worked in Houdini to create the various FX elements on the show. This was quite varied as it ranged from destruction elements such as stonework that gets smashed to the ether effects in Knowhere.
The most challenging FX task by far was the transformation of Mantis into Ribbons. Postviz was done early on that established the idea but we found that getting an effect that had to hit certain marks and maintained a human form whilst also feeling like naturally flowing ribbons was incredibly hard. The end result was to have a more art direct-able layer that controlled the overall form of the ribbons and then running a cloth simulation on top of this to allow for the ribbons to fall and drape naturally over their surroundings.
Which sequence or shot was the most complicated to create and why?
Other than Mantis turning into ribbons, the shot that was the most complicated was the transition of Knowhere from deserted to destroyed. The environment alone was extremely heavy to render and had to effectively be rendered twice for both the before and after states, the shot involved a camera transition between plates and also a design process to establish the way in which the transition happens. The end result became quite pulled back and simplified to help tell the story but we went through a lot of development to explore different options for the effect.
Is there something specific that gives you some really short nights?
We were awarded the section of the Wakanda sequence that we worked on fairly close to delivery so the volume of work on that sequence to be done in the time we had was quite daunting. A lot of the work was reliant on accurate camera and body tracking which is always time consuming so there were not always ways to speed up the processes.
What is your favourite shot or sequence?
I’m really proud of the Edinburgh sequence. It was great to work on a sequence with the location playing itself, and I enjoyed the chance to spend some time in Edinburgh which is a city I love.
What is your best memory on this show?
I have lots of good memories on this show, we had a great team at DNEG and had always enjoyed our calls catching up with the production team in LA and our team in Mumbai so we generally had a really good sense of camaraderie on the show, so my best memory is the fantastic team all round.
How long have you worked on this show?
Since February 2017.
What’s the VFX shots count?
We had 496 shots in the final film.
What was the size of your team?
Just under 800 people worked on the show at various points during the project across the three sites.
What are the four movies that gave you the passion for cinema?
WALLACE AND GROMIT – A GRAND DAY OUT
MONTY PYTHON AND THE HOLY GRAIL
A big thanks for your time.
// WANT TO KNOW MORE?
DNEG: Dedicated page about AVENGERS: INFINITY WAR on DNEG website.
© Vincent Frei – The Art of VFX – 2018