How did you choose this project amongst the Marvel movies?
I previously worked on both the first CAPTAIN AMERICA and IRON MAN 3, so getting to work with the other characters in the Avengers team was an exciting prospect. I also worked on Ang Lee’s HULK, back in 2003, and found the big green guy one of the most challenging characters to bring to film; I really relished the prospect of having another go at him.
Can you tell us more about your collaboration with director Joss Whedon?
Joss is not only a great screenwriter, but also very visual as a Director; with his his experience in comic books, and his passion with the Marvel Universe, he’s incredibly knowledge with this source material, so often had very clear ideas of what he wanted.
What was his approach about the visual effects?
He understands the power of visual effects, but also the advantages of shooting things practically. We’d always discuss the best way to solve a problem. Having photographed reference can be very useful, but it was my job to determine if shooting that extra pass was imperative, and whether doing so would inadvertently affect the rest of the shooting schedule. He wanted to make sure that if I said we needed something, then we really did.
How did you work with Marvel Studios Executive Producer Victoria Alonso?
Victoria has an overview of all the projects going on at Marvel, and at the same time, manages to be pretty hands on. She juggles a lot of different things, and manages to do so deftly, while remaining passionate about all the films. In post production, I generally presented the shots up for discussion and for final, to Joss and to the Marvel folk, Kevin Feige, Lou D’Esposito, Jeremy Latcham, and of course to Victoria; she’s got a great eye for detail.
Can you describe one of your typical day during pre-prod, shooting and post?
The great thing is that every day is different, so there’s not really a typical day….
In Pre-Production, we start by breaking down the script treatment, and then the various versions of the scripts. We have to figure out what the nature of the work is, about how many shots there are, how we’re going to do it, and much we think it’ll cost. We work on previs, and animatics, to help inform what the work is, and how best to create it. If we’re not doing that, then we’re in meetings with Joss and the other heads of the departments planning the shoot, talking about who does what, location scouting and figuring out any technicalities of complex issues.
During shooting, I’m on set every day. My job is to watch everything that is going on and make sure that we get what we need, for the future. I usually stand next to the Director, making sure that I hear any bits of information that I’m going to need to refer to in post. As so much of these types of films are crafted in post production, we’re involved in the vast majority of shots in the film. I’m in charge of a team of VFX folks on set: the data wranglers record all the camera and set information on every shot, as well as marking up the green screens etc; photographers and Lidar specialists capture the set, so that it can be digitally recreated later, if need be, and PAs, whose job it is to make sure that everyone has what they need to work efficiently and get the job done. The VFX Producer (in the UK it was Ron Ames) is my partner, and splits their time between set and the production offices, continuing to work on the logistics of the upcoming shoot.
In Post, we take edited sequences and break them down, figuring out what we need. We often do postvis, in order to help tell the story, inform the cut, and pictorially figure out what the effect should be, in rough form. Then, along with the VFX Producer (in Post it was Ryan Stafford) we decide which VFX companies are potentially the best fit, who can do the work and then bid the work out to them. Once a company has won the bid, we award it, give a more specific creative brief on the sequence, turn over all the data and the background plates, then start to review the shots as they are submitted.
However, often, we may be planning an upcoming shoot, while we’re shooting something else and reviewing other shots which are already in post, so it’s not exactly a linear day at the office!
How did you approach this big project?
One bit at a time. If you look at everything all at once, its overwhelming. You have to have choose the right team on the production side and then have faith in the them, and do the same with the VFX facilities and the incredibly talented artists and production folk who are crafting the shots.
What was the most complicated sequence to filmed and to completed in post?
There were so many. Piecing together the end battle, shooting in various locations and piecing them all together was a real challenge, as was the complex Party Fight sequence, where we’re trying to follow each individual Avenger, in a difficult environment. However, the hardest individual shot was right at the beginning of the film, which is over a minute long, and stitches together multiple plates and CG environments as we travel though a forest, catching heroic moments with each of the main characters, as their introduction in the film.
How did you collect all the necessary on-set informations for the various vendors?
We have our own onset data wranglers, and sometimes the bigger VFX companies send their crews too. We generally LIDARed every set and photographed it thoroughly.
You worked closely with ILM and Trixter. Can you tell us more about it and the collaboration with their VFX Supervisors?
I worked at ILM, as an artist, for over a decade, so I know them well, and even though this is the first time I’ve worked with them in this capacity, I really was working with old friends. I knew most of the people from before, and had a very close working relationship with both Ben Snow, their VFX Supervisor, and Marc Chu, their Animation Supervisor. It was very useful as we had a shorthand, and were definitely on the same page. I knew what they were capable of, in terms of quality, quantity and scope, and also knew that they were passionate about pushing the envelope.
I’ve worked with Trixter many times before, and I love their approach and the aesthetic they bring to their shots. Alessandro Cioffi, their VFX Supervisor, knows what I want, and the way I like to work, so again, we have a shorthand, which is really useful when we’re all under pressure.
In fact, many of the companies and the people we worked with on AOU, I’d worked with before, including Dneg, Animal Logic, Lola, Method, Framestore, Luma etc, so we were able to find creative solutions more quickly and efficiently.
How did you work with director Joss Whedon on the previs process?
Sometimes Joss gave specific initial direction to Gerardo Ramirez, The Third Floor’s Previs/Postvis Supervisor, sometimes to me, and sometimes to the picture editors, Jeff Ford and Lisa Lassek. I would review some shots individually, but generally they were cut into the scene and presented to Joss that way.
Can you explain in details about the design and the creation of Ultron?
Ultron was initially designed by the VisDev team at Marvel, using various pieces of comic book art as a starting point; the VisDev team are responsible for all initial character designs for the Marvel films. These were then taken to ILM, who started building him out, creating their own concepts, riffing off the sketches we gave them, showing us rough 3D models, doing movement studies etc. From those, the design was refined and the articulation created.
The VFX studios shared assets. How did you manage the continuity and the look for all the shots?
I always want a very open dialogue with all the VFX companies with whom I work; on a film like this, it’s imperative that everyone’s on the same page, so clear communication is always key. We would task one vendor to create the asset, and the look, then hand that over, along with any specific guidelines. Most of the time, with a hard surface assets like Iron Man (ILM) or the Sub Ultrons (Dneg), it’s pretty straight forward to share; others that involved FX work, like Pietro and Wanda (Trixter) meant that we sent Houdini files and recipe guides, along with a variety of looks, so that the other companies could match.
The vendors were all around the world. How did you manage the work progress?
Long hours. We had a great Production team, led in post by VFX Producer Ryan Stafford, with a couple of Production Managers, 5 VFX Coordinators and several PAs, and 3 VFX Editors. We also had an Additional VFX Supervisor, Geoff Baumann, who took a lot of pressure off me being the sole creative bottleneck in the process. Geoff handled several vendors, Dneg, RiseFX, Black Ginger and the In House team, and I took the rest. In the end, I looked at all the shots, from everyone, comparing one to another so that there was consistency between them. So yes, very long hours, and you need a dedicated, passionate and well organized team to get things through the pipeline consistently.
Many of the VFX studios have worked with you on IRON MAN 3. How was this new collaboration with them?
Really good. I love working with new people, but when I get the chance to work with teams with whom I’ve built up good relationships it makes things go really well.
Marvel have announced that the VFX shots number is over 3’000! How does that affect your work?
Well, the film was big. We all knew that going in, so we planned with that in mind.
What was the main challenge and how did you achieve it?
Getting everything done in time. Open communication with everyone is key; people have to know the rules, the standards and what’s going on. And no one can be too afraid to share things.
Was there a shot or a sequence that prevented you from sleep?
Many. Well, no, nothing stopped me from sleeping, but there were many that were very challenging… Ultron Prime and Vision were both characters that were really difficult to create; Ultron because we were trying to make the most advanced robot ever that could perform humanistically but remain a rigid metal character. Vision was creatively difficult because I was aiming for something which was surreally perfect, strangely synthetic but clearly still human; I wanted to dip down into the Uncanny Valley from the other side, from a photographed actor, into a slightly odd CG look, but retain the humanistic traits, soul and performance of our actor, Paul Bettany; I think Lola did a terrific job for the face, and Framestore the body. There are several shots which are very long, as I’v mentioned before, and long shots are always tough both creatively and logistically. And the End Battle was something which we settled on, from a storytelling point of view, very very late in the day, so that was pretty stressful.
What did you keep from this experience?
That in order to succeed, everyone needs to be accountable for their work. If so, we can get through.
How long have you worked on this show?
About 22 months.
How many shots have you done?
In the end, the film has over 2800 shots in it, but there are several hundred more that didn’t make the final cut.
What was the size of your team?
We had about 15 people including our VFX Editors, led by George McCarthy.
What is your next project?
Watch this space!
A big thanks for your time.
© Vincent Frei – The Art of VFX – 2015