Guy Williams had talked to us here about the visual effects of Weta Digital on THE AVENGERS a year ago. He is back with this new Marvel production and explains the many challenges on this film.

Aaron Gilman started his career in animation more than 10 years ago. He worked for studios such as Meteor Studios and Tippett Studio on projects like MATRIX REVOLUTIONS, HELLBOY, CONSTANTINE or JOURNEY TO THE CENTER OF THE EARTH. He then joined Weta Digital to work on AVATAR. After that he worked on films such as THE ADVENTURES OF TINTIN, THE AVENGERS and THE HOBBIT: AN UNEXPECTED JOURNEY.


How was the collaboration with director Shane Black?
It was great. He has a good sense of what is going to reinforce the story.

What was his approach with the visual effects?
Shane and the gang at Marvel are very keen on making sure the story comes first. That doesn’t mean that the action can at all suffer (laughs). It just means that the beautiful action scenes have to reinforce the story telling. It is so easy to forget the story and get caught up in the visuals and it is nice to have a client that can adroitly navigate those waters.

Can you tell us more about your collaboration with Production VFX Supervisor Christopher Townsend?
Chris is an excellent supervisor and an even better human being. He helped to make the show so fun to work on.

What have you done on this show?
Weta Digital worked on the third act battle, the seaport battle. We did most of the shots starting from when Tony and Rhodey arrive at the seaport to when the last suit explodes in the sky.

What was your approach for the final battle?
For us, getting this show done on time meant we had to leverage economy of scale as much as we could. We knew we could not cut any corners visually. We spent much of our time on the show in setup and preparation so we could build the best engine possible for getting the shots done. As complex as the seaport was as a build, it was just one seaport. This meant we could put all of our effort into making it as detailed and pipelined as possible and the effort would be payed off over all of our shots.

The battle involved many Iron Man suits. How did you created so many armors?
We started with the 12 hero suits from Digital Domain. The models they made were excellent and we had to do little to get them into our pipeline. We did ROM’s (Range Of Motion tests) on the suits and then put the suits back in modeling to fix any issues that the motion tests uncovered (intersecting plates, limited motion due to design and so on). Once we had the 12 hero suits modeled, we started in on the background suits. These were the 24 other suits built out of the parts of the hero suits. We also attached information to every part so that it knew where it came from and could reference its textures from the hero builds. This allowed us to spend most of our time getting the 12 heros to work and then let the system use the legacy attributes to pass the info down to the background suits. This meant that we were able to get the background suits done in a very short amount of time but they still worked as good as a hero suit.

Can you tell us more about their rigging and rendering?
The challenge with rigging the Iron Man suits is that you have to have a rig that makes it look like the plates can all move rigidly, sliding over each other. The truth is that this causes a lot of intersecting plates and limited motion. To fix this, you need a rig that deftly uses deformation of the plates to allow the freedom needed. This has to be subtle though in that it still needs to feel like the plates are solid and rigid. Our guys took the lessons they learned on THE AVENGERS and applied them to IRON MAN 3. The suits came together real fast and we had to do very little shot specific cleanup.

For the rendering, we rewrote the shading pipeline for the Iron Man suits from scratch. We used a layered car paint shader like we had on THE AVENGERS but we rewrote it to properly deal with the travel of light between the layers. We also rewrote the individual layers to handle the light transfer in the layer correctly. For instance, the emulsion layer used to be a reflection layer with a tint. Now, we use a proper absorption layer so the color is handled correctly. This means that under blue light, we get the proper color as a result. Also, it opens up some room for some new tricks. We can adjust the thickness of the emulsion layer to get variation in the richness of the color, thicker paint would go more of a deep red. We could use tricks like this to add more detail and complexity to the suit without having to paint complex maps.

Can you tell us more about the action choreography?
This was a tricky one. There are two levels to the action. The first is what you are focusing on and the second is the background action. The foreground fighting and choreography is pretty straight forward. It was the background action that took a while to get right. Animators are trained to make action as impressive as possible and bring it to the forefront. We found that early on, we were having too many interesting things happening in the background action, the viewers eye was being drawn away from the story telling and into the background. We had to work on ways to make the background action detailed and interesting but also make it work in such a way that it didn’t detract from the foreground performance. To do this, we didn’t change the specific action as much as we changed it’s framing and timing. We made sure that it didn’t always take place in the negative spaces in frame. We also avoided traveling through the center of frame. Lastly, we made sure the background action wasn’t perfectly timed to the cut. In general, we allowed the action to be messy in the background. We allowed it to be behind the foreground and clip the ends of the cut more often.

How did you created the huge environment of the seaport and the cranes?
Lots of hard work from a talented group of people. We decided early on that we would not skimp on detail. We knew we had to use the seaport for our all CG shots so we made sure to set the spec for its build very high. The teams didn’t shy away from it at all. They rose to the challenge and created some great assets. We used some tricks in the lighting to get the seaport to render but in general, we accepted the long render times as a cost to getting the detail.

What were the real size of the set?
The main set was the gantry crane. It was about 300 feet tall. Our CG build included all of the cranes on the seaport as well as the entire seaport yard. We built the set all the way back to the edge of the yard. All of the shipping containers and buildings and other set props were built. This allowed us to put the camera wherever we wanted and move it however we wanted.

Can you tell us more about the impressive destruction of the seaport and the cranes?
The crane destruction was handled as a two-pass rigid constraint sim. We drove the first pass with animation to set the overall timing and then let dynamic forces take over. The first pass was for the coarse animation of the falling crane. Once we were happy with this, we could run the second pass of the rigid constraint sim to give us all of the detail on the crane. This two pass system allowed us to finesse the overall speed and weight of the crane rapidly and then apply that directly to the final result.

How did you create the digital doubles?
To create the digital doubles, we started with the scans of the actors and the extensive photo reference. We used these to build highly detailed versions of the actors, knowing that we would need to use them often and close to camera. We payed special attention to the Robert Downey Jr double and the Guy Pierce double. We used our Tissue system to deal with the torso’s and necks of the characters so we could get the best result possible.

Can you explain in details about the creation of the Extremis effect?
We knew that we would have a large number of shots with the extremis effect. To make this viable in the time we had left, we had to make sure that our process of creating the effect could be pipelined, the less done by hand, the better. We created internal geometry in the characters for the muscles, veins and other internal bits. We used a raytracer to fire rays into the skin, and then a raymarcher on the results of the rays to deal with the diffusion and absorption of light. We were able to add any number of theoretical lights for the raymarcher to get the sense of a glow coming from the core of the character. These lights could be animated along with textures on the veins and skin to get a sense of light traveling inside of the chest and arms. This result, along with some helper masks output as AOV’s from renderman, were passed to comp and dialed in from there. By having a good solid starting point that was consistent, we were able to cut out a lot of time wastage on the process.

What references and indications did you received from Shane Black for the Extremis effects?
The art department at Marvel did some amazing concepts for what the effect would look like and how it would evolve. Some of the other companies working on the project had already started on some test also. This made for a nice rich foundation for us to build upon.

How did you create the animation under the skin?
We had theoretical lights that could be moved around and animated in size/shape/intensity. By using a large number of these lights, we were able to get a nice sense of large glowy waves of energy moving around under the skin.

Can you tell us more about the use of deep compositing for these effects?
We use deep compositing for all of our work. For the extremis, it meant that all of the layering was handled in the proper depth without having to render special layering mattes or holdouts.

How did you manage the challenge of matchmoving and face replacement?
Killian was the only one we did a lot of facial extremis for. We had already built a good facial system for him because of his change to lava god. All we had to do was to track the face and then match the animation of the face to get the effect under the skin to track good. For the body, we didn’t bother tracking the effect in. For the most part, we just replaced the parts of his body that were extremis. Voila! Perfect match of the effect to the skin every time if you do it this way (laughs).

What was the biggest challenge on this project and how did you achieve it?
The project itself was one big challenge. We had some considerably hard work to do (crane destruction, extremis, all CG shots, final Killian “lava god”) and scant little time to prep and do it in. It was through careful planning by the production team and also by the shear passion of the artists that we were able to get this all done.

Was there a shot or a sequence that prevented you from sleep?
Not really, all of it came together as planned. No part of the plan ever fell apart.

What do you keep from this experience?
I have been doing this for a while now, watching artists create amazing work. But it still floors me, especially on a show like this, where the artists surpass the challenging expectations put upon them and create truly stellar work. I am humbled by them. It is why I love this job.?

How long have you worked on this film?
We started in October but the plates didn’t come in in full force until early December, so 5 1/2 months total but only about 3 1/2 months from getting the plates in.

How many shots have you done?
We delivered 502 shots.

What was the size of your team?
Around 600 people contributed to the work we did.

What is your next project?
I am getting to work with Janek Sirrs again on THE HUNGER GAMES: CATCHING FIRE.


What is your background?
I got into animation relatively late in life. At the age of 26 I enrolled myself in the 3D Animation Program at the Vancouver Film School. At 27 I received my first job as a animator with Meteor Studios in Montreal, working on Discovery Channel dinosaur projects. 3 years later I was working at Tippett Studio in Berkeley, California, animating on my first feature films. For personal reasons, my family moved back to Montreal where I began the computer games portion of my career. I worked as a Animation Director with Ubisoft and Electronic Arts, and would eventually return to film. I later decided to move my family out of Montreal to New Zealand, where I joined Weta as a Senior Animator on AVATAR. It’s now been 5 years at Weta, and since arriving I’ve been given some wonderful opportunities in the Animation Department, including a chance to Supervise animation on such cool projects as THE AVENGERS and IRON MAN 3.

Can you tell us more about your collaboration with Production VFX Supervisor Christopher Townsend?
Chris is a very talented guy that had a strong vision concerning the VFX we did on IRON MAN 3. I’ve actually had the pleasure to work with Chris before, back on JOURNEY TO THE CENTER OF THE EARTH 3D in Montreal. Chris trusted us implicitly on the project, and the creative collaboration between Marvel and Weta Digital was strong, as it has been in the past.

What have you done on this show?
I was the Animation Supervisor. My job was to help the animation team achieve Marvel’s vision of the 3rd act battle in the film.

What was your approach for the final battle?
We had numerous challenges on the project, such as designing the aerial battle, shooting tons of cool motion capture performances for various fight sequences and hero vignettes, and animating the mechanics of a crane being destroyed and collapsing. But perhaps the greatest challenge was dealing with the technical and creative complexities of transforming the suits so Tony could jump in and out of them.

How have you enhanced your Iron Man pipeline since THE AVENGERS?
In order for us to achieve the various suit transformations throughout the 3rd act battle, we needed to optimize our work flow to allow the animators to cut up the suits over top of their animation blocking. We created a series of proprietary tools and a puppet system we called the Guide Rig. With a newly developed UI giving animators all the tools they needed to stay as creative as possible, we were able to cut geometry panels, create custom animation controls on the fly, mirror our work from one side to the other, and ultimately design transformation concepts directly on top of our puppet blocking. The Guide Rig system also allowed the animator to change the functionality of limbs on the puppet, so they could be separated from the character during shots where limbs needed to be removed. Animators could also tag individual panels on the suit to determine their damage continuity throughout the rest of the sequence. If a panel was damaged, the animator could decide if it would never be seen again and thus flag in our pipeline so any animator working on future shots would never need to deal with it, or it could be flagged to appear in all future shots for continuity purposes.

Can you tell us more about the action choreography?
Animation spent a lot of time designing choreography for the 3rd act battle. We had hundreds of shots featuring dozens of suits flying around in the background, so it was important we stage each shot with a very clear view towards creating a realistic style to the combat. We noticed if we tried to stage the background performances in the same way we stage hero animations, there was a tendency to draw the eye to regions of the background and thus distract the viewer from the main intention of the shot. We also had a number of hero vignettes featuring never before seen suits. Each vignette was a key framed or motion captured performance, depending on the type of motion required, and in some instances a hybrid of the two. We also shot a lot of motion capture for the fight choreography between Ironman suits and mercenaries.

What was the biggest challenge on this project and how did you achieve it?
Developing the Guide Rig was our biggest challenge. Through very close communication and constant meetings with our Modeling and Creature departments, we worked tirelessly to develop the tools and establish a workflow that allowed the animations to go through the pipeline as efficiently as possible. Abs Jahromi, one of our Creature gurus, was instrumental in the design of the Guide Rig and its tools.

Was there a shot or a sequence that prevented you from sleep?
I had dreams nearly every night for about 4 weeks in a row where Iron Man’s boot thrusters would be firing in my face. It was incredibly annoying.

What do you keep from this experience?
I am amazed by the collaborative experience at Weta. We bonded together under a very difficult deadline, and all of the departments worked cooperatively to achieve a creative vision for the show, but also problem solve pipeline and technical related issues that made delivering the shots on time possible. The bonding experience that happens when artists need to come together in a pinch, never ceases to amaze me. This show was particularly special because so much worked was stacked against us, and we pulled it off with stunning results.

How long have you worked on this film?
4 months.

How many shots have you done?
Animation completed 422 shots.

What was the size of your team?
We ranged from as little as 5 animators coming off the Christmas break, to 40 animators by mid-March.

What is your next project?
I am currently animating on The Hobbit: An Unexpected Journey.

What are the four movies that gives you the passion for cinema?
When I look back on what was achieved on AVATAR, it gives me a huge hunger to continue working on cutting edge VFX projects. AVATAR was a major inspiration, even more so because I had the opportunity to be involved in it. Perhaps my favorite movie in the world is Martin Scorsese’s THE LAST TEMPTATION OF CHRIST. I can’t even count how many times I’ve seen that film. As an animator, I was heavily inspired by the work on JURASSIC PARK. And as a child trapped in a man’s body, nothing beats the original KARATE KID (don’t laugh – I watch it at least once a year!).

A big thanks for your time.


Aaron Gilman audio podcast: Audio interview made by Pascal Chappuis during FMX 2013.
Weta Digital: Official website of Weta Digital.

© Vincent Frei – The Art of VFX – 2013

2 Commentaires

  1. Aaron gilman…i’ll follow you all my life…i love the iron man3 movie…great works guys…i know and believe i will animate in WETA DIGITAL someday…and im gonna shake you hands someday mr.Aaron….

  2. Nice work, great mind, and good visual solution, all r know as digital visual motion solution in the coming time. WETA IS one of my favorite vfx studio. have a great moment.


S'il vous plaît entrez votre commentaire!
S'il vous plaît entrez votre nom ici