In 2013, Guy Williams explained to us about Weta Digital‘s work on THE HUNGER GAMES: CATCHING FIRE. Today he talks about his new collaboration with VFX supervisor Christopher Townsend, this time for GUARDIANS OF THE GALAXY VOL 2.
How did you and Weta Digital get involved on this show?
We were invited to bid on the show and for a variety of reasons – we enjoy working with the creative team at Marvel, we loved the first film, we loved the script for the new one – so we pursued the opportunity to work on this project.
How was the collaboration with director James Gunn and VFX Supervisor Christopher Townsend?
This was our first time working with James and we loved it. He is a focused director who has that awesome quality of protecting the core of what he is creating. He is always thinking about what the film needs and steering every conversation to the best way to get there. This was our second time working with Chris and like last time, it was a blast. He is a creative soul that seeks to pull the best out of people.
What was their approaches and expectations about the visual effects?
Marvel knew what they wanted on this film. The Guardians universe is a very colourful wild pladace to be and any VFX that were to populate it had to build upon that idea.
What are the sequences made by Weta Digital?
Weta worked on the third act battle mostly. Specifically, we picked up the action in the palace once Ego reveals to Quill that he gave Quill’s mother cancer. From that point on, it is mostly our work (with the major exception of Rocket and Groot’s excursion into the core of the soul chamber). We go all the way until Yondu and Quill rise above the planet as it is destroyed behind them. Weta also worked on the Death of Tulk (the poor henchmen who gets ejected from the airlock and dies frozen in space) and the escape from the mutineers (where Yondu detaches the Quadrant from the Eclector as the Eclector is destroyed).
How did you organize the work at Weta Digital?
I was lucky to have three great VFX Supes, besides me, on the show, Kevin Smith, Daniel Macarin and Charlie Tait. We split the show into three large chunks and each one of them lead a full production team in tackling the work. We kept the animation side all together and I had the ever-talented David Clayton leading that effort as our Animation Supervisor.
The interior of the Planet Ego is really massive. How did you design this environment?
It was quickly apparent to us that the vast majority of our efforts would need to go into the build for the planet hollow. Not only does the third act take place inside the hollow, it moves around constantly and we knew it would require a large continuous volume that could hold up to a hero level at many different locations. We knew that we had to commit to building it as a proper 3D space so we started by referencing the artwork of Hal Tenny (an amazing fractal artist). The filmmakers had a few of his works that they really liked and wanted to base the planet build on. We picked up their designs and worked on them with the filmmakers to create a fully realized environment. This early design round was hugely helpful for us as we got to think about how we would build it at the same time we were designing it. From here, the hard part began… more on that later…
What was the real size of the sets for this sequence?
The physical sets that the live action was shot on were about 15 meters. The final digital platform the action takes place on ranged from 80-120 meters (it changed from setup to setup to indicate the action was moving around to different platforms). The planet hollow that the platform is in was a bit larger, it was 8km across. As I mentioned before, we had a few places the action took place that we had to do extra prep for. In addition to the hero platform area, there was also the self-chamber in the middle that had a bunch of action around it and an area we called the Lower Rift, which is where Gamora and Nebula fall to.
Which references and indications did you received for this environment?
We received artwork from the production as well as the previs. The artwork was designed by Marvel’s team and heavily based on the works of Hal Tenny (as stated before). We were also lucky enough to get some of his math to base our work on. The previs was excellent in defining the intent on the action but unfortunately it wasn’t based on a continuous environment so we were only able to use it as a visual guide.
Can you explain in detail about its creation?
Ok, this is the tricky part. We laid out our best plan (called creatively, Plan A) that had the planet being built as a fractal that we could control. This program would spit out a low resolution model for lighters and animators to use but the high resolution model would be resolution free and created at render time to meet the needs of the current raster frame. This plan just wasn’t going to happen as the code turned out to be impossible to write in the time we had. We thought this might be the case and had secretly been hypothesizing a Plan B that would allow us to use some of the code from Plan A to build the tricky parts and then flesh it out with hand-modelled structures. We started running into difficulties with Plan B (turns out this is quite hard to write) so we quickly came up with Plan C which was to gather a bunch of our talented modellers onto it and just brute force it out. The Models Department came back to us and told us that it was unbuildable. As you can guess, there was no Plan D, so we doubled down on Plan B again (and in the end the models team came up with some tricks to help build it also).
So, why was it so hard? A fractal is an amazing thing, in that it has infinite detail. The closer you get, the more resolution it creates. This infinite detail is impossible to model, you have to make a choice as to when the detail can visually stop which sounds ok, but remember, the Hollow is 8km across and we get close to a lot of the structures in it so the apparent detail is very high. The next issue is that fractals are driven by formulas and are extremely precise in how they behave. They don’t care what you think, feel or want… they do what they do. If I use a fractal to create an image and I LOVE the image, but want to change one small thing about it, there is no control for that. We needed a model that captured the detail of mathematical fractals but gave us the usual film style control of deciding the shape of every little detail – and don’t get me started on what needs to happen once you like how it looks, but you need to warp it, age it and do a variety of other things to make it feel real and organic!
So, how did we get there if it wasn’t possible? Well, nothing is really impossible in the world of VFX, you just need smart people and time. We were lucky there (at least with the smart people, time is never on your side in VFX). First, the code group from Plan A had made some great progress on creating the most complex structure in the Hollow, the Sierpinski Gasket. We needed these beautiful towers to have insane detail but due to their role in the hollow (they are the long arms reaching across spaces, the support structures if you will) we needed to be able to control them completely. The code group made a tool where we could define a curve to be the center-line and another curve to be the edge and it would loft a shape and then sphere-pack it with mathematically proper spheres that we could then boolean out to get the right detail. The models team were able to get the Boolean approach to handle huge numbers of objects and we were off to the races. Models also came up with the idea of scouting actual fractals and then lifting the models out of Mandlebulb. In the process, we ran into a resolution problem where the most complex model we could spit out was about three orders of detail too simple for what we needed. In a moment of brilliance, one of the modellers decided to render an orbit around the scouted-out area and then use photogrammetry to lift the model out. The result was the level of detail we needed for some detail pieces. Models also used this to lift other fractals for the wall shapes. They then took all the parts and started stitching them together to line up with the hero layout we had. They then knocked out spans and broke towers apart to give the entire volume a sense of untold age. After all of this, we had a hugely detailed model that looked like the approved concepts. But we still weren’t good on detail. I don’t know if I have said this before but that frigging hollow is 8km across and we are often within 10-100 meters of its structures. We needed more. We came up with the idea of modelling coral and bone-like structures to grow on the walls and towers to further give a sense of detail. We called this growth plaque (kind of like the gunk on your teeth if you never brush). We placed insane numbers of these coral structures everywhere. This was only possible due to some of the great work by our in-house rendering and instancing teams. At the end of the day, we had over 361 billion de-instanced polygons in the render just for the Hollow.
This environment is getting destroyed. Can you explain in detail about this massive destruction?
Oooff. Ok, so on top of the 361 billion polys, we then had to add a few more billion in destruction falling from the ceiling all the time, but I’m get ahead of myself…
There was a variety of destruction types going on in the chamber. Jason Lazaroff and Cornelius Porzig, our FX team supervisors came up with a bunch of excellent solutions for giving us some large-scale destruction on often extremely-heavy models. For instance, to break a spire and have the end fall, you needed to voxelize the spire so that you could shatter it at the break point. Then, they took the plaque and re-bound it to the newly broken parts. Then they ran a fluid sim to get all the dust that comes off these large structure as they fell as well as secondary sims for all of the finer grit and gravel from the break point being transferred into the fluid sim so that it would fall properly.
We also took some creative license. The whole place is pretty dark so we would add the odd explosion here and there (rocks shouldn’t blow up, but hey). We used all the neural pathways that were built into every part of every model as a sort of detonation cord that would cook off in the final destruction. We created blue fireballs that erupted along the pathways and then would cool to proper oranges and reds. The final look was kind of fun.
How did you manage so many elements?
Our layout team uses a database called Atlas to make sure that everything was in its proper place (to be clear, they managed what went where and when and Atlas locked that concept down and passed it to the artist).
Did you use procedural tools to help these destructions?
Not for the chamber, we were keen to make sure that the destruction felt large and relate-able so we kept it more grounded in traditional destruction.
The Guardians are in the middle of all this destruction. How did you created their digi-doubles?
Some of the digital doubles were created by other companies and we ingested them into our pipeline. The sharing tools for that sort of thing are so good nowadays that this was actually one of the easiest parts of the project. Weta created the digital doubles for Ego, Mantis, Nebula, Tulk and Yondu.
Can you tell us more about Rocket and Baby Groot?
We were lucky enough to get a few nice scenes with Rocket and Groot in them. We worked on the confrontation between Rocket and Yondu on the bridge of the Quadrant before they head to Ego and also the scene where Rocket says goodbye to Yondu as the Hollow is destroyed (I am glossing over all of the Rocket action scenes in the Hollow, but we did those as well). We received the Rocket asset from Framestore (passed to us using the Alembic format). Framestore also supplied excellent documents outlining the shading and how they used the textures. In the past, hair has been something that isn’t shared (or at least not easily and successfully), but with Alembic and an Alembic reader in our hair tools, we could get a hair-to-hair match with the Framestore model.
As for the animation, David Clayton and his team tackled the problem with gusto. It was fun working through the challenge of capturing the essence of who Rocket really is.
Let’s talk about Ego that can morph and change his size. How did you handle this aspect?
The destruction and subsequent regrowth of Ego took a bit of time to figure out. I’ll hit them one at a time…
For the destruction of Ego (when Quill is triggered and just goes all John Wick on him) we started by figuring out how to destroy a human body but keep it inside of a PG rating. We ended up using the fractal motif to help confuse the edges of destruction so that we could still blow chunks out of flesh and bone but keep it somewhat clean. Once that was figured out, we then removed the areas of destruction and used fractals to fill back in the edges so that there was an alien quality to it. We did every single hole and then revealed them over time so we didn’t have to do a specific sim per shot. Add in some chunks being blown out of the hole and the celestial cage and we were good.
For the re-growth, we did a similar thing but we did a sim per shot so that we could animate the growth out. We grew 3D fractals at the edge of growth and then meshed it all back into a single renderable surface so it would seem like the skin was growing in a fractal way.
How did you created the digi-double for Kurt Russell?
The onset team captured the likeness of Kurt with both standard photography as well as a photogrammetry booth. We used the turn over model and textured the Kurt digital double. For the facial, we snuck in a quick FACs session with Kurt right at the end of the shoot. This was key to getting his expressions just right. Kurt’s digital double was used extensively in the destruction and regrowth so we knew it had to be high spec.
During this shot, Ego is talking. How did you manage his face animation?
We built the digital double to a very high spec and made sure that we captured a good FAC session with the actor. This allowed us to use the digital face more than would be commonly done.
What was the main challenge with the Ego character?
It should have been the fractal regrowth he did. That was high-concept and was supposed to be a technical challenge, but we had an amazing FX group working on it and they made it seem easy (I know it wasn’t… they worked really hard, but as we viewed their work, they made it seem easy). The hard part was the celestial being (the blue light cage version of Ego). Creatively, this went back and forth trying to find the right speed and character for all of the pulses.
Let’s talk a bit more about the FX with the destruction of the Eclector. Can you explain in details about its destruction?
The destruction of the Eclector was achieved using large scale fluid sims layered with other destruction sim (fireballs flying out, debris and so on). Another thing we did was run a sim on the skin of the ship so that as each large explosion cooks off, you see a ripple travel through the skin of the ship around the hole.
How did you created the various explosions and flames?
We used our in-house Synapse tools for building fluid sims and merging them together. We then rendered all of this in Manuka, our in-house renderer. It has the advantage of letting everything (even a fluid sim) participate in the lighting. This let the explosions not only light the side of the ship properly, but also block the light from other explosions.
Which sequence was the most complicated to create and why?
This is almost too hard to answer. The truth is that about half of the work we did was insanely hard. Usually, you have a couple of key beats that require some extra effort, but on this show, it felt like every other shot was like that. The Hollow build was hard, the destruction of said Hollow was hard, we had to blow Ego up one chunk at a time and then have him talk as he transforms into the Hoff. The list goes on and on.
Was there a shot or a sequence that prevented you from sleep?
About half of them.
What is your best memory on this show?
Sorry, but this is a cheese-ball answer coming up… Working with the amazing artist at Weta. It isn’t just my favourite memory from the show, but why I love working here. I get to be constantly amazed at the creativity and passion of so many skilled artists.
How long have you worked on this show?
Just under a year now.
What was the size of your team?
Hard to say because various people come on and off the show. The number of people that was on the show at some point was 490.
What is your next project?
I don’t think I can answer this yet. Let’s just say, watch this space.
A big thanks for your time.
// WANT TO KNOW MORE?
Weta Digital: Dedicated page about GUARDIANS OF THE GALAXY VOL. 2 on Weta Digital website.
© Vincent Frei – The Art of VFX – 2017