Hanzhi Tang has been working in visual effects for over 20 years. He joined Digital Domain in 2003 and has worked on a number of shows including The Golden Compass, Ender’s Game, Thor: Ragnarok and Captain Marvel.
How did you and Digital Domain get involved on this show?
Marvel was clear that it wanted to partner with us from the very beginning of the project. Digital Domain has collaborated many times with Marvel Studios over the past decade on features and now episodics as well, so it made perfect sense to team up again for Black Widow.
How was this new collaboration Overall VFX Supervisor Geoffrey Baumann?
Our team at Digital Domain quickly established an easy and effective rapport with Geoffrey Baumann, Jesse Chisholm and the team at Marvel Studios. This was a complex shoot, and that shorthand helped ensure we were all on the same page throughout. It was truly a collaborative process from day one.
What were their expectations and approach about the visual effects?
Geoff Baumann and the entire Marvel VFX team were very open and communicative from the early stages. Black Widow was always going to be a major undertaking, but we knew the standards would need to be kept at a high level so both Marvel Studios and Digital Domain were happy with it. Everything else, including schedule and the pandemic were secondary.
What are the sequences made by Digital Domain?
We took on the third act work, beginning with the look of the Red Room in the clouds. Then once Natasha and Melena trigger the destruction of the Red Room and the structure begins to break apart, we were responsible for nearly every scene. We continued through the skydive descent all the way to the fight on the ground and the epilogue where the characters say goodbye.
How did you work with the art department for the Red Room?
Marvel Studios’ art department collected a lot of style and material reference, and sent over some fantastic concept paintings of the Red Room. That gave us a great foundation for the lighting style and time of day. It also helped visualise how the Red Room sat in the clouds, and how hidden in the mist it might be.
What kind of references and influences did you receive for it?
The Red Room was inspired by Soviet-era brutalist architecture crossed with the weathering and complexity of an oil rig exposed to the elements. The core of the Red Room was also designed to suggest that there are several communications towers built into the structure, with all kinds of radio and microwave antennas.
Can you explain in detail about the creation of the Red Room?
The Red Room asset had to be built specifically with the intention of being destroyed in our FX pipeline, and to be manageable once we tried to render it in lighting. So even as we started modelling it, we had to plan far ahead and ensure it included closed, destructible geometry. It also needed to include a construction plan that made the Red Room able to be very highly instanced.
We decided very early on that we wanted to use GPU rendering for the Red Room because we wanted to show fully rendered iterations of the destruction on a regular basis for review, and to keep the edit updated as we went along. To aid with that, we built a construction kit at the beam and panel level that was instanced out to form the whole structure. This benefitted both FX simming and the instancing at render time, saving us huge amounts of memory and load time.
The Red Room is flying in the air, how does that affect the lighting work?
The outdoor lighting was mostly driven by “golden hour” sunlight shortly before sunset where the sun was low in the sky. We also had to pay more attention to the bounce light coming off the cloud volumes surrounding the Red Room, and shadowing from mist or clouds above.
How did you enhance the various interior rooms?
We built an entirely CG corridor of cabinets with shattering glass that Natasha had to stumble through as the Red Room is rocked by explosions and deteriorates around her.
During the final battle, the Red Room gets destroyed. How did you handle this challenge?
We approached the destruction of the Red Room at several different scales, combining rigid-body solves of instanced components at the beam and girder level, with finer hero shatters and debris creation in hero customised sections. These were then combined and published as a single layout to lighting. We set up our Houdini pipeline to also render using the same lookdev and shaders as final lighting, so we were able to get GPU rendered previews that very closely matched the look coming from lighting.
Can you elaborate about the destruction process?
The first part of the process was to tech-viz the timeline of the destruction over the course of the sequence. We had to map out which limb of the Red Room was breaking and in which order. This gave us a good guide to what should be happening in each shot. We had to keep our FX and Animation departments in sync, because some of the hero destruction pieces were driven by timing from animation that was taken over by FX sims. This also provided a guide to the timing of each key explosion that needed to happen, from the chain reaction of the engine core blowing up, to spreading outwards towards the arms of the Red Room.
Can you explain in detail about the FX creation and simulations?
Because of the very modular way we built the whole Red Room down to beams and bolts, we were able to get fantastic simulation detail from straight forward rigid body simulations. The heavy lifting work really went into the setup of all the constraints of many thousands of connected pieces, so when we finally ran the simulation, it broke apart in a realistic and believable way. The trick was to lock down the design and layout of the structure which sometimes proved elusive!
Only when we needed hero bending and breaking of metal did we then promote those pieces into a hero simulation, excluding it from the base sim. This really helped us simplify and keep the entire Red Room structure light enough to handle and publish over and over, with lighting simply receiving an updated set of transforms. This included the base structure, additional features (like cranes, solar panels and cargo containers) and finally any aircraft and props that were sliding off the decks.
Explosion volumes and drifting smoke was then placed either as a library volume or a hero volume simmed in place. The altitude and wind speed where the Red Room was located also factored into how the smoke and fire would spread as the sequence went on.
The end of the sequence is completely crazy with a free fall. How did you approach this part?
Creating believable skydiving shots meant having some realistic camera moves to create that sense of air-to-air photography. This was helped massively by the production having a real skydive unit. We also shot some amazing footage that we could steal camera moves from, and get great reference for how to pose during flight. It also helped with the small body adjustments that skydivers use to change direction or to speed up and slow down.
Surrounding our heroes’ flight to the ground is an enormous debris field that is falling with them, and we had to make sure it all made sense shot to shot, including ensuring any large recognisable pieces maintained continuity.
Can you elaborate about the shooting of this sequence?
As well as the real skydive unit, the production also shot actors in a robot control arm against blue screen to get close up flying shots. There was also a simulated skydive wind tunnel machine that provided a safer way to shoot some medium distance shots that needed realistic physics. Often these were augmented by digital character work wherever the stunt team or rigs needed to be removed.
Which shot or sequence was the most challenging?
Probably the shot where Natasha releases Taskmaster from a holding cell, and just as they face off the whole room snaps in two and she begins the slide down toward a now open precipice. The amount of destruction work for a one off shot like that is intense when compared to the other shots that we have built a system for.
What is your favorite shot or sequence?
There are two shots that really capture the free fall in a unique way: Natasha emerging from a falling Quinjet, and making her way toward an unconscious Yelena, who needs a parachute. The camera is disorienting, almost to the point where for a moment you don’t know what is up and what is down. These shots also showcase the amazing digi double work our team was capable of. We think these shots are really thrilling on the big screen.
What is your best memory on this show?
There was a great sense of accomplishment in seeing the first test render using our FX solver in combination with our instanced layout system. It received a fully lit and textured render, and we saw both how little memory it used, and how fast it rendered. It was that moment of happiness and relief that made us realize that maybe we were going to be OK.
How long have you worked on this show?
I originally went to set in London back in August 2019, but we started the asset build in May, so I think about 13 months. The onset of COVID-19 really changed the dynamic of the delivery months of what was already a very complex scope of work.
What’s the VFX shots count?
What was the size of your team?
Approximately 250 at peak.
What is your next project?
We’ll be moving onto a couple of different projects for Marvel Studios this year, including Shang-Chi and the Legends of the Ten Rings and Spider-Man: No Way Home, plus several non-Marvel projects for both features and episodics. We’ll be kept very busy!
A big thanks for your time.
© Vincent Frei – The Art of VFX – 2021