The team of Luma Pictures (composed by Kevin Souls, Brendan Seals and Raphael A. Pimentel) is back on The Art of VFX to present us their work on BLACK PANTHER.
How did you get involved on this show?
Kevin Souls (KS): Luma has a very long-standing relationship with Marvel Studios going way back to the original Thor, so that got our foot in the door. Regardless of the relationship, you still have to better yourself and prove you’re the right studio for the job. For this film, we needed to be sure we could not only animate and render Black Panther himself to look photoreal, but maintain a good sequence continuity, hit some demanding deadlines, and also nail what was looking to be extensive CG reconstruction and cloth simulation in the car chase. From there it was a matter of sitting down as a group and ironing out the details, and we’re very happy they chose us for these sequences.
How was the collaboration with director Ryan Coogler and VFX Supervisor Geoffrey Baumann?
KS: Both Ryan and Geoff were incredibly passionate about the project, and they really wanted the VFX vendors to take ownership of what they were doing. That level of involvement started during the shoot in Atlanta for the Casino and Market shoots, and continued into post-production as sequences evolved and changed. At the beginning we worked together to hit their creative targets, and as things progressed, they asked us how we could make it better and tweak it. That’s when it makes it rewarding for the artists, because their ideas are heard.
What was their approach and expectation about the visual effects?
KS: It was important to the both of them that the visual effects felt grounded in reality and—first and foremost— that the VFX tell the story and have weight, both visually and emotionally. So for our sequences, that meant having a solid base in set and location photography, but also acquiring everything for potential virtual set pieces since we knew there were some shots that we weren’t going to be able to capture in-camera.
How did you organize the work with your VFX Producer?
KS: As work turns over, it’s first distributed across the facilities and we set the most aggressive schedule we possibly can for first turnarounds. I’ll weigh in on the creative challenges and unknowns and he’ll tuck into the details from there with the production managers and CG supervisors to make sure it’s all laid out correctly.
What are the sequences made by Luma?
We were responsible for the Busan Casino sequence, the Busan Street chase and the CIA Black Site.
Can you explain in detail about the Casino fight?
KS: We shot the Casino fight in Atlanta on a giant soundstage. It was an incredible set—a fully realized casino interior with multiple floors and stairways, populated with gaming tables and dozens of extras. To allow for camera and stunt rigging, the ceiling and the majority of the chandeliers would always be digitally added, while the awnings and railings were all breakaway, and would be replaced when necessary. Conceptually, Ryan wanted to have sections of the fight scene work as a “oner”, a series of shots that play as one long take. The limited space and hand-held feel of the scene prevented the use of motion control rigs, so the shots were carefully staged to match end frame to start frame and “seams” would need to be compositing transitions. To achieve this, the action and camera work had to be carefully choreographed and, because of the progressive destruction and FX, be shot in order. The Big Stitch, as we called it, would end up playing at the end of the fight scene. It comprised of seven individual shots stitched together and culminates with Klaue firing the Sonic Disruptor. Mixed in with this we added destruction, retimed parts of plate, removed people, and replaced them with virtual characters. Sadly, if we did our job well you didn’t notice most of it.
How did you create the various digital doubles?
KS: The Black Panther production team used Clear Angle on set to do full scans of all the digital doubles in our sequences. From those sessions, we received hi-res 3d scans and bracketed photo textures. The models and textures were ingested, reparameterized for our pipeline and rigged for animation and cloth sim. Each character had specific needs: Nakiya required a very specific hair build, whereas Okaya required a very elaborate cloth set up for her red flowing dress. For T’Challa, we had to give extra care to the clothing which had to burn and tear off as he transforms into the Black Panther.
Can you explain in details about the creation of Black Panther?
Brendan Seals (BS): The Black Panther asset was a shared asset amongst vendors and Luma was fortunate in having early turnovers of real world photography to integrate Panther into, which compelled us to finalise the look development ahead of time. We continued to refine the specular properties of the shaders throughout production to enhance the glyphs and triangular details of the suit, particularly to give him greater dynamic range in our night time sequences.
One key aspect that Luma lead the charge on was the build up of purple kinetic energy through the suits design lines and the subsequent expulsion of energy and resulting shockwave. We designed particle simulations in Houdini to burst from impact points on the suit and weave through intricate patterns, way-finding towards the vibranium necklace. The particles were rendered natively in Houdini and combined with a subtle metallic shimmer treatment in Nuke.
The action then moves to a car chase in Busan. How did you recreate the city?
BS: Recreating key roads and intersections of the city was quite a challenge and required us to build a new tool to handle the variety and complexity of incoming geometry and set photography. Extensive Lidar scans were done of the city and probably the most comprehensive photography capturing the locations I’ve ever seen. We received incredible amounts of data covering stills, HDRs and panoramas of every inch possible of the chase and so we developed an environment projection tool in Nuke to allow our matte painters and compositors to correctly map this vast imagery onto the Lidar geometry. Spatial accuracy was key with such large environments being reconstructed, so we tracked the HDR and panorama cameras from on set and fed those into the projections. The artist could then combine several panoramas together with the tool providing an automatic blend where the projections would start to degrade. They could also bake the projections to high resolution UV space to finesse or even remove details if necessary in Nuke, Mari or Photoshop. Once we had full coverage of the environment we would then parse out certain aspects to be rebuilt as CG assets due to their specular nature, such as the wet roads, windows and any metallic/shiny surfaces. The careful combination of panoramic projections and CG assets ensured the environment was rendered in the most realistic and highest quality possible. We also developed this new workflow to work alongside our lighting tool “hdrMate” such that lighting artists could use these photoreal environments to light their characters and vehicles.
How did you populated the streets with crowd and cars?
KS: At the end of the car chase, the disabled SUV carrying Klaue careens out of control, slamming into a curb, and flipping end over end into a street market. On set, the production had a group of about 50 extras to populate the post crash scene, but for safety reasons the set had to be clear for the crash footage. The crash itself was covered from a few different angles, and we would need to add crowds to every shot. To accomplish this, we borrowed the extras and set up a three camera rig—straight on, forty five degrees, and ninety degrees—to capture a series of 2D sprites that we could layer into the scene. Since we had camera tracks and Lidar, we could easily place them in the correct 3d space. At the very end of the production we realized we needed a few extra people to react to the crash, so we set up a small shoot in our studio and used the artists from the show, if you look closely maybe you can even spot me.
BS: To heighten the sense of danger with the main cars chasing at speed, we added numerous cars in the sequence, both parked and traveling alongside the action. We developed a variety of CG car assets that needed to be photoreal and build up the density of traffic and be invisible to the audience. We also developed some CG mopeds in which one of the Toyota 4runners crash into and obliterates!
The lighting conditions changes really often. How did you manage this aspect?
BS: We always look-develop our CG assets in calibrated and controlled lighting setups to ensure they will respond correctly and physically accurately when viewed in whatever lighting scenario the sequence calls for. The car chase was no exception with no shot being ‘static’; Black Panther and the CG vehicles were constantly moving through light sources, and in general, no two shots shared the exact same space within Busan. Our lighting workflow has been constantly improving over the last couple of years in how we approach photorealistic integration with our in-house tool “hdrMate”. It allows our artists to map the set-acquired HDRs spatially and accurately to Lidar geometry and extract lights at the right scale and distance all inside Nuke. The benefit of doing this in Nuke is respecting the colour management of the show and having a more accurate understanding of the exposure and colour of the lights before going into 3D. Once we build our base light rig in Nuke, hdrMate exports all this data and builds the light rig automatically in Katana for the artist. This light rig represents the environment geometry projected with multiple HDR panoramas as well as all the light sources extracted from the imagery and textured onto lights set at the correct scale, distance and intensity. This of course, is just the first pass in replicating the photography 1:1 and means that additional beauty lighting can be done on top. It is a process that we have really refined on this show and the artists took a lot of care with faithfully recreating every stretch of the chase so that the characters and vehicles remained photoreal whilst seamlessly moving through ever-changing lighting conditions.
The bad guy, Ulysses Klaue is using a mechanical arm that turns into a weapon. How did you design and create this arm?
KS: Klaue’s arm is essentially a prosthetic that contains a weapon inside with technology stolen from Black Panther. It shoots a sound wave which reverberates through the environment and breaks objects into pieces. The production had done some concept art and the actor portraying Klaue, Andy Serkis, wore a detailed silicon prosthetic glove, which would became our base for the closed shape and material properties of the arm. We set about modeling, but it became clear that the look of gun was dependant on the mechanics of how it opened. To accomplish this, we started with rough geo and went straight into rigging to produce a tech anim of how it would open and close. It was important that the gun was mechanically plausible but also scary and imposing, so we carefully rigged the barrel to fit inside the hollow forearm and telescope as it opening. Based on feedback, we quickly flipped the flared out forearm and tucked it underneath, to put emphasis on the barrel. With the hard part done, we refined the geo, added high resolution detail and accent lights. To complete the effect, the cloth of his sleeve was often replaced with cg cloth to add subtle interactivity.
Can you tell us more about his animation?
Our Animation Supervisor, Raphael A. Pimentel, was responsible for overseeing all the animation for the show. As I mentioned above, the arm design was driven by the animation. We had to split open the arm in sections and then fold and stack the armature to reveal the sonic disruptor inside. Initially, we tried staggering the actions (like a shotgun begin loaded and pumped), but Ryan wanted something more deliberate and smooth. He wanted it to be carefully designed and machined to really make it look and feel like Wakandan high tech. Ryan instructed that it should feel like one action: the pieces would fold in unison, while the gun simultaneously unfolded and extended. The barrel would unfold and “batteries” would roll into place and line the interior of the stock.
Additionally, the first time the audience sees the gun is in the Casino, and there were dramatic beats timed carefully with the camera move and tension in the shot, which our animation team had to hit. Working within the shot created a really good feedback loop. Now that we could see the framing and perspective of the show, we adjusted the timing of the barrel and tucked the arm pieces even further underneath to reveal more of the gun. All this was rolled back into the rig, and for shots going forward, each action became a slider per object that animators could roll back and forth and keyframe.
How did you created the FX when Ulysses Klaue fires with his arm?
BS: Klaue’s arm opens up to fire a ‘Sonic Disruptor’ which distorts space as the energy ripples through the air and then on impact, vibrates and dismantles the target into an array of mechanical parts and pieces. First, we researched various phenomena such as sonic booms and and the way shock waves were created by objects traveling through air faster than the speed of sound. The sonic booms formed the basis of the elements distorting the photography, but given the look was tied so closely to the distortion of the backgrounds in each shot, we had to figure out a way to make the design more consistent across the sequences.
We also researched bullets travelling through different mediums in slow-motion such as water and ballistics gel. The tendril forms created in ballistics gel were incredible to behold (almost like lightning), and so we layered that into the design as blue tendrils of energy. To tie the technology closer to the world of the Wakandan technology seen in the film, we looked into cymatics and the patterns they create. Cymatics are the result of high frequency sonic vibrations and create complex geometric patterns. We started experimenting and building various forms of cymatics in Houdini and layered these in as orange echoes printed in the wake of the sonic disruptor. The combination of the tendrils and cymatics on top of the shockwave distortions made the look much more art-directable and greatly helped to set the look of Klaue’s weapon and other weapon related FX in the film.
The impact of this weapon completely destroy the cars. How did you handle these destruction shots?
BS: For the shot involving the Lexus LC500 being destroyed with Panther leaping towards a building we envisaged primarily in animation. The nature of the destruction was much more a dismantling of the mechanical parts and pieces rather than an explosion, and so it made sense to art-direct in animation all the key large and mid-sized pieces of the exterior and interior of the car. The shot was full CG so we had control over the movement of the car as well as how the camera viewed the action. We researched car accidents to see how an impact head on would rip through the engine bay and tear the front apart. The way the metals bent and components shifted or broke was very specific in the references so we decided to replicate this as a destroyed version of our car model. The animation team then triggered blendshapes using these customised versions of the model and combined keyframe animation with rigid body dynamics to get the right timing of the debris and play the action to camera. Once we had locked timing and the trajectory of the destruction, we then layered in FX simulations on top of glass shattering and smaller car debris.
KS: Near the end of the Busan Car Chase, Klaue turns the tables on Okaya and Nakia by using the sonic disruptor to destroy their Lexus GSF. It offered a series of challenges, not only did we have to destroy the car, but we had to destroy it over 4 shots, while switching between real-time and slow motion. It also provided the clearest view we would get of the sonic disruptor waves as they ripples through car and dissipate in the env. We used a hybrid approach for the characters, animated digital doubles so that we could choreograph the movement precisely, and get just the right dress performance, but seamlessly integrated face replacements from plate photography to add realism and keep the audience guessing.
The car chase ends with the funny shot. Can you explain in detail about it and how you created it?
KS: The car chase between Klaue, Nakia and Okoya ends with the Lexus GSF being obliterated by the sonic disruptor. Both women tumble forward in a curtain of debris but emerge out the other side unscathed, sliding down the hill on shredded car parts. They really wanted to play it for laughs. Geoff and Jesse asked for a shot layout, so we could get exactly the plate we wanted during reshoots. Our Animation Supervisor, Raphael A. Pimentel and our animation team used the Busan location Lidar, virtual technocranes and camera lensing in Maya to design the shot framing and the rigging needed to execute it.
On set in Atlanta, the crew matched the tech viz and got a wonderful performance from the actors. With those plates ingested, we were able to design shredded FX pavement, sparks and smoke around those performances.
What is your favorite shot or sequence?
BS: My favourite would be the shot where Black Panther expels his suits kinetic energy to create a shockwave and flip the Toyota 4Runner. It’s a visually rich scene with the black silhouettes of Panther and the car set against the purple and blue hues of the Diamond Bridge. Once the destruction begins and sparks fly, the scene comes alive with warmer hues to sculpt Panther and bring out his heroic flip, turn and landing.
KS: Hands down, the Casino big stitch. It was such a puzzle to put together and contains so many things that will remain invisible to audience. I got to use every trick in the book, from something as simple as just matching the camera position using the playback monitors on-set to simulating thousands of gambling chips and money raining down after the sonic disruptor. I also get to tell people that we used a virtual T’Challa to jump to the second floor, but when the sonic disruptor blows him over the balcony and he slams down onto the poker table 30 feet away – that’s a real stunt guy…and he did it four times.
What is your best memory on this show?
BS: Seeing some of our most challenging work shown so early in the trailers! It was rewarding for the artists to have their work seen with so much exposure and a great opportunity to really refine and polish the aesthetic as much as possible for the film.
KS: What Brendo said.
What was the most creative/technically challenging aspect of Busan sequence?
BS: The shot of the blue Lexus being destroyed by Klaue really commanded almost every aspect of work slated for the car chase sequence. It required a full CG environment, CG traffic, a destroyed Lexus LC500 with debris, the sonic disruptor FX and of course Black Panther!
KS: Whoop street. We got to explore a slow motion sonic disruptor, tear a car to shreds, and integrate some very technical plate photography into a fully virtual set.
How long have you worked on this show?
From Dec 2016 to Jan 2018.
What’s the VFX shots count?
Approximately 227.
What was the size of your team?
Just under 200 including operational and support staff.
What is your next project?
ANT-MAN AND THE WASP – plus a few others we can’t talk about yet!
A big thanks for your time.
// WANT TO KNOW MORE?
Luma Pictures: Dedicated page about BLACK PANTHER on Luma Pictures website.
© Vincent Frei – The Art of VFX – 2018
