In 2016, Steve Emerson explained Laika‘s work on KUBO AND THE TWO STRINGS. He talks to us today about MISSING LINK.
How was the collaboration with director Chris Butler?
Chris and I have worked together since the CORALINE days, so we already had a strong rapport heading into the project. What’s unique about Chris as a Director is that he is an incredible artist and can draw very fast. As a VFX Supervisor you want to get into the head of directors as quickly as possible. You need to understand their sensitivities and the types of images that get them excited. With Chris, there was very little mind’s eyes translation. If he wanted something very specific in a shot, he’d grab a pad of paper, draw it and say “Do this.”
What were his expectations regarding the approach to visual effects?
Chris had a very specific vision for the world of MISSING LINK. Some of the key characteristics included vivid color, simplified shapes and explicit patterning. He expected to see these qualities in all components of the world – including the visual effects. Outside of the design elements, it really came down to servicing the story in artful ways that weren’t intrusive.
How did you organize the work with your VFX Producer?
Richard Thwaites was our Visual Effects Production Supervisor. He oversaw all of the production management for VFX. Here’s how it worked – We’d start with boards and animatics. Once a sequence cut was approved, the studio assembled all of the heads of department and key creatives into a room with the Director to break down the sequence shot-by-shot. At the end of that meeting, we’d have a clearer grasp of exactly what our obligations were in regards to the VFX work. At that point we bid out the work with our leads, strategize and talk assignments. I worked with stages to make sure that we had the animation, plates and survey required to do the work while Richard ensured that we had the artists and resources. When a shot was ready to kick-off, assignments, schedules, plate information and reference materials were published to and organized using Shotgun.
What were the main challenges on Missing Link?
It was scope. It’s a recurring theme here at LAIKA. For whatever reason, every film ends up being a greater challenge than the previous. KUBO was huge in terms of scope, but it also took place in a singular culture — a mythical Japan. This meant that if we had show assets, say a background character in Japanese wardrobe, that asset could be leveraged in a scene at the beginning, middle or end of the film. But with MISSING LINK we were dealing with a global odyssey that spanned many real-world locations and cultures. The film begins in London, then moves to the Pacific Northwest, then California, across the United States to New York, France, Switzerland, Nepal, the Himalayas… Needless to say, a background character used in the Pacific Northwest couldn’t be leveraged for Nepal. This meant that we had to build more characters, more props and more locations that any of our previous films. It was important to Chris, and everyone here at the studio, that we appropriately represented and were respectful of the many locations and cultures represented in the film.
Can you explain in detail about the design and creation of the various digital characters?
Eric Wachtman was our CG Look Development Lead on the film and oversaw this process for visual effects. Eric has also been around since CORALINE and was instrumental in developing and implementing the interdepartmental workflow that we follow here at LAIKA.
The typical process for digital characters tries to mirror the puppet workflow as closely as possible. This ensures that everyone is given the same information and using the same terminology. We start with artwork or sculpts from development. From there we either sculpt a digital maquette or scan a practical one. Once the form and volumes are approved, we then remodel them to fit within our VFX topology. To keep everything consistent we use the same topology for all the bodies and heads. We work with the costume department to make sure seams and costume design match the style of the show and leverage a variety of techniques including fully digital cloth shaders, scanned and photographed materials.
We’ll then shoot a turntable and reference puppet under consistent lighting conditions using three light sources. We shoot all lights, each individual light and capture HDRIs.
There are internal and cross-department reviews where CG is presented alongside the reference puppet. Once approved, it goes to the director for final review and sign off. All of this is to ensure that our digital characters look and feel the same as the puppets.
How did you handle the rigging and animation?
Everything that come out of visual effects at LAIKA needs to co-exist in the handcrafted worlds being photographed on the stages. This means that we’re constantly taking our cues from the stop motion puppet makers, craftspeople and animators here at the studio. Rigging is no exception. Terence Jacobson is our Lead VFX CG Rigger. He worked with the armature builders and riggers to ensure that everything from joint placement to deformer choices and influences had both the benefits and the constraints of the stage puppets. The same artistic leadership that approves every replacement face of the puppets, also went over every expression from our facial rigs to confirm continuity with their practical counterparts.
The other challenge Terence and his team faced was the sheer scale of the scenes. They needed to ensure audiences didn’t get a sense of repetition between characters and distract from the hero puppet performances. We built a rigging system of interchangeable bodies, faces, hair, facial hair and clothing so that each scene could be populated quickly with reliable and standardized rigs on otherwise unique characters.
Can you tell us more about the animation challenges?
It’s important to remember that what we’re trying to do here is drive an emotional connection between a puppet and an audience member. We’re working to create empathy between human beings and a ten-inch object being pushed through space one frame at a time. In order to accomplish this, LAIKA enlists some of the very best stop motion animators in the world. Our biggest challenge is creating computer-generated performances that can co-exist in a frame with that level of animation.
One thing that helps is all animation within is our films – stop-motion or computer-generated – is created under the supervision of the same Animation Supervisor. For MISSING LINK, it was Brad Schiff. We’re also fortunate to have animators that have been here long enough to understand the nuances and be mindful of them. Joe Gorski, our senior CG animator is one of them.
There’s a great shot in the film where computer-generated yetis are sliding across oil and crashing into each other. That was Joe’s animation. When we launched on the performance, Chris Butler let us know that he wanted a big comedic moment. He had specific timing, energy and posing in mind. Joe was able to work closely with Brad to deliver Chris’ vision. I love that shot.
Which character is your favorite?
I like the chicken. I don’t know why. It just makes me laugh.
The movie opens with an impressive sequence in Loch Ness. Can you explain your work on it?
Water doesn’t play nicely with stop-motion animation. You can’t grab it or pose it one frame at a time. Prior to KUBO we’d done smaller water moments in-camera –- splashing, puddles, water droplets — using printed animation cycles or petroleum-based products like K-Y Jelly. But with sequences where there are large bodies of water and a lot of interaction, the studio looks to VFX. Because of the success we found with KUBO, they trusted us to deliver a Loch Ness environment that was unique, on-style for the film and felt like it belonged in a stop-motion universe.
For Loch Ness, the puppets were shot on a greenscreen stage inside of a row boat. We had a Nessie stand-in puppet for stage animators to use that became the foundation for the VFX monster animation. The water, surrounding environment and ultimately Nessie herself were handled in VFX.
For the opening shot of the movie we shot the camera traveling over the beach separately from the camera that glides across the water and lands in a Lionel close-up. We were hoping to hide the transition in a bank of fog, but you could feel the cheat. Our CG Supervisor Rick Sevy stepped up and worked with his team to troubleshoot a myriad of technical challenges with that one. I also have to mention Michael Cordova, our Lead Compositor. He understood the importance of that shot and grinded every pixel to perfection. I don’t think he left his desk for a month.
The water work was done in Houdini with David Horsley, our FX Lead, overseeing the simulations. Chris Butler wanted naturalism, but he was also intensely design-oriented in regards to the look of the water. He wanted to feel the design without being too explicit. We accomplished a lot with some creative sky dome orientation for pattern reflections as well as breaking up surface displacements with show textures.
In regards to the surrounding environment and skies. I remember early in the project we presented Chris with a dark, moody, mysterious look for Loch Ness. He hated it. Chris wanted to open the film with a color palette that set a lighter tone for the movie. Ultimately, we ended up using pinks and lavenders and then swung the palette to green for underwater moments.
How did you use you experience of Kubo for the water creation on Missing Link?
The greatest lesson that I learned regarding water for KUBO is trust David Horsley. He’s one of the very best when it comes to water FX and we’re so lucky to have him as part of our team.
What’s interesting is that despite the praise we found regarding the water work for KUBO, Chris Butler water something entirely different. That was very important to him. He wanted water that was distinct in its own right. Because of the efforts of David and the many others that contributed, from FX to lighting and ultimately compositing, we found a solution that was naturalistic in regards to motion but also had the vibrant color palette, simplified shape language and explicit patterning that made it unique to the world of MISSING LINK.
Can you explain in detail about the water simulations?
This is David’s world, but here’s how he would explain them — Houdini software provides a FLIP fluid solver that is unbounded, meaning it can flow freely throughout the scene. FLIP uses a container to calculate the velocities, but it is recreated at every frame, so you can translate the simulation to follow a steamship and keep realistic-looking water interaction close to the ship. Our fluid domain did not have to extend to the full distance of the boat’s travel.
In addition to this, Houdini’s guided ocean layer technique creates a FLIP sim of a thin particle layer on top of an ocean surface which they call “narrow band,” freeing up a great deal of memory. Knowing this, we decided to use the method to simulate an entire rogue wave using animated and deforming sculpted geometry to guide the surface water keeping a narrow band of water on the surface. The effect gave us some important surface detail around the boat but also provided velocities that were used for long lines of surface foam.
One feature with guided oceans is the ability to instance any number of hero waves to add variation to the oceanscape. On a few shots we decided to position hero waves directly in front of our steamship to see some ferocious splashes as it cuts through the instanced waves. To do this, we’d input points on the ocean surface and the operator created a wave train where the chop is controlled, radius and rotation variance to form the shape. You can run a thin layer of fluid on this surface to make the hero wave break and form foam, churn, and crestmist.
To keep water on style for the film, we applied 2D textures on almost all our surfaces. We selected textures that had a natural flow to the pattern like marble or wood grain. Technically we accomplished this by using dual rest fields on the FLIP fluid surface which is used to track the position of the water over time. If a texture became unrecognizable due to rough seas, a new dual rest field was created every 50 frames or so and the old mangled texture was faded out. We did not want this effect to dominate the look so we kept it subtle.
The use of saturated color blended with colorful surface foam and underwater churn aided in the stylizing, yet this style could have existed in nature under just the right conditions.
Missing Link has many beautiful locations. Can you explain in detail about their creation?
There are so many locations in the film and each one created its own challenges. One interesting example is the jungle location which included a stop motion elephant that walks a 35-foot path with a camera move that pans almost 180 degrees while booming backward. It required much more real estate than we typically have to deal with. We ended up shooting the backgrounds late in the production so that we could expand the stage into an area that is ordinarily used for foot traffic. We also ran out of jungle, so the set dressers ended up dressing the background for the front end of the shot then, after we had it, they broke everything down and redressed for the tail. There’s a great timelapse after the film credits that shows them in action.
Which one was the most complicated to create and why?
The ice bowl. It serves as the location for the film’s climax. There were plenty of challenges with that one, mostly in terms of scale and executing on the specifics of what Chris was looking for in terms of design. As is typical, we started out with a small-scale maquette version of the environment which was then scanned and become the foundation for our digital asset. There was a subtlety to the topology of the mountain walls that didn’t translate well when we went from the maquette that, in terms of relative scale, would have had flea-sized characters to the puppet-world scale version. The walls, shelves and ridges lost their nuance. Unwanted shapes and shadows suddenly reared their heads. Ultimately, we had to deal with the majority of the sequence on a shot-level basis to get them right. It was far more time-consuming than we originally anticipated.
Can you tell us more about the lighting work?
We used Katana for lighting. Joe Strasser, our lighting lead, had a team of six which seems crazy considering the volume of work that we turned around. One advantage we have is that because we’re working on shots while stage animation is happening, we can easily call over the Director of Photography to clarify intent for lighting when the HDRIs, survey or camera plans fall short. There were plenty of one-offs on this show with green screens and significant set extensions. Having that access really helped streamline some intensely complex shots that needed to be turned around in a matter of days. That, and the fact that our lighters were rocks stars. They really came through.
Can you elaborate about the render farm and render time challenges?
LAIKA’s VFX team is small and we have many artists who wear multiple hats. On MISSING LINK we were fortunate that Mia Sires Nelson, who also served as a Junior Animation TD, was able to manage the complexities of and constant strain that we put on our render farm. Because our productions span the course of years, we’ll oftentimes be forced to do hardware or software upgrades mid-show. It’s less than ideal, but thanks to the efforts of Mia as well as our IT and production technology teams, we made it through some bumpy waters and ended up with a tier 3 state of the art data center that featured 400 fast blades and more cpu cores and memory than our previous architecture.
We put plenty of stress on the farm, but strategized to get the show done. The average shot was around 150 frames with a single frame from lighting taking around 2.5 hours to cook. We prioritized low-quality renders in order to get elements to compositing for first looks with the Director and allocated the nights and weekends for high-quality renders.
Are there any other invisible effects that you want to reveal to us?
There are several scenes in MISSING LINK that take place in windy environments. Because our hero is a sasquatch covered in fur, there was plenty of discussion early on in regards to how we should handle fur movement.
We ended up treating Mr. Link’s fur with an invisible UV paint. It’s the type of stuff that you see on those CSI shows. It glows when blasted with UV light. The puppet team painted each piece of Mr. Link’s fur with a gradient – the paint was denser near the tips and more transparent at the base. Then, after every frame of animation, we’d switch the lighting setup to hit the puppet with UV light. In those exposures Mr. Link looks like a Christmas tree covered in red lights. We then leveraged the red channel for an alpha to drive distortion. Because the paint was denser at the tips, they moved more while the gradient to the base made them feel rooted. They fluttered – similar to real fur blowing in the wind.
I should also mention the roto/paint team here because they are truly the unsung heroes of these productions. The stage animators use some pretty complex rigging in order to get the nuanced performances that the studio demands. All of those rigs needed to be painted out in post. More than one thousand shots required rig removal.
There is also plenty of cosmetic work that is done on the puppet faces. All of the facial performances are printed up on 3D printers and delivered to stages for animators to do replacement animation frame-by-frame. Oftentimes there are color differences on some frames and imperfections that are cleaned up in VFX. Roto/paint handled all of that as well. In total they did cosmetic work on more than 106,000 puppet faces.
Which sequence or shot was the most challenging?
There’s a shot during the bar fight. It’s a high angle looking down on the chaos with Mr. Link standing in the middle of the fray. There are over 20 characters performances in that shot, including a dog. — Scheduling for these films can be a real challenge. There are never enough sets or puppets. So, in this case, we ended up receiving the puppet performances as separate passes on green screens. Sometimes they were single performances, other times we’d get two or three puppets in a pass. There were also FG lamps, set dressing and a tarp that came through separately. Rigs were everywhere and the environment shifted and distorted over the months that it took to shoot everything. We also added CG performances and debris. It ended up being this crazy, complex puzzle of a shot. But in the end, I think it feels seamless. I’m really proud of that one. The team really came through.
What is your favorite shot or sequence?
I mentioned the ice bowl and the climax of the film earlier. That’s my favorite. Mostly because I think it’s a real breakthrough moment for stop-motion animation. It’s a fast-cutting action sequence with nearly two hundred shots. What’s important to remember is that it takes A LOT of time to set up a shot for these films. We do camera lineups, blocks, and rehearsals. And with each step the director is giving comments. Puppets are being repaired, sets are beings re-dressed, tie down holes are being filled, rig are being tweaked, lighting is being adjusted… After all that, there are final checks – and then the animator finally launches on the first take. Now, it doesn’t matter if a shot is 12 frames or two hundred – you still have to go through that process. So hopefully that explains why a cutty action sequence is something that you’d have to be crazy to attempt in stop-motion. So of course, we did it. That attitude, it’s one of the many things that I love about this place.
What is your best memory on this show?
We have a tradition of building a piñata for a VFX crew party at the end of the show. We build it in the form of something that represents a really tough shot or sequence. For MISSING LINK we built a Loch Ness monster piñata and filled it with tiny bottles of booze. That moment – all of us together at the end of the show after years of hard work beating the hell out of a Loch Ness monster piñata and celebrating when it busted open – that was a great one.
How long did you work on this show?
I did my first script breakdown in early 2015 and continued VFX and fixes all the way through the final color grade in February. That’s four years. But I just heard we may be doing some additional marketing work. So technically, I’m still working on it.
What’s the VFX shot count?
1486. VFX ends up touching every frame. Sometimes we’re simply cleaning up dead pixels, other times we’re adding oceans and mountains.
What was the size of your team?
97 including production management.
What is your next project?
I’m not allowed to discuss it, but I’ll say this — We’re on a mission here to realize the potential of stop-motion animation. We want to keep raising the bar and surprising audiences with stories that are unique and ultimately timeless. Every film is a greater challenge and more exciting than the previous. Number six won’t disappoint.
A big thanks for your time.
MISSING LINK – VFX BREAKDOWN – LAIKA
WANT TO KNOW MORE?
LAIKA: Official website of LAIKA.
© Vincent Frei – The Art of VFX – 2019