In 2019, Martin Hill explained to us the work of Weta Digital on the final season of GAME OF THRONES. Today he talks to us with Nick Stein (Animation Supervisor), Thrain Shadbolt, Philip Leonhardt (VFX Supervisors) about their work on Netflix‘s tale, THE CHRISTMAS CHRONICLES 2:
// Martin Hill – Overall VFX Supervisor
How was the collaboration with Director Chris Columbus?
Terrific, Chris is very collaborative and welcomed input and ideas into the film, both in the details and the wider scale ideas and designs.
What was his expectations and approach about the visual effects?
He wanted to advance what had been done in the first film. The scope and scale is a lot broader, and he wanted us to come up with epic scenes. He also wanted tight integration of the cartoonish elves into the sets and environments. Wherever possible, we preplanned the gags and arranged to have some sort of physical interaction for digital elements with the sets and actors. This required planning and commitment to the shots – you don’t want to puppeteer a stuffy of Jola cat licking Julian, and an elf dancing on his shoulder if you change your mind on the VFX later!
What are the main advantages to be part of the show since the first day?
It allows us to shape the show and help it evolve creatively. We started previs straightaway, forming the action scenes and a lot of the gags. This put everyone on the same page in regard to the action and composition of each shot, which helps hugely with set-up and planning. Chris Columbus was able to formulate details of each shot well in advance of the shoot, and Art Director, Jon Hutman had the benefit of knowing the performances during the design phase and could incorporate that into the creature and environment designs.
We worked closely with the DoP, Don Burgess on lighting, camera moves, and tech-vis to make sure they worked with the VFX. This collaboration included the development of interactive lighting for the FX, the timing and speed of pre-vis (e.g. travelling through the wormholes), and making sure our tech-vis was achievable by the AC’s and Special Effects Supervisor, Cam Waldbauer, who was in charge of the gimbal rig.
There are a thousand small advantages, like being able to work with the costumes department to get an actual woollen costume made for the elf maquette we used on set for lighting reference.
Can you tell us more about the filming of the flying shots with the actors?
Certainly. First, we built pre-vis to a high-level to establish the right energy and flow for each scene. It’s worth noting that most of our pre-vis was precise enough that it was almost a one-to-one match to the final shot compositions. We then tech-vis this, which involves copying the sleigh and camera motion into 3D space and inverting the forward-motion of the sleigh. Then we establish if the camera moves and gimbal rig motion are possible within their physical constraints. If not, we invert some of the camera motion onto the sleigh, or vice versa, until we have a physically viable solution. The moves are programmed into the gimbal rig and rehearsed by the camera department. We also rehearsed manual puppeteering of the gimbal rig with special effects operator, Andrew Verhoeven, to give ourselves the latitude to adapt for changes to dialogue or performance.
Which sequence or shot was the most challenging?
Probably the party scene with the DJ and crazed elves. It was one of the few scenes that wasn’t prevised, and we had to shoot the plates off artboards, and work the gags into them. Some of the restrictions inspired some creative gags in the end.
What’s the VFX shots count?
Around 850 across the vendors.
What was the size of your team?
Always a difficult question as there’s so many support staff at Weta, Lola, and onset, all of whom are completely necessary.
Is there something specific that gives you some really short nights?
Only the getting up early, excited to keeping working on the shots!
What is your favorite shot or sequence?
Such a difficult question—maybe the opening shot – (that was my first time directing second unit), combined with the book transition into the Antarctic environment. The original idea for book transition was many layers of theatrical set dressing that transformed into the scene. For budget reasons that didn’t happen, but you can see a hint of the idea in there.
What is your best memory on this show?
There were so many fun times, and it was a terrific team. Taking my kids, Mina and Theo on set and them meeting Kurt in costume in Santa’s village was pretty magical – the looks on their faces!
// Nick Stein – Animation Supervisor
The Village is full of elves. How did you use Massive to populate the environment?
We looked at using Massive as it is truly an amazing tool, but in the end, we actually opted not to use it for our elf crowds. Massive has the power to deal with armies of hundreds of thousands, which we weren’t looking at here. There were a lot of elves in THE CHRISTMAS CHRONICLES 2, I think our top shot came in somewhere around the 800 mark. Our motion and crowd pipeline gives us the ability to work with a wide range of crowd numbers, depending on what is required for the shot.
How was simulate their tiny presences on-set for framing and the cast?
The only simulations from an animation standpoint are the ears and tails of the elf crowds. We keyframed them for more hero/bespoke animation when they are close to camera. From a VFX standpoint, we had all the bells and whistles with cloth, fur and muscle sims.
Let’s talk about the “bad guy” Belsnickel. How did you use the Weta Digital experience for CG character?
We used a combination of mocap and keyframe to bring all elves to life, and Belsnickel was no exception. We had a treasure trove of reference from Julian Dennison playing his scenes as human Belsnickel. We referenced this and included his physical characteristics in his elf form to sell the audience on them being the same character.
Can you tell us more about the transformation effects?
There were two transformation moments in the film, one where Bel turns from elf to human, and the second when he turns back into an elf. There was some good ol’ movie magic tricks used here to find the right balance of both selling the story of the transformation and trying not to kill the budget. We actually see our VFX supe, Martin Hill’s shoes in the shot where they pop to human size as we needed something on the fly. We were able to imply that Belsnickel is growing with a PoV shot looking at the elves starring up at him, his shadow growing over the top of them, then back to the fully grown human cloaked in shadow. When we needed to transform back to an elf, Julian on the day played it wonderfully just by slowly crouching down, all we needed to add there were his growing ears and eyes getting slightly larger.
Can you elaborate about the reindeer’s creation?
As a starting point, we augmented horse motion capture and tweaked it to better match the gait, weight, and locomotion of the more stocky reindeer. We used Dasher as our hero reindeer, and the rest of the gang was modelled from her with fur groom and antler variations to give them their own unique look. For shots where we needed our reindeer to show their personality, our talented artists would keyframe their performance.
Can you tell us more about the eyes and fur work?
Large, realistic eyes are something Weta has developed in the past (ALITA: BATTLE ANGEL springs to mind). But we had to make sure they did not look so realistic that we would fall into ‘uncanny valley’ territory. We dialled it back a bit to get a nice level of realism with a more cute/cartoony style. At one point in the production, we had the elves eyes glow and go swirly like candy canes to show they had been affected by the elf bane. They ended up looking pretty scary, so we had to abandon the idea.
Our talented VFX team created a range of fur styles and groom variations for the elves to give each of them a unique look. Hakan and the Forrest elves had their own look and style—more roughed up, ragged fur and dressed in leather, whereas our Village elves were more light and fluffy, and wore knitted outfits in Christmas-y colours and designs.
How did you manage the challenge of the reindeers in “flight mode”?
We explored lots of options of how reindeer could fly through the air. Would they look like they are treading water? Do they skate on ice? Maybe backstroke?
There were a lot of sleigh-chase sequences and dogfighting with the jacklotes, so we wanted them to look like they are gunning through the air at top speed, but still have the agility to duck and swerve and loop-de-loop. We liked the look of a powerful, energetic run. Their feet never hit the hard surface of the ground, so we had their hooves scoop the air beneath them and made their body mass ‘float’ not jiggle as it would with impact. Our animator’s ability to blend motion from a slow, stationary hover through to ludicrous speed was essential to the energy of these shots.
Is there something specific that gives you some really short nights?
Some of the sleigh fight shots were tricky, especially when we are blending between live-action plates, to CG, back to live plates again. Our amazing artists and TD’s got to work and built a new live plate system that could handle these crazy motions and camera angles, so in the end, there was not too much sleep lost.
What is your favorite shot or sequence?
Not sure I’m allowed to say this, but my favourite shot turned out to be quite different in the final edit. It’s the shot where we start high up above a beautiful, sunny arctic wilderness. Serene flute music is playing. The camera begins to drop towards the ground, speeding up, and getting more rumbly. As we hit the ground, BOOM! Led Zeppelin’s ‘Immigrant Song’ kicks in just as Jola the Yule Cat rushes by camera! Perfectly timed to the iconic vocals of Robert Plant we pass through a train of reindeer antlers, camera swings around to reveal none other than Kurt Russel as Santa Clause riding at full speed to old-school rock and roll. It was super epic, and I loved it. Unfortunately, Led Zeppelin didn’t make the final cut, but if you play that song over the shot in the final movie, it still times up beautifully.
What is your best memory on this show?
It would have to be the day we had a huge group of children come into the studio. We suited them up in mocap suits, and they went completely crazy playing all sorts of games, which we captured as motion for our little elves. It was such a valuable capture session for us. A child’s motion has a genuine, playful, almost goofy kind of quality to it that you don’t quite get from an adult performer. A lot of the motion of our background elves throughout Santa’s village are the kids running amuck. They were all great, thanks kids! =)
What is your next project?
Some movie no one has heard of with large blue alien creatures.
// Thrain Shadbolt – VFX Supervisor
What kind of references and influences did you receive for the lighting?
We received all the usual technical lighting reference from set, i.e. HDRI capture, plates with silver and grey ball reference, character maquettes or stuffies, LiDAR scans etc. This gave us a good basis for realism, especially for lighting CG characters in a live-action. From there, we would make sure we were lighting our hero characters in a sympathetic way that matched the emotional and artistic goals of the scene, as per the direction from Chris. We also had many fully CG environments to create, and these by their nature would not be something that could be captured on set. In these cases, our starting point would be concept art or similar scenes from the first film.
The movie is full of magic. How did you create and animate these FX elements?
A good example of this is Mr. and Mars Santa’s magic. This we portrayed using particles systems and procedural geometry created in Houdini, then sprinkled with a liberal dusting of Nuke. Again, our starting point was examples of these kinds of effects in the first film, which we further developed and extended through multiple iterations with Chris. As is common with subjective effects like these, it’s useful to be able to show a number of different options on a similar theme, to help narrow down what kind of result the creative team are looking for.
Which sequence or shot was the most challenging?
They were all pretty challenging, really, but one of the trickiest was also one we tackled early on, involving Kate and Santa time-travelling to Boston airport in the1990’s in a blizzard! For this sequence, we had to create the entire airport, surrounding environment, and blizzard snow effects, and integrate them with the blue screen performer elements. The airport and surrounds were created by our matte painting team in Clarisse, while Houdini particle systems were the basis for the driven snow, along with practical snow elements added in Comp. For the all CG shots in the sequence, we animated CG digi-doubles, sleigh and taxiing aircraft, which were then rendered in Manuka. One lovely little touch our Compositors added here, and in a few other sequences was occasional snowflakes landing on the actors, then melting away. It was subtle, but the Director loved it.
What is your next project?
I wish I could say!
Is there something specific that gives you some really short nights?
Our shots where Mrs Santa is cradling naughty elf Belsnickel in her arms were the cause of a few sleepless nights! This tested suspension of disbelief, having a hero elf closely interacting with a live-action character for long stretches. Needless to say, this pushed our Lighting and Comp teams to the limit, at times working frame-by-frame to ensure the elf integrated with the plate.
What is your favorite shot or sequence?
I particularly enjoy the battle Santa and Belsnickel have in the night sky while flying their respective sleighs. It takes place in a fantastical moonlit sky-scape and has lots of great action. Belsnickel’s sleigh is a hot-rodded monstrosity, pulled by creatures called ‘Jackalotes’, which are crazed, tongue lolling hybrids of hyenas and French Bulldogs), which were really fun to create.
What is your best memory on this show?
I started on this show as our country went into lockdown for COVD-19, so my first meetings and dailies sessions were conducted using a whole new approach to working together, with everybody at their respective homes, logging in to video calls. I’ll never forget how despite this, everybody rallied during a truly scary time, keeping the team morale going in an amazing group effort. Of course, even better was seeing everybody again after returning to the office three months later!
// Philip Leonhardt – VFX Supervisor
How did you work with the art department to design the Santa’s village, the elves and the reindeers?
We designed Santa’s village based on the beautiful set designs the art department did for the physical builds centered around the town square area. We digitally extended all the houses and shops to create the full scope of the village and placed it in a valley surrounded by alpine mountains ranges. This was both inspired by concept art and real locations in Austria and Switzerland.
We knew we had reference to follow for the elves and reindeer of the first movie, however, as time and fashion evolves even at the North Pole we were supplied by a huge variety of new knit patterns, texture samples, fabric and materials by the costume designers to help create brand new digital costumes with intricate detail. It was of great joy to introduce a whole new group of elves that dwell in the forests of Turkey with their wise leader Hakan. We developed those characters based on early artwork that defined the type of weapons and clothes they wear. Our art department further assisted in sketching out color schemes and variation of hair grooms which our texture and shader team used as reference to generate all the different variants required.
Can you explain in detail about the Village creation?
A large portion of the main story takes place in the village square that is dominated by a very large Christmas tree in the center. The square was part of a substantial set build and acted as an anchor point to our digital village extension. It dictated the architecture and materials to be used and we honoured this by including every little detail, from the awning decorations down to single light fittings, into our building models. We then designed a layout plan of how to sit the entire village into a picturesque valley surrounded by large mountain ranges. Making sure that important story points in key locations such as the Reindeer Stables or Santa’s House had extensive views across the village – including its magnificent Christmas tree standing 6 stories tall. We procedurally distributed thousands of ornaments onto its branches and covered the tree in a sea of twinkling lights. Our village and the surrounding landscape with all its houses, roads, hills and vegetation had to be eye catching during day and night – with full control over the lighting and rendering aspect. That meant we created a full 3D environment that included all those assets dressed into place so we could bathe it in afternoon sun, moonlit skies or the magic hours of dawn.
The movie is full of magic. How did you create and animate these FX elements?
FX played a major part in this movie and we knew that, besides the magic FX, snow interaction FX was going to crucial to a large number of shots. Whenever you have an elf run across a snow surface or a reindeer galloping through the artic wilderness you require that element of snow integration between the CG character and environment which often was plate photography. Right at the very beginning of the show, we designed a system in FX that would analyse the character or prop animation and trigger a snow collision method containing volumetric snow spray, chunks and footprints based on the contacts and impact forces of the character or objects with the ground and their surroundings.
Which sequence or shot was the most challenging?
There were a lot of challenging moments, but the trickiest ones generally were when complex cg animation had to link up and integrate with practical action in the shot plates. For example, the scene in which Jola the cat licks Belsnickel’s face whilst Speck, the little elf, is dancing on Belsnickel’s shoulder after they flew through the aurora into the North Pole. Having those three characters interact so closely with one another in a believable way every tiny detail has to be convincing. Your match moves have to be spot on, the animation on the characters timed perfectly, the shading and lighting and fur detail absolutely accurate and the comp integration and contacts seamless. The audience knows immediately if just one component is off and it compromises the experience. I find mastering those shots extremely rewarding.
What is your next project?
All I can say it has nothing to do with Christmas at this point in time.
Is there something specific that gives you some really short nights?
Nothing too specific, but we were tackling some particularly difficult shots towards the end that required CG character interaction with actors, FX simulations and CG environment extensions across long frame ranges. As the project is close to delivery and time is precious instead of counting sheep at bed time you tend run through the outstanding scenes in your head to see if anything springs to mind that could optimize technical aspects of a shot or solve any outstanding creative challenges. I knew though that I could rely on the talent of my amazing team of artists and therefore never lost a whole night’s sleep on this project.
What is your favorite shot or sequence?
I love the sequence that tells the story of how Belsnickel became a human. The different locations and book illustrations that come to life and the character animation of Belsnickel elf himself are a real highlight. But do they beat the moment Santa swings a giant cat by its tail? Or all the great scenes in the village, the Turkish forest, Reindeer flying through skies or Jackalotes through wormholes? There are so many outstanding moments!
What is your best memory on this show?
Working with the filmmakers and the team here at Weta and seeing everyone contribute their part to make this such an enjoyable journey through tough times caused by a pandemic that shook everyone’s world and caused stress to all friends and families. It was important to us that THE CHRISTMAS CHRONICLES 2 endured and we made it happen.
A big thanks for your time.
WANT TO KNOW MORE?
Weta Digital: Dedicated page about THE CHRISTMAS CHRONICLES 2 on Weta Digital website.
Netflix: You can now watch THE CHRISTMAS CHRONICLES 2 on Netflix.
© Vincent Frei – The Art of VFX – 2020