Sean Mathiesen has been working for nearly 30 years in visual effects and animation. He has worked in many studios including Weta Digital, Rising Sun Pictures, MPC and joined DNEG in 2016. He has worked on many projects such as the LORD OF THE RINGS trilogy, TERMINATOR SALVATION, MUPPETS MOST WANTED and BLACK MIRROR.
How did you and DNEG get involved in this show?
DNEG worked on a pitch with The Jim Henson Company and Netflix for THE DARK CRYSTAL: AGE OF RESISTANCE. The initial idea was to make CG Gelfling and set extensions while the Skeksis would be traditional puppets on a limited set. When I joined, this test had already been completed and Hensons, Netflix and DNEG all agreed that CG Gelfling was not the correct look for the show. It wasn’t true to the original material.
I was brought on about 10 weeks before the shoot was scheduled to begin. I had a long history with puppets, starting my career in Stop Motion and more recently using Visual Effects with THE MUPPETS, so this project seemed a natural fit.
How was the collaboration with director Louis Leterrier and the showrunners?
Louis explained all of his ideas during several long pre-production meetings, starting quite explicitly on what he was looking for in the early episodes and giving as much detail as possible for the later episodes. As the show progressed it was surprising how many of his original ideas stayed true to those original meetings. We of course had to be adaptable to what was possible with puppets, but most of the big set pieces he described up front are available to view now.
What were their expectations and approach for the visual effects?
Visual effects was always going to support the puppetry in DCAOR. It would also be used expand the world of Thra and represent visually the magic that was more implied in the 1982 film as it was more difficult to execute in the days of optical printing. VFX would remove puppeteers, enhance the puppet faces for greater emotional range than was possible with a puppet. CG Tongues would be added to the Skeksis mouths. Digi doubles would replace the children in costumes which was the technique used in the film to get full body Gelfling to walk. Stock footage would be enhanced and used to expand the world.
How did you organise the work with your VFX Producer?
Based on the first turnover schedules we created a plan to work episodically, having two teams that would leapfrog episodes, finishing one episode and then moving downstream over the other team’s episode to the next available one. However, as with all the best plans… things change. Turnover became more sequence oriented rather than episodic due to the fact that Louis thought about all 10 episodes as one long story rather than individual unique installments. This is part of the nature of the story of Dark Crystal and the deep mythology and rich history of Thra. So we changed to a more technique driven schedule using shot groups. This meant that we would be working on multiple episodes at the same time, groups of artists and supervisors working on the Caves of Grott, Threaders, Lore’s formation and Blue Fire, with others working through the Circle of the Suns, the Crystal Castle, Landstriders and the Aureyal…and more!
How did you split the work amongst the DNEG offices?
That’s a mind-boggling question that involves more that just one single show – it’s a company wide approach that addresses talent and availability among other factors. We tried to keep sequences contained to one facility and place each sequence with the strongest and most appropriate teams around the globe.
How did it feel to be part of this iconic universe?
Amazing! Like so many others, THE DARK CRYSTAL was a huge influence on my youth. It helped drive me towards the job I do today. So to be a part of this huge undertaking, to bring the things that I have learned to do over the decades and to help tell more of the story of Thra is a fantastic opportunity.
How do you approach a puppet show?
Theoretically… you approach it like any other show. You assess what the requirements are for any individual technique and then start to plan, crumple up those plans and make new plans… trash those and come up with a final plan… then go back to the bin and pick out your previous plans and splice them into the current plan… and then eventually… you have something that you can execute, safe in the knowledge that it will change.
But seriously… puppets are the stars and you view the VFX through that filter. You plan out your techniques based on keeping the world consistent, in this case – consistent with puppets. You know you will have Magic and Environments and Creatures. You always think about how the puppets will react to the Magic or fit into the Environment or how the CG Creature would look and act if it were a hand-made cloth and foam and fur puppet.
Can you tell us more about the previs and postvis process?
Previz was pretty limited. I did a little bit myself for the carriage chase (I used to be a Previz Supervisor so I felt I could define what was necessary for the shoot based on Louis’ thoughts – while still leaving things open to the organic nature of what most directors want). I made a roughly one minute sequence that dug into the heart of the sequence and it allowed me to show both the puppeteers what we were looking for, as well as the crew shooting the BG plates a flavour for what the sequence would require.
Postviz was a bit more extensive for the sequence, choosing the BG plates that would fit the FG puppeteer and carriage footage. We used an in-house team from TPO for the postiz and were able to define 90% of the BGs required for the 5 minute sequence.
TPO also did some postviz work on the Dreamfast sequences using a completely 2D approach in Nuke. While this didn’t end up being the technique we used for the Dreamfast, we did use some of the ideas they developed in the Prologue.
Can you talk about a typical day during the shoot and then during post production?
Well, a typical day eh? During the shoot it was breakfast with the team, then off to the stages for some kind of Skeksis Horror, Podling Heroism or Gelfling Magic. It’s hard to make an average day out of it beyond that. Because every day for 10 months it was something different. I suppose it got increasingly more manic as everything became more normal and familiar and everyone became more familiar with one another.
During post I would split my days between the Post Production offices in Soho at first working on turnover and getting clarification from Louis or the editors on the minutiae, then as time went on reviewing sequences or episodes to make sure everything was consistent. Afternoons were generally spent at DNEG, giving briefs, reviewing shots and getting the show out the door.
How did you create the CG versions of the puppets?
We set up a Photogrammetry scanning booth at Langley. The Puppets, when not being puppeteered, were pretty much like props. We made a stand that would hold them in a consistent pose and then placed them on a turntable, scanning them 360 degrees. The data was then sent to DNEG and processed. Scan turntables were then sent back to be approved by Louis, sometimes the neck or legs would be too long when the puppet was scanned, or the face would be a bit too fat. Once the notes were addressed the geometry was cleaned up, toplogised, textured and a final Digi Double would be approved.
Which character is your favorite?
Pockets McGee – Aughra’s faithful podling assistant. I adore his morning routine, and he has great hair.
Many shots involve a large number of puppets. Can you tell us more about your work on those shots?
The goal was, of course, to always have as many real puppets on set whenever/wherever possible. When this was impractical, if not impossible, the next line of defence was to film additional puppets against a Blue or Greenscreen and composite the element into the shot. Finally, when no other plan would work, we would hand-animate cycles, render and place the extra Gelfling into the shot. There were never big enough CG crowds to use Massive. We relied a lot on Editorial to give the impression of more puppets in a scene, rather than actually having a cast of thousands. In the final battle we used CG Flags behind the practical puppets, to imply that there were many more puppets, just out of eyesight, which was very effective to give a sense of many more Gelfling than was physically possible. It felt very much like a trick that a puppet show would use.
Some of the creatures can fly. How did you design and create their wings?
Certainly the main flying creatures are the Gelfling. Toby Froud made a set of Puppet Gelfling wings and we ran that through the scanner to create a digital version. When we showed Louis the textured turntable, he felt that it was too much like paper and he wanted to make them more iridescent. More like insect or Dragon Fly wings. We re-sculpted some of the architectural elements (bones) and then worked on the transparency, translucency, and venation of the wings, we dialled in the specular and iridescent aspects in comp to make the wings that ended up in the series.
What was the main challenge with these flying characters?
Physics. Trying to match a puppet that is not moving towards or away from camera, but is bobbing and weaving and darting back and forth to avoid objects with a photographic plate that is moving through Z space and give the whole shot a sense of real physics was a significant challenge.
The series is taking us to various exotic locations. How did you work with the art department to create these environments?
Gavin Boquet (Production Designer) and his team did a lot of work updating maps and creating concepts, based on drawings by Brian Froud. Gavin and his team looked into all the various environments that our story would need to go. Gavin created a 3D printed map of Thra to give us all a better understanding of the distances and dimensions involved. When I received the map, it was a portion of a continent and it extending beyond the Age of Resistance narrative, but was not a complete world. I extended this map to flesh out the full continent and then created a full Mercator projection so we could look at Thra as an entire planet. All of this material was given to a Stock Footage researcher who found us the most exotic locations on Earth, yet the least well known. When we received those stock scans often much lower resolution than the final 4K deliverable to Netflix, we would create Matte Paintings to add distant mountains or deserts, further ensuring that the various environments of Thra were unlike anything seen on earth. We would add features of Stone in the Wood to nature footage filmed in Borneo, Gelfling cities to plates of Alaskan mountain ranges, Snow and Ice to plates of Volcanic planes. That sort of thing.
Which one was the most complicated one to create and why?
Ha’rar was by far the most complex. The actual exterior street set consisted of approximately 8 buildings, 4 on each side of the cobblestone street. We had to extend this set to a city the size of Machu Picchu at it’s apex, located it within a circular mountain range and place it on the northern coast of Thra with majestic vistas of the ocean. We had stock footage plates of Alaska to use in wide establishers that we had to track in a digital Gelfling city. Then when the story takes us into the city, we had to create a ring of background 3D mountains when looking inland and the ocean when looking further to the north. We had to create a consistent geography to include the main narrative features within the city while placing the entire city within an unrecognisable environment.
How did you recreate the iconic Skeksis Castle?
We started with a digital scan of a prop from the 1982 movie. This was great because it ensured that we were faithful to that Iconic castle. However, the detail in that sculpt was indented for the relatively static wide shots that were seen in the film. The back side of that sculpt was unfinished and had all of the fiber optic cables running down the back side. So we had to clean up the back side and complete the Crystal castle so that it would work from 360 degrees.
Then we knew we were going to go in much closer, combining the digital castle with various balconies and walkways that Gavin Boquet designed as real sets. So we took Lidar scans of the various set pieces and modelled those into a full digital castle and started to add in additional detail to match the physical textures and materials. Moss, water damage, cracks and fissures – things like that. Finally, we knew that the story required the castle to have a bridge across the moat that surrounded the castle at this time in Thra’s history. This was slightly confusing as there is concept art for the « sequel’ in which the Gelfling build a bridge to the castle – but that occurs many years later, after the film. Our story occurs before the film and the bridge we needed was the original bridge built by the Skeksis. So we had to design an all-new bridge with a Skeksis aesthetic. I love the breadth and depth of detail that I acquired over the two years on THE DARK CRYSTAL.
Let’s talk about the Crystal. How did you design and create the various FX elements for it?
So many different effects for the Crystal – but I think the most interesting is the Electricity. We started knowing this would be a 2D effect. Louis and Lisa had discussed staying true to the movie so our first go was to create a procedural electric effect that had a 1980’s feel. However, this didn’t really advance the filmmaking technology very much and was too much like something that would have been in GHOSTBUSTERS. Louis mentioned that we should try to create something that was like the hands of the Skeksis torturing the Crystal. So we made several version of this based on a picture of the scientists hand from the creature shop, but it wasn’t dynamic enough. Then we filmed moving footage of puppeteers manipulating the Emperor’s hands and based our electrical effect on that motion. Unfortunately this was too literal. So we brainstormed and came up with 3 additional ideas and then presented them to Louis. The one he responded best to was a « Manga » like pulsing electricity. At first we rendered this out in pure white and then started to add extreme chromatic aberration, and finally shifting the colours to work within the colour palette of THE DARK CRYSTAL.
Later in the series, we discover blue flames. Can you elaborate on their creation?
When Louis first described this to me he mentioned a light show where animated characters were projected into water mist. Taking on board that direction we started to play with making a multi plane effect in Nuke, projecting regular fire (graded blue) onto cards and moving the camera about. We took an element of Brea and embedded that within the multi plane. When the camera was on the line, everything matched up and we could see the character within the blue fire, but when the camera drifted off axis, the image would soften and distort. We tried various additional techniques, applying motion vectors from the fire to the character and other tricks to distort the image, but we always returned to that first multi plane idea. As the story progresses we had to make an effect to create the columns, so we looked to the real world for floating fire and found will-o’-the-wisps, which are naturally occurring floating fire burning swamp gas. We created a Houdini will-o’-the-wisp element and projected that onto cards that animated upwards, and filled in some of the negative space in the column with sparks and embers.
Can you elaborate in detail on the design and the creation of the Darkening FX elements?
The Darkening needed to be evil (no small task with the Pink colour palette of the Crystal) and it had to resonate with the electric effects and optical lens flares that we had created for the Crystal in the Scientists chamber. Our visual language included sparks and embers and the variations of chromatic aberration. We often communicated the malign effects of the darkening with spores. Early in the series we see a flower in the endless forest contaminated with the Darkening. We created an internal pink glow and emitted black spores using Nuke Particles. Again, later in the series we see the Darkened Landstrider Dung. We created the Darkening here in a similar way to the flower, all in 2D. When the Emperor summons the Darkening in the darkening chamber, we had to create a 3D effect, but had already created the visual language of internal Glow and black spores. Here Louis asked for the Darkening to be serpentine in nature. So, using Houdini we created slithering columns of black dry ice smoke, sinking down past camera. We created a separate internal ‘pink’ lightening that could be added in Nuke as necessary. And finally the black spores from the Landstrider dung, also using Nuke Particles. The Eyes of darkened creatures were Nuke based, working with the way light played on the puppet eyes, though the Darkened Pluff’m which had to show a change from Darkened to dead was slightly more complex. Using a texture of elderberry stems projected onto a torus in Nuke and then sliding the texture around the torus to create the ‘darkened’ life in the Pluff’ms eyes.
We knew that the Darkening in episode 10 would be the culmination of the series and a combination of previous effects – but it needed to be the biggest and baddest of them all for the final battle. On set we used florescent tubes laid out in a lightening pattern to get interactive lighting. Then we re-created the ground plane of Stone in the wood based on Lidar scans and layered this over the fluorescent tubes, forcing the light source under the ground – where the Darkening comes from. On set we used air mortars full of dirt and cork to create the explosions. We filmed elements of these same Mortars and we also created CG versions in Houdini that could be placed in the path of the Darkening as it advanced.
As the Emperor unleashes the Darkening towards the Gelfling, we created a pass of the advancing Darkening that was still embedded within the ground, and then using the hot spots in the simulation we created rays of light that extended upwards and shifted these hot spots to a more orange colour, finishing with the explosions of dirt and Embers created in Houdini. When Deet absorbs the Darkening we created an effect to work with the physical puppeteering where she seemed to be holding a ball – so we created a high energy ball in Houdini, consisting of three passes, the purple of the Darkening, the Black of the Spores and Darkening Tendrils and a High energy blue-white ball that were combined and vibrated in comp and enhanced with real lens flairs. Then when Deet unleashes the Darkening back at the Skeksis – well this was the final moment of the Final battle, so we had to throw everything we had developed at it. And to add the kitchen sink, we created a crawling electric effect in Houdini that was animated like the legs on a spider, one bolt of darkened lightening moving forwards, enveloping the features of the set and then stopping, followed by a separate ‘leg’ moving to overtake the previous one.
The Gourmands Death by Darkening was created with blue screen elements of the pustules exploding individually followed by an element of the Gourmands head exploding. This was fun to film as Jon Savage created a duplicate hollow head, filled with explosives, SFX puss and cake and we had several takes filmed at high speed to really look at this in detail.
Which shot or sequence was the most challenging?
The Carriage Chase in the Endless Forest was, in my opinion, the most challenging. It was always going to be so, knowing what was in the script compared to what I believed was possible with a puppet. It seems there are very few truly dynamic action sequences with puppets – for good reason. Puppets make amazing characters, but they don’t move through space with great ease.
When we began filming the FG Puppets and Carriage against Green screen – it was pretty clear this was going to up the ante on Parkour Frog (my previous experience with a puppet action sequence in THE MUPPETS MOST WANTED). When we started seeing rough cuts of the chase, there were over 100 shots in the sequence and it kept expanding in subsequent iterations. We decided to film the backgrounds using Puzzlewood in the Forest of Dean. We decided to make the background plates 10 times larger than reality, or reduce the carriage to 10% of its real world size. This meant filming along the walking paths in the woods just a few inches off the ground and moving the camera at about 8 MPH – just what Tom Sweet and Louis were able to walk at a very fast pace. We spent 4 days filming in Puzzlewood, on pre-selected paths with the camera making repeated passes looking every 45 degrees of the compass.
Once we got the footage back we worked with the in-house team from TPO, selecting plates from the most interesting environments that also best matched the lighting conditions and camera angles of the FG Plates, creating a journey through a mystical landscape. While at Puzzlewood we also captured Lidar, textures and photogrammetry of the most interesting features and then for the shots that were all CG we re-created the interesting shapes and features of Puzzlewood, but expanded the physical space beyond the narrow walking paths to allow the Carriage, Armaligs and Gelfling to move as required at the end of the sequence.
Is there something specific that gave you some really short nights?
Pretty much everything. But I don’t mean that in the way you might think. It’s not that any single effect was a problem. It was the total number of shots and assets in the whole series. I had never worked on anything so complicated or in depth before and trying to wrap my head around the immensity of the undertaking was a daunting task. But the thing that gave me comfort on those nights was the team around me. Everyone involved in the VFX was a consummate professional and no matter how insane the task, everyone brought their A-game.
What is your favourite shot or sequence?
Well, with 10 hours to choose from that is not an easy question.
But here’s my top 3:
- A) The Peeper Beetle at the end of episode 2 – and believe it or not, what ended up in the series is the tame version!
- B) The Gobbles at the end of episode 4, when Rian’s father, Ordon, gives his own life to save his son.
- C) Deet flying through Grott in episode 1. The cave system is simply magical and the Nurlocs have yet to become fully darkened. They seem like such gentile creatures.
And an honourable mention goes to
- D) Mira being drained in episode 1 – I don’t ‘like’ it per se. It’s horrible. But I think the work everyone did on Mira and the Essence and the Crystal and the Beam was quite beautiful and elegant.
What is your best memory on this show?
Looking at the reviews and conversations – for better or for worse. It’s great that people really seem to enjoy the end results of all that hard work.
How long did you work on this show?
Two years exactly.
What’s the VFX shot count?
What was the size of your team?
984 artists plus Supervision and Production.
What is your next project?
A bucket list dream come true for me. A Western – I am already in New Mexico shooting it.
A big thanks for your time.
© Vincent Frei – The Art of VFX – 2019