In 2013, Ben Morris explained to us about the work of Framestore on LINCOLN, he then worked on GRAVITY. Then he joins Industrial Light & Magic. STAR WARS – THE LAST JEDI is his second film STAR WARS after THE FORCE AWAKENS.
What was your feeling to be back in the Star Wars universe?
It was great. Having started on the project over 3 years ago, it has been an incredibly collaborative process for everyone involved. Now the film has opened it is great to see and hear audiences reacting so positively to the film.
How was the collaboration with director Rian Johnson?
Brilliant. He is one of the most focused, consistent and above all talented Writer/Director’s I have had the pleasure to work with.
What was his approach and expectations about the visual effects?
It’s interesting – in the beginning I would say Rian was cautious of anything to do with VFX and CG in general. He had previously written and directed the time travel thriller LOOPER which, contrary to it’s subject matter, didn’t require that much complex VFX work. When we first met we talked a lot of about techniques, old and new, and how we should aim to get as many in-camera effects as possible.
The first major sequence on his mind was the Canto Bight Fathier chase where we knew we were going to have to place human actors on the back of stampeding CG Fathiers. Whilst this might at first sound quite daunting, it turns out I have experience from previous projects relating to the unique mix of VFX, SFX and CFX needed to build and shoot actors on riding rigs driven by CG animation.
How did you organize the work at ILM and with the ILM Supervisors?
ILM London (Global Hub) – Mike Mulholland (VFX Supe) – supported by Jellyfish, One of Us, Ghost, ILP, Virtuous, Base FX
Key Sequences – Snoke, Bombing Run, Torpedo Run, Canto Bight, Fathier Chase (up to beach), Throne Room sabre Fight and the majority of asset builds, not including Mega Hanger and Crait.
ILM Vancouver – Dan Seddon (VFX Supe) supported by Hybride, Rodeo, Base FX, Virtuous
Key Sequences – Jedi Island sequences, CG Porgs, DJ Ship Interiors/Holograms, Resistance Chased sequences, Mega Destroyer Interiors, Mega Hangar Pre/Post Destruction, Finn/Phasma Hangar Fight, Holdo Hyperspace.
ILM San Francisco – Eddie Pasquarello (VFX Supe) supported by ILM Singapore – Alex Pritchard (VFX Supe), Base FX, Virtuous
Key Sequences – Fathier Chase (beach and grasslands), Crait sequences (including Mine Interior, Speeder/Walker Battle, Crystal Cave Falcon Chase, Mine Exit), CG Foxes, CG Porgs, Luke/Kylo Flashbacks, Rebel Transports Attacked.
Can you tell us more about the previz and postviz work?
We brought in Albert Cheng and his team from The Third Floor to work in Pinewood with Rian and myself during pre-production and shoot. The main sequences they focused on were the Bombing Run, Torpedo Run and Crait Battle finale. Steve Aplin, our global Animation Supervisor, led a team of ILM animators creating the Fathier Chase previs. Once approved, Mike Mulholland started to breakdown the shots into techvis, in order to define how we would shoot the riding elements using a high-speed Bolt robot camera rig, inside an LED light-dome gimbal stage.
During post, Barry Howell led a second team from The Third Floor in LA, creating postvis to support Rian, myself and Bob Ducsay(Editor) as we prepped the Director’s cut for turn over.
How did you approach the big opening space battle?
Rian and I worked with Albert’s team to create a full length previs of the sequence based on storyboards drawn by Dave Allcock. This previs allowed us to discuss camera choreography, lighting and gimbal rig design with the other filmmaking departments. When the sequence was finally turned-over to ILM London, our layout team led by Sonia Contreras jumped in and converted the TTF scenes into Maya scenes populated with latest ship assets to begin final shot production.
How did you handle the choreography of the fights?
We defined and created most of the space battle choreography using keyframe animation in Maya.
Can you tell us more about the light interactions for the cockpit shots?
Unlike other recent STAR WARS films which utilised high resolution LED panels and front/rear projection screens, we went with a light dome designed by our DOP, Steve Yedlin, featuring an array of heavily diffused Arri Skypanel lights and a keylight mounted on a Technocrane. We projected pre-rendered spherical LatLong lighting elements onto a 3D model of the dome in TouchDesigner, which in turn drove the Skypanel array around the gimbal. We controlled the lighting dome through a custom TouchDesigner interface on an iPad.
Can you explain in detail about the design and creation of the massive Dreadnought and the Snoke’s Mega Destroyer?
Both these ships were designed by Kevin Jenkins – Lucasfilm Design Supervisor. Working with our Production Designer, Rick Heinrichs, in the Art Department Kevin refined early 2D concepts into full 3D printed models that could be approved by Rian and passed on to ILM’s DMS team led by our global Asset Supervisor, Ben Lambert.
The larger ships were so complex they had to be built in layers. We tackled the larger sections first to establish the overall forms and then started dividing each piece into sub components to make the overall complexity manageable. Each sub component would be covered in multiple layers of panel work and then smaller greebles added to provide detail and scale.
For the finer details we use a library of model kit parts that are recreations of the plastic model kits used in the builds of the original practical ships. These kits help us ensure each new ship we build shares the same common STAR WARS ‘look’ for surface details and weaponry. We established our build approach on the Dreadnought and then used the same approach on the huge MegaDestroyer – it was massive, with a 60km wingspan!
There are tons of FX during this sequence. Can you tell us more about it and especially about the explosions?
We knew from the start that there was going to be a lot of FX destruction work during all the space sequences. Because of this we invested a lot of time in setting up a new destruction pipeline that allowed us to dramatically increase the complexity and detail in the explosions. Our FX team led by Peter Kyme and Miguel Perez Senent setup the destruction workflow in Houdini and rendered in Mantra and Renderman.
Can you explain in detail about the design and creation of Supreme Leader Snoke?
We’d already created a heavily stylised, 25 foot tall hologram version of Snoke for THE FORCE AWAKENS. At the start of pre-production on THE LAST JEDI, Rian came to me and said he wanted to “bring Snoke out of the shadows and make him real”. This meant scaling him down to a more human sized 7 feet tall and redesigning the look of his face/skin/body in order to create an old aged, photoreal human character. Based on this brief our first port of call was to start researching what can happen to human anatomy when people grow older – including pronounced spine curvature, arthritic hands, age spots, broken capillaries under the skin and in eyes, drooping /weeping eyelids, dried cracked lips and discoloured teeth. Another key visual feature of Snoke was his heavily disfigured face and anatomy. Again we turned to real world examples of healed scar tissue and structural disfigurement seen in online medical journals and archival imagery. In parallel to ILM’s research, the Creature FX team sculpted and painted a maquette which became a starting point for our rebuild of the character.
As the CG build progressed into first pass animation reviews it became apparent the original maquette design didn’t support Andy Serkis’ deep, resonant vocal performance. We had to jump back in and give the character a stronger physical presence – including scaling him up to over 8 feet tall, broadening his chest and shoulders, straightening his back, strengthening his jaw line and most importantly redesigning his damaged throat anatomy.
On the tools side of things – Snoke was modelled in ZBrush/Maya/Zeno by Benjamin Flynn, textured by Sally Wilson in Mari and Look Dev’ed by Arturo Orgaz Casado in Katana/RIS 21.
How did you manage his facial animation?
Starting with the raw Vicon Cara 4-camera video data captured on-set by the Imaginarium team, Animatrix then generated 2D point tracks of the dots on Andy’s face and lip/lid splines which were ingested into our internal FaceCap pipeline in Zeno.
From here we could generate an exact clone of Andy Serkis’ face performance, which was in turn re-targeted onto the Snoke character rig using our SnapSolve toolset. The re-targeting process often requires iterations of refinement and then additional layers of keyframe animation per shot as required – animated by Ted Lister, Lead ‘Snoke’ Animator and his team in London.
Can you tell us more about his eyes and skin creation?
Through Snoke’s teasing and torturing of Rey and going into the climactic kill moment, we frame tightly on his face, and had to deliver a huge range of expression from Andy’s underlying performance – predominantly through eyes which are now a piercing blue, after being simpler dark pools in THE FORCE AWAKENS. ILM standard eye geometry construction helps catch light and scatter believably, but we added a multi-layer iris, with additional displacement control for modifying curvature for a fake caustic effect, to ensure we always caught light and life in the eyes. For close-up shots we also ray-traced a true caustic pass in RIS.
For Snoke’s old and damaged skin, we pushed to get plenty of tonal and hue range between the shallower-scattering tight folds and wrinkles, and the redder hollows and recesses, whilst biasing to a less human, less healthy, more sallow palette. We layered in liver spots, moles, raised dry skin flakes, a parallax effect for subcutaneous arteries, sweat beads, saliva simulation, peach fuzz, grey hair strands and stubble for detail and character.
We also had skin compression data from the rig that could drive fine wrinkles and blood flow, and ran tetrahedral sims to spread skin tension – in particular creating some nice, subtle fat pad motion under the eyes during eye darts.
Yoda is doing a great cameo. Can you tell us more about your work on this iconic character?
Yoda is a physical hand puppet in THE LAST JEDI – designed and built by Neal Scanlan’s CFX team. Frank Oz performed the scene under a specially designed exterior set, including raised ground terrain to conceal the puppeteers and the huge burning Jedi Tree beyond, designed by the Art Dept and incinerated by Chris Corbould’s SFX team.
In terms of VFX work in the Yoda scene, we added his Force ghost effect and subtly moving edge-glow, as well as cleaning up puppeteering rigs and making minimal adjustments to the facial performance in 2D when requested by Rian. We also extended the surrounding mountainside to give the environment a more dramatic feel.
We meet many new creatures such as the Porgs and the Crystal Foxes. Can you explain in detail about these cute creatures?
The Porgs were always conceived as practical puppets – designed and built by Neal Scanlan and his CFX team. This meant building lots of different shot specific puppets, as no individual puppet mechanism could achieve all the performances required in the film. We were often called on to paint out rods, cables and puppeteers from shots involving the puppets.
As we got further into post we realised that some of the puppets weren’t giving the range of performance that Rian wanted, so we built CG versions of the characters too. Roughly half the shots in the film feature CG Porgs animated and rendered by our Vancouver and San Francisco teams.
The Crystal Foxes were based on beautiful 2D concept designs showing rainbow iridescence and reflected, refracted light passing through the complex layers of crystals in their coats. Early in pre-production Neal and his team designed and built an animatronic puppet to shoot on-set. We ‘rolled’ on the puppet for a couple of shots, but ultimately knew that we would have to create the final characters in CG, due to the detailed animation and intricate crystalline look required to make the characters believable in their environment.
How did you handle the challenge of the animation and lighting for the Crystal Foxes?
The Foxes were built, animated and rendered by Eddie Pasquarello and Matt Shumway’s crew in San Francisco. Matt’s animation team paid great attention to the smallest of details – researching how Arctic Foxes move to ensure their keyframe animation maintained the naturalism of a real creature, even with their fantastic alien design and crystal coats.
Lighting the Foxes was another significant challenge as they often ‘correctly’ took on the colours of their environment, which inside the Mine could sometimes look a little dull. The lighting team, led by Erich Ippen, always tried to ensure that the characters had a strong component of back or side light to give their crystal fur a glistening / shimmering quality regardless of how dark their surrounding environment was.
Did you use animatronics? if yes, how did you enhanced them?
No – the Crystal Foxes are entirely CG.
The movie is taking us to exotic places. How did you created these new locations?
The main exterior locations featured in the film are the Jedi Island of Ahch-To, the Casino planet of Canto Bight and the mining planet Crait.
For the early scene when Rey first meets Luke we returned to the tiny island of Skellig Michael off the coast of Ireland to ensure exact continuity with the final scene of THE FORCE AWAKENS. For most of the later scenes on Ahch-To we chose to shoot at a series of mainland locations, again on the West Coast of Ireland. The most significant was at Sybil Head and featured a full practical set build of Luke’s Jedi Village, cantilevered over the edge of the 450 foot sheer sea cliffs. Most locations required some digital extension based on 3D photogrammetry drone surveys and textures shot during pre-production. We also shot hours of helicopter mounted sky and sea plates that were used throughout the Ahch-To sequences.
For Canto Bight we travelled to the historical city of Dubrovnik in Croatia and staged our street action in and around it’s beautiful, polished limestone main street – Stradun. Whilst the location featured some practical set built up to about 3 metres high, there were significant set extensions and set dressing of digital speeder destruction and additional crowd, not forgetting the 15 giant, stampeding Fathiers in nearly every shot. The beach and grasslands featured at the end of the chase were almost entirely digital, with a few shots on the cliff edge being shot on an exterior set built at Pinewood and extended in post.
For Crait we travelled to the vast salt flats of Salar de Uyuni in Bolivia. We spent 3 days shooting tracking vehicle and helicopter plates in this remote location based at 3700m above sea level. Using real Bolivia plates and a full size Trench set built in Pinewood’s Paddock Tank, we aimed to ‘ground’ the Crait battle in reality using as many photographic elements as possible.
As the action progresses we knew we would be creating the majority of mid and wide angles as full digital shots later in the sequence. Our San Francisco team, supported by Singapore, created all the Crait sequences – including the Mine Interior, built as a practical set in the 007 stage with significant set extensions and Mine Exit backlot location nested in one of the deep pressure ridge crevasses, behind the mountain plateau. They also created the dynamic Falcon Chase through Crait’s underground fissures and caverns of red crystal – all built as CG environments by Enrico Damm and Johanes Kurnia, our Crait Environment Supervisors.
Which one was the most complicated to created and why?
That’s a tough one. I don’t think I can call out any one sequence – there were so many challenges to solve in so many fun sequences.
Can you tell us how you choose the various VFX vendors?
Over the years we have setup great working relationships with many VFX studios outside of ILM. These companies are primarily chosen based on their creative talents, but other factors such as global location/capacity and specialist skills such as tracking/layout come into play too. On THE LAST JEDI we were lucky enough to work with some new collaborators, including One of Us in London and Important Looking Pirates (ILP) in Stockholm.
Can you tell us more about your collaboration with their VFX supervisors?
Before any work was awarded I met with each of our external studios and VFX Supervisors to discuss their proposed package of shots. This gave everyone an early heads up as to what the work entails and also allows them to bid the work before final award.
The vendors are all around the world. How did you proceed to follow their work?
During every turnover Rian and I would discuss the shots and then brief the ILM Supervisors who would ultimately oversee the daily/weekly reviews with our external vendors, before presenting the work to myself and Rian. Although we always like to give individual teams complete shots and sequences of their own, it is inevitable that some asset/shot sharing occurs, which is why we like to pair our external vendors with one of our internal ILM Supervisors to ensure continuity and consistency throughout the project.
You made a cameo in the movie. At which moment can we spot you?
Ha, yes… I’ve been rumbled. About 5 minutes into the Bombing Run. I play Resistance Bomber Pilot Teene – “… fighters incoming !!”
Which sequence or shot was the most complicated to create and why?
In terms of shear size and detail I would think the fight between Finn and Phasma in the collapsing Mega Hangar was one of our biggest sequences. There are literally hundreds of individual pyro simulations in some of the bigger shots, along with the collapsing digital environment and digi-double Troopers.
What is your favourite shot or sequence?
To pick one shot/sequence is impossible. If I’m forced, I have to admit I’m a sucker for a good sabre fight and we have a pretty cool one in Snoke’s Throne Room.
What was the main challenge on this show and how did you achieve it?
I think my main concern was keeping such a large body of work as consistent and photographically real as possible. All the incredible Supervisors, Artists and Production who worked on this project put so much passion and enthusiasm into their work that it made my job easy. I hope all that hard work shines through when audiences watch the final film.
What is your best memory on this show?
Watching Frank Oz perform Yoda with Mark Hamill on a freezing cold night on the Long Cross backlot.
How long have you worked on this show?
What’s the VFX shots count?
There were over 2000 effects shots.
What was the size of your on-set team?
We had a fantastic shoot crew of 10 people, including Richard Bain (VFX Supe) and Felix Pomeranz, Ed Price and Jack George – who wrangled all our data capture for Main and Splinter Units.
What is your next project?
Spending some time with my family… in terms of film project, I don’t have anything confirmed!
A big thanks for your time.
// WANT TO KNOW MORE?
ILM: Dedicated page about STAR WARS – THE LAST JEDI on Industrial Light & Magic website.
© Vincent Frei – The Art of VFX – 2018