In 2018, Tim Burke explained to us about the visual effects work on Fantastic Beasts: The Crimes of Grindelwald. He returns today to tell us about his work on The Little Mermaid.
How was the collaboration with Director Rob Marshall?
One of the first things I wanted to establish was how Rob wanted his mermaids to move, so early in preproduction we did some animation studies showing how gracefully Ariel could swim, turn, roll, dive etc. Rob loved the movement studies and they became an inspiration for how we would move forward in developing the film. We did similar full frame digital character tests to show him Halle as a fully animated CG mermaid and Melissa as Ursula, both wide to close up, which he loved. These developmental studies helped me understand Robs taste and direction and build a strong creative relationship with him, that would see us through a very complicated production. Working with him was a fantastic and collaborative experience.
How did you organize the work for your VFX Producer?
Dan Barrow started the films as VFX producer, organising the budget and helping plan preproduction, production and the awards to the facilities with me, then Leslie Lerman took over from him during post in New York, seeing the film through to its conclusion.
How did you choose the various vendors and split the work?
I chose the teams and supervisors I wanted to work with based on my knowledge of their own individual expertise and skillsets, backed up by the known technical capabilities of each facility, gained through years of working together on different projects. The primary vendors on the show were Framestore, MPC and Rodeo FX, with Union VFX, Vitality VFX and Secret Lab helping us with cosmetic and clean up work.
There are three principal characters in the film who are talking animals, Sebastian, Scuttle and Flounder, I turned to Pablo Grillo, Animation Supervisor at Framestore, to take on the challenge of bringing all three “amigos” to life. We developed Ariel, Triton and the sisters with Andy Kind, VFX Supervisor at Framestore, designing the look of their iridescent tails, relying on Framestores technical expertise in hair simulation, character rigging, photo-realistic CG digital doubles and all round animation strength.
I asked Patrick Ledda, VFX Supervisor at MPC, to look after the development of half human half octopus Ursula and all of her environments, as well as all scenes involving massive water simulations, relying on MPCs proven and sophisticated water pipeline. This would include Giant Ursula, a 100 foot tall CG creature version of Melissa McCarthy, who battles with Ariel in the third act, and all scenes involving Erics ship, which was a practical 100 foot set, built and filmed on blue screen on the pinewood backlot. All three primary vendors did amigo animation work for the film, sharing assets for Sebastian, Scuttle and Flounder, using Framestores look dev as the guide for look and performance. Both MPC and Framestore did Ariel and Triton scenes, again seamlessly sharing assets, including hair grooms for each character.
Rodeo FX, led by VFX Supervisor Pier Lefebvre, did a lot of the set extension work, adding environments and backgrounds to several scenes shot on location, the backlot and interior sets, using array plates we filmed in Sardinia, as well as adding Erics castle into location plates, which was built by MPC and used by both vendors.
What was your approach about the underwater sequences?
The early animation tests made it clear that we could film the actors as much as possible using rigs, but that we’d then only keep their faces and hands, replacing the rest of them with fully animated bodies and simulated hair in order to turn them into their mermaid characters. I was keen to allow Rob to be able to work and direct the performance and movement of actors on the stage floor interactively, so I didn’t want to use a preprogrammed computerised rig to move the actors around, everything had to be manually operated with the actors being puppeteered by a crew of well rehearsed stunt and dance professionals. Steve Warner, SFX Supervisor, developed multiple tuning fork, teeter-totter and wire rigs, all designed to allow specific ranges of motion for the different actors to simulate their underwater movement. The actors would be shot dry for wet on a blue screen stage lit by Dion Beebe, for which he used different caustic lighting rigs.
Robs background in musical theatre meant that a major part of his process involved extensive periods of rehearsals with the cast prior to filming. On such a complex film as TLM this became an invaluable part of the process and allowed us to be incredibly well prepared for the shoot.
Our production pipeline was very linear, Rob storyboarded some sequences, and blocked out others in rehearsals with Dion Beebe filming the rehearsals on a Sony A7lll, which were then cut by the films editor Wyatt Smith. The cuts were taken into previs and used as timing and blocking guides for each scene, where we translated movement from the stage floor into those of swimming mermaids in a water volume, or a swimming or crawling octopus in the case of Ursula. Each underwater scene was worked on by animators from the VFX facilities, this gave us great continuity and feel for the characters, with the animators taking ownership of their scenes throughout, eventually completing the finished shots for the film in post. Once the previs had been cut by Wyatt, it went back into rehearsals on the the stage, but this time with our actors in the SFX rigs. Each previs was broken down into what could be shot for real, what would be live action transitioning into a CG face, or vice versa, and what would be fully digital, based on the complexity of movement or distance to camera. All fully digital shots used ILMs Anyma facial capture system for the actors facial performances, which we shot concurrently on the stages at Pinewood alongside the filming of each scene. Every shot was rehearsed by our stunt/dance team with the actor in the rig, allowing them to drive performance. After weeks of rehearsals we were ready to shoot, and every single scene we filmed matched the previs to the frame, the most complex scenes involved 5 or 6 actors all moving on their rigs in one shot.
How did you create the impressive underwater world?
We had a research team working for us collecting hours of real underwater material and stock photos, our primary reference was the Blue Planet television series, as it contains some of the most beautifully filmed material anyone has achieved to date. John Myhre, designed the underwater worlds using the reference as inspiration for his concept illustrations, and he developed some of the environments such as Ariels grotto, Atlantica and Ursula’s Lair in Z Brush with cg artist we leant him from our facilities. These designs and models were then used as the starting point by each company to develop and expand the underwater worlds, additionally creating hundreds of different procedural corals, plants, rocks, sand scapes, shells, urchins, sea anemones and all manner of underwater life, some with intricate simulations to create the sense of underwater currents, or for interactions with actors. A huge library of underwater creatures was also created by each facility, often sharing models and textures, to populate the environments, as well as hundreds of bespoke hero creatures created for the animation in musical numbers like Under the Sea. A marine snow pass was used to further enhance the underwater feel, flowing with water currents or responding to turbulence created through movement of an actor or creature.
During the look dev of the underwater worlds it became very apparent that when we simulated real underwater lighting conditions, we lost depth and colour, sunlight doesn’t penetrate beyond 200 meters and colour disappears very quickly, red is totally absorbed at around 5 meters depth, which was a problem given Ariels iconic red hair. Therefore in order to make the underwater scenes more enjoyable and watchable we needed to cheat the reality and open up the visibility of our environments, allowing more light, depth and colour to register, pushing the blue attenuation much deeper into the scene than would happen in the real world. Half of our underwater scenes were also at night, so we came up with night and day looks that would allow the viewer to understand what time of day it was without losing information in the nights scenes.
The same was true with the underwater hair simulations, when we let the hair do what it would naturally do underwater, it often looked ugly, it would flatten or drag, or cover the actors face, so we had to manipulate the simulations to give more pleasing controllable results. Rob wanted the hair to be active and flamboyant during motion and musical numbers, but not distracting during dialogue scenes, but always beautiful, so we came up with several settings for different actions, that were then massaged on a shot by shot basis. Given how much of the film was being created in CG we were able to render a lot of the scenes and shots in full stereo for the 3D release of the film.
Can you elaborate about the creation and animation of the various creatures?
Sebastian, Scuttle and Flounder went through an initial concept phase, and once designs were locked they were passed over to Framestore to start their builds. Pablo and his team led the development of each of the characters, constantly changing and updating animation rigs as we learned more about each one and what Rob wanted from their performances. Rob wanted the amigos to look like photo realistic animals who could talk, and he didn’t want them to be cartoony or overly expressive, the performances needed to be nuanced and believeable, as any other actors performance would be under his direction. So we referenced real animals for each of the designs and behaviours, Sebastian is primarily based on a ghost crab, Flounder on a sergeant major fish and Scuttle on a gannet. As the builds and animation studies developed we recorded the voices for the amigos, filming each of our actors playing them with multiple witness cameras. This material was edited and turned over to the facilities with the vocal performances for each scene and used as inspirations for performance and gestures made by the actors, although we were very careful not to humanise the animals themselves. If Deveed raised his hand when speaking to punctuate a point, we might raise Sebastians claw in a similar way. During the shoot we used puppeteers to provide eye-lines for actors using 3D printed models of each of our characters. As mentioned earlier Framestore, MPC and Rodeo all did scenes involving Sebastian, Scuttle and Flounder, referencing each others work for continuity and seamless realism in the way they acted and were lit and rendered.
Which one is your favorite?
Sebastian is my favourite, he literally steals every scene that he is in thanks to the characterful performance of Daveed Diggs, along with the brilliant animation from the facilities. He was one of the characters Rob was most worried about when we started the project, but was actually the first one we found and locked into during preproduction.
Can you elaborate about the design and creation of Ursula and her tentacles?
Colleen Atwood created the concepts for Ursula along with all of the mermaid characters in film. She made a costume for Ursula’s upper body which we digitally recreated and then replaced in every shot. MPC took Colleens designs and turned them into our Ursula character, finding ways to blend her upper body into her lower tentacled octopus body to allow the actors performance to transition through the midriff into the octopus skirt, giving the illusion that the tentacles were always moving and propelling her around.
Ferran Domenech led the animation of Ursula at MPC, developing the previs for her scenes alongside extensive movement studies, referencing how real octopi move, which allowed MPC to develop and refine their tentacle rigs. The tentacles were hand animated but then had a simulation pass added to give them additional bend and twist, allowing the skin to jiggle and move. The suckers were also simulated to give the sense of compression and extensions when tentacles moved over surfaces, as well as variation in the way they illuminated.
Which sequence or shot was the most challenging?
A lot of the shots were very long and would comprise of multiple live action takes blended digitally together to create one seamless continuous shot. Half of the film is set underwater and essentially full CG with the actors faces and hands being the only filmed element, the body tracking and shot sculpting for actors in each shot was difficult and time consuming. Under the Sea was the most challenging scene for the animators, some shots have over 400 individually animated characters in them and the whole scene is several minutes long.
What is your favourite shot or sequence?
My favourite scene is when Ariel sings Part of Your World, it’s when all of the techniques and magic come together in an amazing 3 minute musical number. Ariel moves so balletically and effortlessly through water that you really do believe she is a mermaid.
What is your best memory on this show?
Lots of great memories, the shoot was lots of fun, a great crew and Sardinia was a great location to work in, but my best memory was hearing Halle sing on the stage when we were filming. Her voice is truly remarkable and she held one particular note that gave me goosebumps, and she did all of this whilst being moved around in an intensely physically demanding tuning fork rig, she was truly mermazing!
A big thanks for your time.
WANT TO KNOW MORE?
Framestore: Dedicated page about The Little Mermaid on Framestore website.
MPC: Dedicated page about The Little Mermaid on MPC website.
Rodeo FX: Dedicated page about The Little Mermaid on Rodeo FX website.
© Vincent Frei – The Art of VFX – 2023
