Last year, Christian Kaestner explained Framestore‘s work on CAPTAIN MARVEL. He then joined THE AERONAUTS.
How did you and Framestore get involved on this show?
I personally, only joined THE AERONAUTS at a later stage as I was still finishing up another project. When I got on board, the principal photography had wrapped and Tom Harper was working with Martin Macrae in our art department to establish the looks for our full CG cloudscapes.
However, Framestore got involved at an early stage of the project to discuss methodology and doing early tests as a proof of concept. Stuart Penn, one of our visual effects supervisors in our London office was also covering the pre-production and shoot for us, working closely with Tom.
How was the collaboration with director Tom Harper and VFX Supervisor Louis Morin?
We were lucky in the sense that Tom had a very clear idea of the look he wanted for his film. Tom worked really closely with our Art Department to establish composition and colour palette for the main full CG cloud environments and then he trusted Louis and myself to translate the concept art of the establishing shots into the look of the sequence. Louis and myself worked closely together to ensure we would use the right methodology for each sequence.
Louis also supervised the South Africa shoot, where he shot the live action cloud backgrounds as a six camera array for us to stitch. He provided us with still frame selects of the backgrounds, we would then do proxy stitches of Quicktimes, present rough versions back to him and Tom for approval. This approach allowed us to be efficient and also manage that data wisely, as you can imagine, stitching 6x 8k plates per frame is quite a lot of data.
All in all the process was very collaborative, it really felt we were all working closely together to create the best possible visuals for the film.
What was their expectation and approach about the visual effects?
From the beginning we all knew the visual effects on this film needed to support the storytelling and otherwise be invisible. Seamless integration was just as important as the photorealism of the backgrounds. Out of our 500 visual effects shots, approximately three quarters of them were basket shots filmed against blue screen. This approach allowed the filmmakers the most creative freedom and control while shooting, but it meant we had a lot of backgrounds to create to support the story.
Tom and Louis knew that in order to let the audience follow the journey of Amelia and James, all aspects of our visual effects work needed to be finished at a very high level. We couldn’t stand out or distract the viewers in any way; we just needed to be there, invisible.
How did you organize the work with your VFX Producer?
The show was already well in progress when I joined full time after delivering CAPTAIN MARVEL. Given the scope of the show it was inevitable that we split show between several producers and supervisors. I worked closely with Executive Producer Coline Six to ensure we were progressing on all fronts to deliver the show. We had daily production catch ups in the morning to keep everyone in the loop about what was going on in each department or sequence. It was a great way of working and collaborating in a big team.
How did you split the work amongst the Framestore offices?
THE AERONAUTS was spread through several of our offices around the world. Stuart Penn one of our Supervisors from the London office was covering the shoot and the biggest chunk of work was executed in Montreal and some sequences were finished in our Pune offices. It’s pretty standard for us these days to work across several offices and put the work where it’s best suited. Our pipeline is very robust to handle the work to be done wherever it is around the world. Our supervisors are used to working across offices and doing dailies across locations is also nothing unusual these days.
Can you elaborate about the creation of London?
Creating period London of 1862 was probably the second biggest challenge of the VFX work for us besides the CG skyscapes. We did a lot of research on city layout, architecture and look and feel of the era. It was important for Tom that we established a believable scene for the audience right from the start. Whilst there was a variety of smaller and larger establishing shots and set extensions, the takeoff and the following wide establishing shot, when James and Amelia sweep across the London skyline, caught most of our attention. We had to get this one right as it was establishing the time period, but also, maybe even more importantly, it was establishing the believability of the balloon ride.
How did you populate London and the various locations?
We did a lot of research on landmark buildings and city layout. We used old maps that we projected onto a ground plane to ensure we get the feel of the streets layout right. Then we manually placed important landmark buildings by hand and populated the rest of the city and parks with an internal instancing tool that allows us to populate larger areas with variations of buildings and vegetation. We built a library of residential and factory buildings which in itself could be varied and combined in different ways, combined with texture and detail variations we were able to fill the 270 degree view of the city scape, not easily, but effectively. Depending on the requirements of the London shots, we also added digital doubles, horses and carts, smoke stacks, building sites, a flowing river Thames, trains and all sorts of little details to bring the city alive.
How was filmed the sequences inside the balloon?
Answered by Stuart Penn (VFX Supervisor at Framestore) // For the most part, shots in the balloon were film on a blue screen stage at the LH2 Studio in London. The basket, rope and ring assembly were suspended from a rig attached to the ceiling of the stage to give it a natural balloon swing and allow it to be easily rotated. The rig could be raised and lowered to allow us to get a wide range of angles on the basket. It’s quite a high stage so it gave us flexibility. Director of Photography, George Steel operated most of the shots hand held, in the basket with the cast, sometimes anchored to the outside of the basket or in rigging above the basket.
For shots of Amelia scaling or crawling across the outside of the balloon, inflatable sections of the balloons were built as set pieces in the same stage. Framestore developed an AR app, fARsight, that allowed pre-generated sky backgrounds, developed during pre-production, to be visualised on set. It would key the actors using the blue screen which made it a valuable tool for Tom and George to make sure the lighting would match the backgrounds to be added in post, a very important consideration when working with blue screens.
Not all shots were shot on the stage, one in particular, was shot at altitude with the cast in the fullsize practical balloon. The English countryside in the plate was replaced by our CG London environment.
How did you work with the stunt and SFX team?
Answered by Stuart Penn (VFX Supervisor at Framestore) // Tom is a very collaborative director and really knows how to get the best out of his team by encouraging them to work together. From early in pre-production we had regular Head of Department (HoD ) meetings. From the earliest meeting ls we discussed together approaches for how shots could be executed. By the time we started previs we had an idea of the types of rig the SFX would be able to create and understood the limitations of any stunt work. The aim was to avoid using digi doubles unless really necessary and instead use the cast or stunt performances. We discussed initial versions of previs with Stunt Coordinator Adam Kirley He would advise on the safety and practicality of the action. The stunt-vis his team created from the previs gave valuable feedback allowing us to make informed decisions for any adjustments. During prep we also worked with Mike Dawson’s SFX team to investigate ways of moving the basket including a motion base and basket hanging rig. Rehearsals during pre-production provided valuable information on the techniques that would provide the most realistic feel. A series of balloon flights in real balloons for the cast and key crew meant that we all had a common experience to base our discussions on.
Can you elaborates about the creation of the balloon?
Since there was a practical balloon on set, there was no way to hide for us. We just needed to match the real balloon. It was a real hero asset for us, it needed to be seen really close up, far away, sometimes both in one shot, clean or covered in ice and always appear to realistically affected by wind and weather. We had several LODs for the balloon which helped us with managing this complex asset of the various needs we had, but it was still a big asset with over 10 Million faces.
Can you tell us more about his rigging and animation?
The balloon required a pretty complex rig, as you can imagine re-creating a cloth filled with a gas, held together by ropes is always alive and never fully still. We needed to make sure we captured the spirit of the balloon and not suddenly put up a CG balloon that was stiff and uncanny. Additionally, we had digital Amelia climbing up the frozen variant holding on to the netting ropes as she was climbing, bending and denting the cloth and ropes. We paid a lot of attention to the rig and balloon asset, so that we could ensure a consistent visual result throughout the film. Our default balloon actually was utilising some animation cycles in ropes, so when you just placed the balloon statically in mid air, it would already have some life to it.
The aerial cinematography was also an important aspect for Tom to get right and while we laid out all shots according to the heights and travel speeds, we spent quite a bit of extra time on adding the “helicopter” feel to the shots, sometimes in 3D layout and sometimes we layered our comps into several layers to get small bumps and imperfections into the camera moves.
How did you use the cloth simulation during the storm?
Throughout the film, all shots were mapped out to their appropriate travel heights and more importantly, travel speeds. This already helped the animation team to get a sense for the distances we needed to cover and how fast the balloon should move. Of course, those were only starting points and the composition and feel of the shot needed to be right in the end, but the more accurate we would be to physical movements, the better it was for the cloth simulation team. The balloon was always fully simulated, even if animation had the opportunity to add slight deformations to the silhouette of the balloon. Our CFX team, then “inflated” the balloon so the cloth would cushion through the ropes and then the first pass would be simulated driven by the physical movements – only then did we go in and tweak the simulation behaviour. Sometimes we needed to correct a lot as the physical forces were quite strong based on the theatrical needs and composition, other times, we were quite pleased with the “out of the box” results and only did some art directional tweaks.
Can you tell us more about the shaders and textures creation of the balloon?
The balloon was a pretty big hero asset that needed to hold up in all sorts of scenarios. Texture requirements were very high and with all the LODs we ended up with over 700 UDIMs to cover the balloon from all angles. Since our shaders are physically based and we had on-set reference of the practical balloon, we still had a lot of detail work to do, but it was more a matter of tweaking and matching the look rather than writing new shaders for the balloon.
How did you create the various digital doubles?
Digital double creation for Amelia and James was pretty standard using a full body scanning and texture reference shoot. Similarly, the digital doubles generation for the crowds were standard procedure with the addition of creating cloth and texture variants.
Did you develop a specific tools or techniques for the clouds?
Most of our work for the clouds wasn’t necessarily about creating new tools. It was more about how we could use the tools that we had in an efficient way to generate the images and details we needed. For example, we used our instancing tools that we normally use for trees and landscape environments, for the generation of the large cloudscapes. It helped us populate seas of clouds spanning over 60km wide, seen from every angle.
Obviously, we had to tweak and massage all of our tools and shaders to give us the best possible images in a manageable timeframe, only because the environments were so detailed and of grand scale.
Can you explain in detail about the creation and animation of the storm?
The Storm sequence provided a particular challenge in terms of lighting and physics of the animation. It needed to be threatening, yet believable. Lighting wise, Tom wanted the audience to be ‘there’ with Amelia and James, but still be able to portray a mood and a sense of light. Being inside the cumulus cloud would make visibility pretty hard, but at the same time we always needed to feel the mist and fog, the rain and the wind.
The sequence required all departments to pull together. FX and animation had to work closely together to create compositions that worked well for storytelling and editing, at the same time the environment and lighting department had to place the lights and more importantly the lightning bolts in a way that it would work with the animation and simulated rain and clouds. Lastly the comp department had to put everything together in a believable way and re-grade some of bluescreen photography to match and sit with the environment. The comp team used a lot of live action library elements, like water on the lense or real rain shot against black screen to really make the audience feel they can be there with them.
How did you handle the lighting challenge when the balloon is in really high altitude?
The lighting at high altitude was a problem we normally face in space photography, a very bright key light, hard shadows and very little bounce light. However, Tom had a very clear understanding of what he wanted these scenes to look like and based on the principal photography and the concept art from our Art Department, we had a very good guide on where to go.
A really nice sequence involve butterflies. Can you explain in detail about their creation and animation?
The butterfly sequence is a beautiful moment in the film; it needed to be visually spectacular and at the same time support the performance of Amelia and James, not distract. We had quite a few hero butterflies that required key framed hand animation for the foreground performance and interaction with the actors. Additionally we had to fill the sky with over seven thousand butterflies to create a sense of awe when we reveal the swarm. The background butterflies were simulated swarming particles influence by wind and turbulence that were using animation cycles.
Since some of the butterflies were really close up, we put a lot of detailed work into their texture, shading and surfacing. Visual details like the yellow scattering and subframe motion blur also became quite important, as butterfly wings move very fast and we didn’t want to end up with just straight lines of motion blur from frame to frame.
The movie has many long shots. How does that affect your work?
These days long shots don’t scare us so much anymore. If you approach them strategically and break them down into beats where you can we can create long shots pretty efficiently. Launching the final renders still is a bit nerve wracking as you have to make sure you are not wasting render time on the farm, as that can be costly, but most of our tools and working methods today allow us to do a thorough quality control beforehand.
Did you want to reveal to us any other invisible effects?
I always say, if you don’t see the work we have done then we have done our job right and if there are scenes or shots in the film that seem to have no visual effects, then maybe they are non-visual effects shots, or maybe we have just done a very good job.
Which sequence or shot was the most challenging?
I want to say the big establishing cloud shots were amongst the most challenging shots of the film, because they required so much attention to detail and often were long shots, hence the audience had a lot of time to see them on screen and judge whether they were believable.
At the same time, the Storm sequence was probably just as challenging to get right, the cinematography, the physics and the final integration of the blue screen photography into our CG environment demanded a lot of detailed skills from so many departments.
Is there something specific that gives you some really short nights?
Usually what keeps me up at night is not a particular shot or sequence, but on every project I wonder if the audience will believe our work. It is our job to support the narrative and help tell a story and let the audience escape reality for a little while. We look at our work so often and repeatedly that for me the hardest part is to keep that sense of outside perspective and ensure that we don’t spoil the movie experience. For sure, there are always challenges on a show that might keep you up a night or two, but mostly I wonder if we have created believable movie magic for the audience.
What is your favorite shot or sequence?
It’s always hard to find a favourite, because I am very proud of the work we did for THE AERONAUTS, but Amelia climbing out of the basket on top of the frozen balloon at high altitude must be amongst the most rewarding sequences. It’s just Felicity Jones’ performance and our visuals that have the audience at the edge of the seat, supporting that sense of vertigo, the struggle and the beautiful cinematography of George Steel with our skyscapes was every so rewarding to see on the big screen at the Toronto International Film Festival.
What is your best memory on this show?
The teamwork. I always consider myself fortunate to work with some of the most talented artists in the industry, but this project in particular required a great deal of collaboration between the departments to create these images. How the teams worked together to tackle some of the technical and creative challenges was fantastic to see and makes me so proud to have been part of his team.
How long have you worked on this show?
I joined the show right after my delivery of CAPTAIN MARVEL in March and delivered the show in August.
What’s the VFX shots count?
501 shots delivered including 497 shots with CG.
What was the size of your team?
Over 200 artists.
What is your next project?
I don’t know yet.
A big thanks for your time.
WANT TO KNOW MORE?
Framestore: Dedicated page about THE AERONAUTS on Framestore website.
© Vincent Frei – The Art of VFX – 2020
