Back in 2018, Graham Page told us about the work DNEG did on Avengers: Infinity War. He went on to work on Captain Marvel, Avengers: Endgame and Infinite. He joined Framestore in 2020 and worked on The Midnight Sky.
After telling us about Framestore’s work on Avengers: Endgame in 2019, Max Solomon went on to work on Fast & Furious Presents: Hobbs & Shaw, The King’s Man and The Midnight Sky.
Patricia Llaguno started her career in visual effects in 2001 at Mill Film. She joined Framestore in 2003 and worked on many shows such as The Martian, Thor: Ragnarok, Mowgli: Legend of the Jungle and Downton Abbey.
How did you and Framestore get involved on this show?
[MS] We worked on character studies for the “friendly robot” characters 2 years ago during pre-production, exploring how they would interact with Neo and Trinity and so we were expecting to get that work but were very pleasantly surprised when we were awarded the swarm attack sequence at the end of the movie.
What was your feeling to be part of such an iconic universe?
[MS] It was very exciting to be involved in the new film, especially the chase sequence through San Francisco which has some really standout moments.
[GP] The Matrix was such an important film in VFX history and so inspirational to me when it was released so getting to be a part of the film was really exciting.
How was this new collaboration with Director Lana Wachowski and Production VFX Supervisor Dan Glass?
[MS] Dan has worked with Lana on most of her films and they have a fantastic relationship and I had worked with them on Jupiter Ascending which was a lot of fun, and again with Dan on Hobbs and Shaw so I was very excited to be working with them again.
[GP] It was a great team to work with, Dan is really trusted by Lana, and he always guided us towards her vision in a very clear way.
What were their expectations and approach about the visual effects?
[MS] They are very collaborative and open to ideas. Dan is very laid back and patient, happy to talk through issues and give detailed feedback. They really love the process and are invested in every aspect of the shots.
[GP] The expectations were very high with both of them having worked on many huge visual effects films. Their approach is quite an organic one, they gave a lot of trust to us to develop our sequences encouraging us to come up with ideas and solutions.
How did you organize the work with your VFX Producer?
[GP] Danielle Waldren, our producer, did a fantastic job breaking down the work into sensible chunks of work and working with all the supervisors to get a really talented crew onto the show.
How was the work split between the Framestore offices?
[GP] All of the work was completed by our London studio, in the most part we were all working remotely although with various restrictions in the UK changing throughout we were able to carry out some of the work in the office.
What are the sequences made by Framestore?
[GP] Framestore’s work focuses mainly on the third act of the film – the huge crowd swarm sequence, ending with Neo and Trinity’s final leap of faith from the top of the Montgomery building.
[MS] We also animated the Analyst’s cat in the final confrontation at the Analysts house and some “bullet-time” digital extras in the Simulatte coffee shop confrontation.
Can you elaborate about your work on the third act?
[GP] Our work covered the scene in the Simulatte cafe, through to Neo and Trinity jumping off the roof. As well as digital doubles, explosion enhancements, wire removals and digital crowds we had to develop a few key effects such as Neo’s shield and the transitions of Agent Smith and the Analyst being shot and turning into code.
Can you elaborate about the transformation of Jonathan Groff into Agent Smith?
[GP] The agent transformation was something which had been handled in previous films as primarily a 2D morph between two performances. For a couple of reasons we handled the effect slightly differently in this film. We approached the shots by building two digital doubles which we were able to blend between, this was partly to allow for a more exaggerated performance than what was in the original photography but also so that we could do a more sophisticated transition in areas such as the hair which simple dissolved from one persons to the others in previous films. The animation and performance required a lot of work to get to a level which wasn’t too extreme but still had an impact. Once the animation was complete different teams handled different parts of the transformation. Our CFX team led by Michael Thomson handled the hair blending from one groom to another. Nicolas Sauval in our FX team worked on a transition for the costume wrapping the geometry of one cloth sim onto another and creating organic mattes to help the composition which was completed by Demis Lyall-Wilson who put everything together enhancing the effect with a final morph to and from the digi doubles to the original actors.
How did you recreate this classic The Matrix effects?
[GP] We ended up analysing a lot of the effects from the previous films as a starting point for what we were doing. Some shots we referenced directly such as a shot looking up at the helicopter as shells come towards the camera. Others like the bullets being stopped we regularly referred back to and compared to. The filming style was different between the films with the original matrix having a very comic book feeling with precise framings and Resurrections having a more organic aesthetic, using hand held camera work and less staged images. This meant our work whilst often having similar effects we often had to solve new problems mainly to do with their composition.
How did you create and animate the crowd that are chasing Neo and Trinity?
[GP] We built the crowd based on cyber scans created by Clear Angle Studios and Gentle Giant VFX. Even though they were a crowd we treated them as full digital double builds paying attention to their hair and clothing as well as their modeling texturing and look development. As the scene was at night it was important that their silhouettes looked natural incorporating folds along edges of clothing and flowing hair.
What was the main challenge to animate that swarm of people?
[MS] For me the big challenge was creating digital crowds who’s movement would blend seamlessly with the extras in the photography. For health and safety reasons they were unable to film the extras close to vehicles and this meant that in most shots the most hero close up extras were digital.
Can you tell us more about the motion capture process?
[MS] We started out by analysing the sequence and creating a long detailed list of the actions we were going to need, then we found four stunt actors of varying sizes and set up the Framestore motion capture rigs in the Chancery Lane studio, where we spent two days working through the list of actions, capturing every kind of variation we could think of. Often we would capture all 4 actors together so that they interacted and responded to each other’s movements which I think gave them an extra sense of realism. Then we made our selects and cleaned up the capture in animation, removing glitches, posing hands and feet which don’t get captured, improving arcs and contacts. It took a team of six animators about two months to create the extensive library of actions we used in the sequence. In parallel, I made an animatic of the sequence with draw overs showing how many extras were in each shot and what actions they were doing. We reviewed this several times with Dan Glass so that by the time we started dropping in our animation clips we had a really solid idea of what was happening in each shot. We also started animating clips for the actions we couldn’t capture, the digital dive bombers were a particular challenge. We started off using simulations of the impacts but found that while this gave some nice variations they felt too bouncy and so we started animating more realistic variations.
How did you create the various explosions and car crashes?
[PL] We had a lot of practical explosion and car crash footage from the shoot, both for reference and as hero plates. Some shots featured fully CG cars and FX for the crashes as well as explosions, and in others CG was combined by comp to enhance practical photography. The explosions were almost all enhanced in comp using a mixture of 2d elements from our in-house library and machine learning to restore detail lost in the highlights.
Can you elaborate about the city environment?
[GP] Our city environment was very much based on the real San Francisco streets. We used 3D scans and photography to aid the building of a few blocks to use for some of the more extreme. We also made use of photogrammetry using various array and drone photography which was shot to maintain lighting decisions from the set.
The sequence happens by night. How does that affect your lighting work?
[GP] The main factor which made the lighting work difficult was the amount of smoke and atmospherics which were used on set. This meant it was hard to judge lighting levels as there was a lot of lift. The dark setting also meant we were dealing with a fairly small latitude which can be tricky as even a fraction too much ambient or fill could make the characters look overlit and none at all meant they lacked detail and looked simplistic. However the bold lighting choices in the sequence were fun to match with strong backlighting and contrasting colours.
The final part of the chase ends on a roof. How did you help Neo and Trinity to jump into the void?
[PL] The jump from the roof was shot as a practical stunt with Keanu and Carrie-Anne braving the leap from the Montgomery building on a wire rig over multiple takes. We had to remove all the rig obviously but additionally we did a lot of work in comp to also remove the oscillating motion as they bounced and hung from the wires. The building itself was modified for continuity as the helicopter had just crashed into it prior to the jump, so most shots had a CG building or DMP building elements thrown in. There were some sky replacements done for continuity across the beat with smoke, fire, debris and cars also added at street level for shots where the camera is looking down on Neo and Trinity.
Can you tell us more about the Neo power and helicopters destructions?
[PL] The Neo shield was an exciting concept for the compositing team to develop closely with FX. The look needed to resonate with the bullet trail effect in the original films and had to be both subtle and grounded on physical reality. The ripples on the shield correlated with CG bullet force and trajectory but we quickly realised they also needed to be able to be flexibly art directed on a per shot basis in comp, sometimes to maximise their visibility, add visual impact to bullets being stopped or deflected or simply to not obscure our hero’s faces in close ups. We developed a way to do this procedurally in comp by importing the bullet collision map from FX and using the position and vector information to drive a particle emitter in Nuke. This allowed us to drive all the facets of the shield look in comp separately and affect the properties of the various elements, (angle, distortion, brightness, refraction, etc) in a dynamic and nimble way.
[GP] The helicopter crash itself had some fantastic FX sims. The building itself had to be a very detailed CG model with individual bolts and rivets. The sim of the ripple travelling through the building had to account for different materials and details so that it didn’t look like a wobbling piece of jelly.
Which shot or sequence was the most challenging?
[MS] Our most challenging animation shot came early on in the swarm sequence. A fully digital shot where the camera follows the bike closely with Neo punching and kicking through a crowd of oncoming attackers as Trinity skids round the corner. We animated it as a slow motion shot and the speed ramp was done afterwards.
[GP] That shot really pushed our pipeline with hundreds of digi doubles in the shot most requiring simulated cloth in slow motion, and some travelling directly past the camera.
Is there something specific that gives you some really short nights?
[MS] Not really, we’re really good at what we do!
[GP] My son Caspar was born during the show so he managed to help with the short nights!
What is your favorite shot or sequence?
[MS] I love the section where the swarm start dive bombing Neo and Trinity from the high rise buildings. It resonates on many levels and is a surprise while also being a logical escalation in their attack. That was a moment of genius from director Lana Wachowski.
[GP] I always love a helicopter crash and getting to pay homage to one of the all time classics was great fun.
What is your best memory on this show?
[MS] Danielle, our producer, was listening in on animation dailies and could see we were struggling to find reference of a cat jumping and walking along the wall so she shot something with her cat on her phone and it really helped us get a much more physically real performance. A perfect moment of synergy!
How long have you worked on this show?
[MS] We spent about 2 months working on character studies in pre-production at the end of 2019, then resumed in post-production September of 2020, delivering the show in July 2022.
What’s the VFX shots count?
We were initially offered 204 shots, but ended up delivering 443 shots for the film
What was the size of your team?
[MS] The teams grow and shrink throughout the production depending on requirements and the demands of other projects at Framestore, but the animation team on Matrix 4 peaked at 11 animators.
[PL] We had around 35 artists in comp at our busiest stage.
What is your next project?
[PL] I have just delivered VFX for Simon Curtis’ Downton Abbey: A New Era, which has been a lovely contrast after The Matrix.
A big thanks for your time.
WANT TO KNOW MORE?
Framestore: Dedicated page about The Matrix Resurrections on Framestore website.
© Vincent Frei – The Art of VFX – 2022
