Julian Foddy began his visual effects career in 2006 at DNEG. He works on films like GREEN ZONE, PAUL, FURIOUS 6 and HERCULES. He joined the ILM teams in 2015 and took care of the effects of films such as TEENAGE MUTANT NINJA TURTLES: OUT OF THE SHADOWS, TRANSFORMERS: THE LAST KNIGHT, READY PLAYER ONE and SOLO: A STAR WARS STORY.
What is your background?
I had a somewhat unconventional path to where I am now – I spent 14 years working as an international DJ and record producer before deciding on a career change. In 2003 I went to university to study games design, and the 3rd year of my course was an internship. I’d become very focused on the 3D modelling / animation / rendering side of things, so decided to try for something in VFX, and I was lucky enough to be offered an internship with Double Negative which then led to a permanent job. I worked as generalist TD, then CG supervisor, then VFX supervisor. In 2014 I got an offer from ILM to join their London team as a VFX supe and decided it was time for change! Since I joined ILM I’ve worked on some amazing projects: READY PLAYER ONE, SOLO: A STAR WARS STORY and this show (SPIDER-MAN: FAR FROM HOME) being the highlights!
How did you and ILM get involved on this show?
We were actually quite ‘late to the party’ – shooting was already well under way when ILM was awarded the work, but luckily, we were just in time to get on set for our sequences. I think the show awarded on a Friday, and we were on set in Bethnal Green shooting the charity event on the Monday!
What was your feeling to be part of a Spider-Man movie?
It was really exciting actually – this was my first Marvel show and I couldn’t have asked for a better or more iconic character to work with as my introduction the the MCU!
How was the collaboration with director Jon Watts and Overall VFX Supervisor Janek Sirrs?
Jon was great to work with on set, very easy to talk to and accommodating. I also had a great collaborative relationship with Janek; we’re quite similar characters and we seemed to be very much on the same page creatively most of the time. We also share that rather dry English sense of humour – many of our regular cineSync session often consisted of myself and Janek reminiscing about 1980s British TV shows, often to puzzled expressions from the younger members of the crew!
How did you organize the work with your VFX Producer?
Myself and Claudia Lecaros, the ILM VFX producer on the show, worked very closely together to manage the schedule and bid – we hadn’t worked together before but made a great team and are currently working together again!
How did you split the work amongst the ILM offices?
The work was actually carried solely in the London office – the overall shot count was 177 shots so we handled everything in the one location, with a team that peaked at 114 artists. From initial bid to final delivery was almost exactly one year.
What are the sequences made by ILM?
We worked on three sequences primarily (two in the final film), each with their own challenges:
-The ‘Charity Event’ (pretty much the first sequence of the movie), where Aunt May and Peter have organised a fundraising event for people made homeless by ‘The Blip’ (as the ‘Thanos Snap’ from AVENGERS: ENDGAME is referred to in this movie). This sequence required intensive ‘match-anim’ as the shots were filmed with Tom Holland wearing a grey ‘fractal suit’, and it was really important to Jon that we captured all the nuances of Tom’s physical performance, so even when he’s on the stage in the full Iron Spider suit, you really know its him.
-The final New York sequence which ends the movie and resumes mid credits. Here we blended plate photography shot on location in NYC with fully digital environments, to have Spider-Man swing and fly through midtown Manhattan before picking up MJ and taking here along for the ride, before some revelations shown on a jumbotron screen outside Madison Square Garden
In addition to these sequences we contributed to the final battle scene in London – ILM handled the shots of Nick Fury and Maria Hill in the ‘SHIELD’ ops centre in the iconic ‘Shard’ building.
Can you explain in detail about the creation of the Spider-Man suit?
We actually restored or ingested shared assets from other vendors – for the HOMECOMING blue and red costume seen in our trailer shots we restored the suit from CIVIL WAR and re-look devved to meet our needs. The Iron Spider was a model developed by Framestore and Weta for INFINITY WAR, but again we added our own look development and finessing to suit the requirements of the Charity Event sequence. Marvel loved how the suit looked in those shots!
Similarly, the ‘Enhanced Suit’ (Black and Red) was a shared asset with Sony Pictures Imageworks, so we got their model and textures and look developed to suit our NYC sequence (we had great reference as Tom H wore a practical suit in the plates for those shots)
Can you tell us more about the rigging and animation?
Our creature team was headed up by Andre Metello who I have worked with on numerous shows over the years, and here at ILM the creature dept handle both the rigging and simulation tasks for any given character – Spider-Man ran through a process of muscle and flesh simulation to preserve his volume throughout the performance. On top of which the Enhanced Suit cloth simulation would be solved, with different material properties to replicate the practical suit.
The cloth would require several resolutions of wrinkle simulations, mixed together depending on the shot requirement, and artistic direction from myself and Janek.
The Iron Spider suit ran through a similar process to preserve underlying volume, and the suit was then solved on top, to slide and retain stiffness into the harder metal areas.
For the shots in which he was not wearing a mask, we also solved additional simulation layers to achieve telescopic sliding of the collar for more accurate interaction with the neck and outer metal suit.
What are the main challenges about his animation especially when he is moving in the air?
For shots which had spider-man swinging through NYC, the creature rigging team provided animation with a pendulum tool to solve physically accurate simulations of the swing motion arcs. Myself and Janek were very keen to make this swinging sequence feel as physically plausible as possible – this is reflected in the camera moves as well as the animation. Something we spent a lot of time on in the key framed swinging shots was his sense of weight and really getting a feeling for the physics of how a body would react to the jerks of web lines pulling taught etc. We paid a lot of attention to pivot points, and how his centre of gravity would alter based on pose etc.
How did you create the shaders and textures?
The shaders for each of the different Spider-Man suits used new ILM technology known as LaMa – a new node-based shader approach for Layered Materials. This offered us a more flexible framework for modelling the different looks and responses of the different suits. For example, Marvel manufactured a fabricated cloth suit which was worn on set and on location by Tom Holland, and also filmed the suit under outdoor lighting conditions. This provided the best reference for us to study the properties of the material; how it reacted to light when it was far from camera and small in screen space, and also when it was close to camera and you could see all the details of the fabric. By leveraging the LaMa shading framework, we have extremely flexible nodes to be able to control lobes and layering in a way which we didn’t have control over before.
The shading is of course physically correct with correct energy conservation, leading to very realistic shading results.
Can you tell us more about the design and creation of the new webs?
The look of Spider-Man’s webs had already been established in SPIDER-MAN: HOMECOMING, so the FX crew led by Tobias Keip had a pretty good visual target. The design and behaviours of the web was primarily driven in Houdini. We developed a set of tools that would allow us to simulate individual threads of web and then procedurally add extra detail in a post process.
For the dynamics aspect we used a combination of Houdini’s Vellum Solver as well as Houdini’s Grain Solver. Particularly for complex web sims we found that Vellum would result in a much more stable outcome than the equivalent particle constrained grain solve. However, in simple scenarios e.g. Spider-Man swinging through NYC the grain solver proved to be very fast and reliable. The toolset also included VEX based tools to art direct web shooting, splatting, sticking to and wrapping around objects. We also added the functionality to apply motion filtering to remove very high frequency flickering from the simulations. Once we were happy with the animation we procedurally added web detail by replicating the web threads many times and creating very fine details on the curves. On top of that we added additional connections between the curves to emphasise the characteristic spider web look. The webs were rendered as curve-primitives in both Mantra and Renderman.
How did you handle the lighting challenges?
Lighting was handled with our Katana & PRMan pipeline. Extensive onset high dynamic range photography was captured on set, which our lighting TD department processed for use as lighting and reflection sources. One particular challenge was the Iron Spider suit; being a reflective metallic surface, the suit lighting was extremely sensitive to subtle rotations and movements of Tom’s performance. The many small light sources in the scenes would often need small position adjustments to stop a bright light source appearing or disappearing throughout the shot which wasn’t visible until we’d seen the daily renders; often changing as a department’s work was contributed to the shot, for example a new animation take or new muscle simulation.
How did you filmed the shots with Spider-Man and MJ when they are moving around NYC?
The NYC shoot was great fun – myself and Steve Ellis our CG supervisor were lucky enough to head over to New York. We shot at a couple of locations with Tom and Zendaya, with Tom wearing a practical ‘Enhanced Suit’ (which was ultimately replaced with CG). Both at Madison Square Garden and Grand Central station, we had a crane with a ratchet rig that allowed us to shoot practical, in camera take offs and landings for the web swinging, which we augmented with hand-offs to digi doubles in the final shots. Its near impossible to close an NY street for filming, so in many shots, the background ‘extras’ are genuinely members of the public passing by. Its a lucky thing that real Spider-Man would still attract peoples gaze so we didn’t have to worry about eyeline corrections for anyone who looked at him in shots!
We also shot the ‘bodycam’ swinging shots in the parking lot of the Brooklyn school that doubles for Peter’s high school, with a large blue screen, a couple of ritter fans, and a wire rig that would hold both Tom and Zendaya without interfering with her hair when we turned the wind on.
There was also some second unit work for the shots where Spidey bounces off the roof of a bus – this was shot on a closed street near 7th Avenue early on a Sunday morning, using an ‘Edge Arm’ camera tracking vehicle.
Can you explain in detail about the creation of the various environment and especially NYC?
The NYC environment was done by the ILM London Generalists team, led by Frederic Fourier. Following the success of using Vray for Maya on a previous show, the team exclusively used the same approach for all Spider-Man Gen work.
At ILM we have an enormous catalog of very high resolution, 360 degree spherical panorama photography covering a huge area of Midtown New York. This photography was initially captured for THE AVENGERS, but has been extended and updated on numerous occasions since, including the ground photography captured specifically for SPIDER-MAN: FAR FROM HOME.
Our environment creation process started out initially with the selection of a panorama to use as a shot background from a chosen location within New York City. We would build up an idea for a shot and for the sequence based off the photography available; often trying out different heights and positions to link the panoramas together to form an interesting journey through the city tops. Each of the spherical panoramas are positioned into their respective real-world position’s within a master 3D set of Manhattan, so as we were trying out new ideas for the journey through the city, we can see and select alternate panoramas which may lead us down a new street or Avenue, or up a new building or too a rooftop. Using the panoramas enabled us to block out the whole sequence very quickly using photography – initially to match to the pre-vis, but then later to develop our own creative suggestions and ideas, and present back to Marvel.
As we progressed the shot ideas and the cameras for the sequence became locked down, some shots required developing into an asset build stage, however the panoramas provided a grounding in reality, which showed the environment artists the style of buildings as well as the details and the contrast ratios in the photography which we would ultimately need to match too.
One of the approaches was to use some NYC panos that were shot previously at different time of the day and at different locations, but as the show progressed the shots got more and more complicated. We then decided to rebuild part of the city in full CG, and then to use the panos as a distant background image. This allowed us to be more flexible in terms of camera movement and lighting direction. Not having to clean up the mismatch that arise when you use camera mapping enabled us to focus on designing the shot rather than fixing issues. We could also turn new shots over much faster once the assets were modeled and look dev’ed, as it was just a matter of doing some layout and the shots where almost ready to render.
The efficiency of Vray proxy allowed us to create some highly detailed NYC environments while keeping the memory footprint fairly low and the render time reasonable. Being able to bake the GI as an irradiance/lightcache map was one of the reasons why we could keep the render time low and almost noise free, although we didn’t user this approach on all the shots are some of them were using Brute Force. We also used MASH in combination with Vray proxies to scatter vegetation all over the rooftop garden/atrium that Spider-Man flies through, as well as using Mash to scatter office furniture into the building interiors.
In order to keep the scenes manageable and flexible, we used a very limited amount of shaders: concrete, glass, brick, metal. To get some variations per object, we added « vrayusercolor » or « vrayuserscalar » node into these four shading networks, then drove the look by adding user attributes to the object itself. One of the key benefits was the consistent look we would get across the shot, as it may have looked a bit like a patchwork otherwise. On the other end it could look a little too uniform sometimes, so finding the right balance was very important. All the shaders were built using procedural/triplanar textures, so we didn’t waste time redoing uvs over and over. We relied a lot on the vray dirt as well as the curvature node to achieve the slightly grubby look and the weathering on all the objects. It was a tremendous help as we knew the shaders would work in pretty much every situation, so we would just need to add some more geo/details into the scene and assign those shaders.
For the traffic, we used traditional keyframe animation for the hero cars, and designed a specific tool for all the distant cars. Once again, we used MASH with Vray proxies to scatter the cars along the roads. The tool was very flexible as we could randomize the type of cars, the distribution, change the colour or the density … It was just a matter of a couple of clicks to populate the entire background.
Is there something specific that gives you some really short nights?
As is often the case, it was trailer work that proved to be the most stressful. Probably the hardest shot that went closest to the ‘the wire’ was one of the ‘trailer only’ restaurant shots – at the end of the beat we see the fight aftermath while Spidey explains to the cops what’s gone down. There are numerous ‘goons’ attached not just to the walls by webs, but also hanging from the ceiling in ‘web cocoons’ – these sims required super tight match anim to stick to the actors, and lots of (time consuming) FX iterations to fix pops and intersections. We had a few late nights delivering this for the trailer!
What is your favorite shot or sequence?
Some of my favourite shots are in the NYC swinging sequence – whilst of course aiming for a photo-real look, I was keen for certain ‘key’ frames of shots to look like they could have been a Steve Ditko frame in a classic Marvel comic. You can see these where he hits a certain pose, framed against the mid-town skyline, often with a sun flare for extra dramatic effect!
What is your best memory on this show?
The whole show was a really great experience – we had an awesome team here at ILM, who were an absolute pleasure to work with – not just artists but production and the behind the scenes support staff who often don’t get a mention. On top of that, the relationship with the clients couldn’t have been better!
What is your next project?
I do have a new project but can’t currently disclose what it is… it’s very different to SPIDER-MAN, I will say that!
A big thanks for your time.
WANT TO KNOW MORE?
Industrial Light & Magic: Dedicated page about SPIDER-MAN: FAR FROM HOME on ILM website.
© Vincent Frei – The Art of VFX – 2019