Earlier in 2022, Oliver Schulz explained to us the visual effects work of RISE made for The King’s Man. Today he talks about his work on the new Fantastic Beasts.
How did you and RISE get involved on this show?
Our very own Executive Producer Florian Gellinger was in talks with Production Supervisor Christian Manz for a longer time already, even before Fantastic Beasts 3 started production. When that happened the Sequences containing our signature work of Set Extensions and FX heavy Magic where a good fit for the Rise Crew.
What was your feeling to be part of the Wizarding World universe?
The Wizarding World has of course a long history spanning all Potter Films and now the Fantastic Beasts Franchise. So not being a complete Potter Fanboy but definitely interested in these Movies I was very much looking forward to get started on this project. It was very nice to get into the usual project habbit again after the initial Covid Lockdowns.
How was the collaboration with Director David Yates and Production VFX Supervisor Christian Manz?
We exclusively worked with Production Supervisor Christian Manz. What can I say about the collaboration? I could have worked for another year on that project without any problem because it was so much fun to work with Christian and his fantastic team. We would have weekly calls to discuss shots and talk about potential feedback from Director David Yates. Christian was also very keen to break that usual routine and jump in to help if any problem would arise that kept us from moving forward to schedule.
What were their expectations and approach about the visual effects?
Here of course I could tell the usual stuff about being photoreal and support the story but in the case of the history rich Wizarding World, Christian and David also wanted to make sure that every VFX element belonged to that specific World. That starts with magic spells that have already been defined in previous films and therefore need to match exactly this visual quality and ends with specific requests about animation timings to get that little comedic element already present in the Potter films.
How did you organize the work with your VFX Producer?
On FB3 I once again teamed up with Senior VFX Producer Katrin Arndt. Having worked together on some past shows we already had a very good working relationship. Knowing each other’s way of working resulted in having a great spirit throughout the project. With her being in this industry so long I could always trust in her vast experience and could focus on creative problem solving knowing she would keep my back concerning staffing and scheduling.
How was split the work between the RISE offices?
This was a single office show and exclusively run in Berlin. The only exception would be Matchmove that was kindly provided by Rise Cologne and of course a lot of pipeline support from various location like Munich or Stuttgart for example.
What are the sequences made by RISE?
The work was split into two steps. The first one was Postvis for all shots together with Lookdev shots for key features that helped us with asset development. The second step after Postvis approval was to create the final shots. Doing the Postvis first was great for us because we had a very clear vision of what every shot needed and what the final approach would be.
Overall our main Sequences were:
- The Berlin Arrival sequence where we build the whole historic Friedrichsstrasse train station, lots of digital crowd, a very detailed train and some establishing shots for this very train going through Brandenburg landscapes which were either completely or partly full CG.
- Some Berlin shots at night for the Sequence where Newt, Jacob and some others are going towards the German magic ministry.
- The Dumbledore followed Sequence where, who would guess it, Dumbledore is being followed by Credence, through a partly digital Berlin using a magic pocket watch.
- The Jacobs bakery Sequence, where Jacob first meets Lally, which had two big clothing transformation shots in it, some nice New York set extensions, a cool Portkey shot at the end and a very cute digital mouse at the beginning.
- Some shots of different Magical Ministries all over the world during the final election Sequence where people are watching a magic transmission of this election, including one all CG shot of the never before seen Chinese Ministry.
- The Tina`s wedding Sequence at the very end which included a digital snow covered New York with traffic and crowd, digital snow, digital Butterflies and some more cosmetic cleanup shots throughout.
- A dialogue Sequence between Tina and Credence which required the addition of dusty atmosphere. A Sequence inside the German Ministry where Newt, Lally and Jacob search for Theseus which required some Extensions of the physical set, magic writing equipment and a digital door.
- We also created the Postvis for the Sequence where Dumbledore and Credence fight in the dark mirror Dimension. To see how close this, by several other vendors beautifully crafted Sequence, matched our Postvis was very rewarding.
Can you elaborates about the various steps to create an environment?
Everything starts with the questions: what are we seeing in this environment and how close do we get to it? Is it plate based or completely digital? This kind of drives the whole approach of going for a fully 3d environment or relying on a fully or partly DMP approach. Nowadays we tend to go full 3d most of the time to be more flexible towards changes at the end of the schedule.
In case of the majority of work, which is plate based, it starts with reviewing and layouting everything accordingly to the lidar scans of the shooting location which you would typically get from production. In this phase we would make sure to get nice and interesting shot compositions for all camera angles we need to cover. If we have this coverage we can decide on which asset needs to be detailed and what is going to be visible in background only and might be just a silhouette in defocus. So moving forward with asset build you would than build everything as detailed as necessary to be as efficient as possible, because in case of complex environments you could end up with hundreds or thousands of individual ones. So it’s extremely important to stay true to screen size and just build what you need.
The next step would be to lookdev as neutral as possible to make sure that color and materials react in a way that is predictable and works in a wide variety of lighting scenarios. It is always important to lock one element at a time, be it albedo colors, specular roughness or lighting colors to not end up having to tweak everything in lighting or compositing. Once you have your major assets in place and everything is locked placement wise we would typically go over this layout and dress it up with lots of small details to give it some life. Be it small pebbles or grass, some trash or dirt here and there. Put in puddles and start layouting some background animations like traffic and crowds.
When the whole thing moves into lighting you could split this into two steps. Most of the time you have a foreground or mid ground plate action that needs to be inserted into a digital background. The first complex task is to analyze what I would call the “local lighting” a.k.a. what has been done on set and integrate this into the “global” lighting context of our digital scene. This can sometimes be very straightforward when the shot is for example outdoors with the sun as the main light source. In other cases this might become very tricky if lots of light sources have been used on set that for example couldn’t be there physically in terms of space or have a very different falloff compared to what light they should resemble. Imagine a moon that would have almost no visible falloff compared to a big silk on set that has a very visible falloff.
We try to sort most of this out in lighting, but in lots of occasions this is a combination of lighting and compositing going hand in hand to solve these various issues. In some cases it might be even easier to go fully CG to have more control over the lighting. An example would be one big train establishing shot where the camera pans down from sunlit mountains into a valley with than train going past. The following train interior scene has very overcast lighting outdoors therefore we needed to match this in the valley. Instead of grading sunlight down in comp we decided on rebuilding the whole valley in CG to have a much more controlled and clean lighting transition into the train interior shots.
The last step then is the detail lighting that counterparts the layout step to create a compelling image and guides your eye through frame by accentuating parts or let others fall into darkness. Another example would be lighting up windows and signage to give life to a city.
What kind of references and influences did you received?
As this movie played in the 1920s we looked at lots of period reference for set extension related stuff. On top of that a lot of the layout and style was already preplanned by the Production designer and Christian himself. So for the Berlin Streets Production scanned lots of building that where already preassembled into layouts that we could just match in our internal pipeline. So it was pretty clear from the get go where which building was located in our shots. That of course sped things up tremendously.
It wouldn’t be Wizarding World if we wouldn’t reference Potter movies or the two first Fantastic Beasts Films for certain aspects which we did for Environments or Magic. We also received some great conceptual art to help with building assets and locations.
Where was filmed the exterior scenes?
Most of the plates we received where either filmed on a backlot in Leavesden or on a soundstage for interiors. Some of the plates we got for the train establishers were from eastern Europe if I’m not wrong.
Can you explain in detail about the environment creation?
One of the most complex environments we needed to create was the train station. All of this was based on a scan of the original location where we integrated the set build which was done on a soundstage. The hero shot dives down from the roof level looking down at a train, through the windows of the station onto the platform, takeover to a real camera move filming extras on set, digital takeover again through a window into a plate from the magical carriage interior of the approaching train where Newt and his gang arrive in Berlin, through the carriage and then again looking out of the window with Lally to the platform.
Not only was this a huge set build, we also needed to create Digi doubles that did hold up for more screen size than usually. Because of covid shooting restrictions Production couldn’t get too much extras onset so we needed to make up for this and populate not only background but mid ground as well. Because we are transitioning from a CG camera into a filmed plate we also needed to extend all of the action filmed on set to cover not only the longer shot but also the vastly different perspective. This led to building around 10 more hero versions of the extras on set for a seamless transition. On top of that we were the first project to switch to a PRMAN Pipeline at Rise which was a joy to be honest (at least for me to watch), but required a lot of work from our Head of Lighting Bernhard Esperester together with Head of 3d Oliver Markowski and Pipeline Team Luca Scheller and Simon Ohler.
For the various shots in Berlin we used the scanned buildings from production as a base and recreated them to build the digital 1920 Berlin version. That was completed by digital cars and crowds.
For the shots of the train going through the landscape we build a lot of vegetation seen in plates shot by production. These were then also used to create the fully digital Establishing shots.
For the New York Sequences we received assets from the previous film that we ingested and reworked where necessary for our needs. We added a digital Manhattan Bridge and covered everything in snow for the wedding Sequence while also adding traffic and crowd for background life.
We received assets for the Macusa, the British Ministry as well as the Département Français de la Magie to recreate these Environments for the election Sequence. It was really fun to work on these iconic locations known from previous films. On top of that came when we were asked to create a fully digital shot of what should be the Chinese Ministry of magic. We received some Digital Characters from Digital Domain Sequences to use in this shot and with an artwork from production depicting the scenery started to create that shot. We received lots of support material from production who shot stuff in china in the beginning to base everything off of real places. The location was based on the spectacular topography of the Chinese Li river. We also ended up including one of the Cormorants that are traditionally used for fishing on this river.
The cities are seen in various light conditions. How does that affect your work?
Wet cities at night tend to render longer… this is one of the effects of different lighting conditions. We try to lookdev everything in a way that makes them work in any lighting condition. Of course a night city looks realistic a lot faster than one in broad daylight for obvious reasons, but is also more work in lighting with integrating set lighting sometimes or just managing the sheer amount of lights than can quickly grow into the thousands.
Can you elaborates about the FX work?
The biggest shots for our FX Department led by our great FX Supervisor Akin Goecmenli were probably the clothing transformations of Lally and Jacob.
From the first briefing I got of these shots it was pretty clear to me that we needed to spend a good amount of time first to get the assets looking really good, even in close-up, but also work out a plausible mechanic of how these transformations should function. Christian was very clear from the beginning that those transformations should convey a feeling of being done practical by an invisible guy from the costume department rather than blending two cloth layer with a lot of glow or particular effects.
The first shot we got to work on was Lally coming really close towards the camera talking to Jacob while the transformation is happening. So there was really not a lot to hide here, therefore we needed to make sure our mechanic worked or at least kind of. We utilized the sewing patterns of the costumes to cut those into pieces and unrolling these from top to bottom, one after another, always creating a layered approach where we could use an upper layer to hide the fact that we needed to bring in an additional layer underneath. Senior FX Artist Tighe Rzankowski was essential to set up all these techniques to make this work. We not only needed to transform all clothing but the hairstyle as well that was quite different from style A to style B. In the end we kept only Lallys face and hands from the plate, the rest is completely digital. All of this FX mayhem was driven by the thoughtful timing input of Animation Supervisor Alexander Schumann. To go in concert with previous seen transformations in the Potter Universe, David Yates asked for all kinds of breakup and little quirks to give this transformation a character of its own. This was done by emulating this invisible costume guy pushing and pulling the necktie around into the vest or having a little chain behave like a snake. Once these mechanics were defined we applied this to Jacobs transformation as well. The challenge here that made this transformation difficult was that Jacob on set was wearing already his Outfit B which was a long coat covering a lot of his body. So we needed to blend our digital Outfit seamlessly into the plate at one point while recreating a lot of his body digitally that was exposed in his much shorter Bakery outfit A.
The Comp Department led by longtime collaborator Oliver Hohn again did a stellar job blending all layers into the plates seamlessly.
Another fun shot for the FX Department was the Portkey effect Lally was creating with her magic Book at the end of the Sequence. The of course digital Book was left floating out of Lallys hand, creating a Tornado effect of ripped out sheets. Senior FX Artist Artem Smirnov created this whole effect and added all those little details like papers getting stuck on Lally and Jacob or on the facade behind them, or little water droplets being dragged out of puddles on the ground to make this effect much more believable. After a Digi Double takeover of those two actors all the geometry was turned into cloth sheets that was sucked into the portkey origin.
How did you create the train and the huge mountains environment?
The train was created using parts of real train carriages and a steam train that had been scanned by production. We took those scans and adjusted them to our needs in terms of, for example, how many windows the carriages should have, which again was designed by the Production Designer. We incorporated the interior scan of the magical carriage that had been built on stage to fit into one of the cargo carriages. The clue about this magical carriage was, that it’s looking like a normal carriage from the outside and if you’re looking through the window, but once inside it’s the elaborate cognac lounge like interior.
The kickoff for the Mountain environments were plates shot by Production. For the first big shot we used the plate for the head of the shot as is and just rebuild the environment down in the valley to fine control the lighting.
For the second closer shot going towards the train we again used the vegetation assets and snow shaders etc. build for the first shot to create an entirely digital shot. The Background Mountains in this shot is simply a 2.5d Matte-painting.
Which shot or sequence was the most challenging?
I think there were many challenges on this show like, for example the cloth transformations or the big train station shot, but there was one shot that completely stood out. The bad thing is that I will probably never be able to reveal which shot exactly this was but I can tell that it was around 900 frames long, completely in your face full frame CG replacement that got all a vfx shot could ever ask for. Digidouble, rotomation, super complex lighting, shading and simulation work. To top this off it was one of the most complex comp shots I ever had the challenge to work on in my career so far.
So again I would like to take the chance to say thank you to Niklas Roth, my first time CG Supervisor, Mona Wiemer, Laura Staudenmaier, Alexander Schumann, Andreas Schulz and Sercan Topaloglu for knocking assetbuild , lighting and animation out of the park. Akin Goecmenli and Artem Smirnow did the Simulation work of their lifetime and Oliver Hohn and the whole Comp Department, where at some point almost everybody did help out with Roto and Paint, were invaluable to achieve that shot . At last a special THANK YOU goes to Jan Volckmann who outdid himself and created a Compositing Masterpiece that probably won’t be matched for quite some time to come… I think in the end we succeeded and hopefully no one will ever be able to tell that we touched that shot at all.
Is there something specific that gives you some really short nights?
I was pretty confident that my team could pull off all challenges thrown at us during that show so what would keep me awake from time to time was only my 2 year old daughter. However I was pretty relieved once that bespoke shot I can’t name was done and dusted.
What is your favorite shot or sequence?
I just like all of them, even in small shots you find amazing details I can still enjoy after many years in the job.
What is your best memory on this show?
Working with Christian Manz and his production team who made this one of the smoothest and most enjoyable project of my career and of course working with these guys inhouse:
Niklas Roth, Akin Goecmenli, Oliver Hohn, Katrin Arndt, Anja Nussbaum, Belen Heydt, Viktoria Gutheil, Florian Gellinger, Mona Wiemer, Laura Staudenmaier, Timo Scheu, Merten Tremer, Joshua Koilpillai, Daniel Bloecher, Sven Klimm, Carsten Schulte-Braucks, Andre Brumme, Daniel Schindler, Eugen Kelle, Jenny Leupold, Stefania Cancemi, Riccardo Dacquino, Thorsten Schulz, Alexander Schumann, David Hall, Andreas Schulz, Tighe Rzankowski, Artem Smirnov, Benjamin Sugar, Bernhard Esperester, Sercan Topaloglu, Steve Parsons, Christoph Hasche, Jan Volckmann, Thorsten Scholz, Tirza Oudolf, Charlie Wagner, Arne Kirchberg, Eilin Pham, Jan Piccart, Johannes Masanz, Anton Graeven, Giovanni E.Vincenti, Ruth Meridjen, Laurette El-Alam, Rory Wheble, Oliver Markowski, Falk Hofmann, Luca Scheller, Kolja Huebschmann, Johannes Vogt-Reimuth, Jari Koski, Jana Rexhausen, Tomasz Herbst, Florian Wischer, Gregor Loock, Jonas Heinig, Sebastian Elsner, Simon Ohler, Jan Grundmann, Alexander Wlk, David Salomon, Marian Dobrunz, Nadine Heinz, Heike Fekonja, Mareike Zeidler, Constandina Iliopoulos, Valentina Wagner and Julia Boldt.
You rocked it!
(Also again thank you pipeline for implementing a new render engine for us on the fly….xxx)
How long have you worked on this show?
Longest show so far for me, from first Kickoff to last final delivery about 14 month.
What’s the VFX shots count?
So from what made the cut it was around 160 shots for postvis and around 150 for final.
What is your next project?
Something very cool I can’t reveal yet 😉
A big thanks for your time.
WANT TO KNOW MORE?
RISE: Dedicated page about Fantastic Beasts: The Secrets of Dumbledore on RISE website.
© Vincent Frei – The Art of VFX – 2022
