Back in 2019, Scott Edelstein explained Digital Domain‘s work on Avengers: Endgame. Today he returns to tell us about Free Guy.
How did you and Digital Domain get involved on this show?
We were approached fairly early in the filming process. We have a successful history collaborating with Free Guy’s director Shawn Levy, and the film’s VFX Supervisor Swen Gilberg. Digital Domain worked on both Real Steel and Night at the Museum, films Shawn directed, and The Watch, which he produced. Swen is also a Digital Domain alum, so we were able to quickly build a rapport with the production, and they knew right away that we were capable of bringing their vision to life.
How was the collaboration with Director Shawn Levy and Overall VFX Supervisor Swen Gilberg?
The collaboration between Digital Domain and the filmmakers was excellent, and part of that was thanks to how well Shawn and Swen worked together. You could instantly tell they had worked together before, and Swen was very in touch with Shawn’s vision. Having clear goals made it easier for us to understand what they wanted and it made everything move faster and smoother.
What were their expectations and approach about the visual effects?
Digital Domain was involved very early on, which we prefer as part of our process. We helped create a lot of the film’s previz, so we had a clear sense of what was expected, and in many ways helped shape it. Free Guy was a unique project in many ways, especially around the switch between the “real world,” a live-action world inside the game and a game-like environment filled with avatars. We worked closely with Swen and the rest of the filmmakers to create those game characters and environments, and make them just as fun and original to fit with what Shawn was filming.
How did you organize the work with your VFX Producer?
The Digital Domain VFX producer for this project was Crys Forsyth-Smith, and we worked very well together. I previously worked with her on Avengers: Endgame, so we had a solid shorthand. Together with Digital Producer Kelly Shin and the Digital Domain VFX Supervisor Nikos Kalaitzidis, we had to come up with clever ways to split the team into two, with one group focusing on the “real world” VFX, and the other working on the VFX for Free City.
How did you split the work amongst the Digital Domain offices?
We had artists and production staff in Playa Vista, Vancouver, Montreal and India. A large portion of the work was done in the Canadian offices, split among the large teams there. This was also one of the first large-scale productions that we received after officially opening our new studio in Montreal, which was really exciting for everyone at the studio.
What are the sequences made by Digital Domain?
Digital Domain worked on a few major sequences in Free Guy, beginning with the opening sequence we lovingly called “The BadAss Oner.” The Oner was several shots stitched together to appear like one extremely long single shot, that served to introduce audiences to Free City, and lay the groundwork for the type of place it was.
We also created a sequence inside an unfinished building that turned into a unique chase, the sequence inside the bank where Guy (Ryan Reynolds) works, Guy’s apartment, the Multiplayer Lounge and many of the montage sequences as Guy powers up. We also had shots of the streets of Free City, the Free City Reboot at the end (which we shared with Scanline), BadAss’ hideout, a few shots inside Soonami headquarters and more.
Along with all the live-action work, Digital Domain was also responsible for all the gameplay footage (the footage of the game version of Guys’ world) that takes place inside Free City. This footage can be seen throughout the movie, anytime you see gameplay being broadcast to a monitor. Creating this was somewhat akin to making an animated film, loosely connected to a live-action film that we were also working on at the same time. It was an interesting experience.
Can you elaborate about the impressive opening sequences?
The introduction to Free City has one of the game’s top players, “BadAss,” making a dramatic and action-packed entrance, all leading up to the introduction of the film’s real hero, Guy. Digital Domain collaborated with the filmmakers closely on this sequence, and combined digital and practical effects to create a long, crazy opening shot. It fulfilled the vision of the filmmakers, and helped to establish the world audiences were stepping into.
The sequence begins with the character of BadAss, played by Channing Tatum, performing a HALO jump over Free City. After a lengthy freefall, he deploys a CG parachute at the last second, landing perfectly in a convertible supercar, ready and waiting for him to steal. To make the transition from the fall to the landing, artists at Digital Domain created a digidouble of Tatum, which blended seamlessly with footage of the real actor against a bluescreen.
BadAss is then chased by police through Free City. The filmmakers used a multi-camera rig and a crane on a camera car to film the streets of Boston, with real stunt doubles driving an actual supercar. Digital Domain took that real footage and created a new set of digihumans for each performer, then added CG buildings, streets, props, crowds and a CG model of the car. We then blended the live-action with our CG world, as BadAss dodges grenades, cop cars, trees, motorbikes and more, all added by Digital Domain.
The camera then leaves the action behind and pans up, and through the window of our hero, Guy. As BadAss speeds off into Free City, chased by police and CG helicopters, machine guns blazing, Guy casually wakes up, seemingly unaffected by the insanity happening outside his window.
How did you handle the transitions in this sequence?
Handling the transitions seen in the opening Oner was a huge task. It required several artists working together in unison, under the leadership of Digital Domain’s CG Supervisor Attila Szalma, Comp Supervisor Viv Jim, Associate DFX Supervisor Eric Kasanowski, Comp Lead Chun Ping Chao and Compositor Khari Anthony. We began by breaking the sequence down into 10 unique shots with overlapping actions, then brought them back together. Each required a unique transition. We created a lot of completely CG environments for the sequence, which allowed us to seamlessly blend live-action plates. Most of the time, the car BadAss was driving was CG, as were the passengers.
Can you elaborate about the design and the creation of Free City?
To help create Free City, Digital Domain started with the previz, concept art and real world images provided by the filmmakers. We then created multiple iterations of Free City from both the live-action and in-game perspectives. For the Free City displayed as part of the in-game content, we used the previz assets to build nearly 90 3D environments and created a system that used modular building blocks to increase modeling and texture variation. Artists then added individual flourishes by hand to replicate and stylize the locations and to connect them to their real-world counterparts, including storefronts, residential units, commercial buildings, etc.
Where was filmed the various parts of the movie?
For the live-action shots Digital Domain contributed to, the vast majority were filmed in Boston, Massachusetts. Although there was a huge amount of customizing and stylizing the locations seen in the movie, Boston was the initial inspiration for Free City.
The unfinished building is changing its shape all the time, how did you handle this challenge?
The scene in the construction site, where Guy is chased up and through an unfinished office building, was one of the more demanding sequences we worked on. It’s also one of the most impressive.
We began by building the assets and laying them out, then animating them — that was the easy part. The challenge was incorporating all the additional FX needed to make the constantly moving pieces look and feel real. It also required a lot of additional render power to create the motion blur.
We tried a few different options, but the timing and movements we received from postviz were so specific that it ended up being much more straightforward to just animate sections and continually fine tune portions. Once we got the rooms moving, we then offset the blocks, windows and props that all helped bring it to life.
Once Guy escapes the building, we took live-action footage recorded from a drone and added the Free City skyline. In the film, Guy uses a special item that encases him in a protective bubble that allows him to safely fall down to the street (bouncing off surfaces all the way down). For this, the drone rapidly descended to convey the sense of falling. We then used a combination of practical and digital effects, including the footage of a stunt person standing in for Guy, shots of Reynolds himself and digidoubles.
How does that affect your lighting and textures work?
Working with lighting in a constantly shifting environment can be tricky. We kept having to account for large shadows cast by offscreen sections or props, and each shot ended up essentially focusing on the hero key, lit with animated lights and specific light/shadow/object omissions and the like. Thankfully we were moving complete assets throughout, so the lookdev wasn’t an issue.
Can you tell us more about your work on the Free City’s “Multiplayer Lounge”?
One of the most important moments in Free Guy occurs when “Molotov Girl” (Jodie Comer) takes Guy to the Multiplayer Lounge. It’s here that she reveals to him that his home of Free City is actually just a game. The majority of the Lounge itself was a physical set, but Digital Domain was responsible for extending the environment to make it feel like a massive underground lair that went on forever. We ended up using a combination of CG assets and digital matte paintings, then we added several key details.
For the Lounge, we created all the graphic elements. We also added all the crowds, which were a mix of CG characters and elements from real photography — basically, real people on 2D cards we could place wherever we wanted. The Lounge also displays gameplay on multiple screens, all created by Digital Domain, including scenes showing Guy as a digital avatar, reliving some of his more painful moments in the game.
Guy’s exploration of the Multiplayer Lounge is cut short when the game’s programmers attempt to reset everything. Everyone freezes in place except for Guy, allowing him to wander through the crowd and out into the street. There, the game collapses around him. The filmmakers shot real actors holding VERY still, and we later went in to stabilize them, making them look as if they were frozen in time. Some of those actors were also replaced completely with CG digidoubles and still frames of real people.
How did you handle the stylish destruction sequence of the city?
Scanline was responsible for the destruction of Free City, and they did a great job. Digital Domain handled the de-resing of the city itself. Once Guy was outside of the Multiplayer Lounge and on the street, we used a combination of 3D and 2D effects to pixelate, de-res and ultimately wipe clean the world of Free City. We started with plates provided by the filmmakers, again with real people holding very still while Guy walked through the crowd. The complexity of this shot made it difficult to add environmental elements like vehicles, water, fire, smoke and even hero FG elements like the frozen bullet Guy avoids, so instead we built a lot of the environment in CG and completely replaced the practical elements. That gave us total control over everything, including the ability to properly light and layer all the pieces we were adding.
What was your approach to create the “digital” version of the characters and the city for the game version?
For the gameplay look you see throughout Free Guy, we went through several iterations and stylized versions with the director and his team before we all agreed upon the look you see in the film. We looked at existing games, and took a lot of inspiration from Apex Legends, Grand Theft Auto, Fortnite and Overwatch. One of the things that attracted us to those titles specifically is that they all blend a heavily exaggerated “toon” look with photorealism (to a degree).
Once we settled on the style, we needed to make sure that the characters were still recognizable in their digital avatar forms. It was a tricky blend of techniques to ensure that the gameplay characters were different enough from the real-world characters so that there was never a moment when audiences felt like they were staring into the uncanny valley when the film cut from plate to gameplay, while still looking like the real performers. Ultimately, we based our gameplay character assets on our digidoubles, but dialed back the detail in the models, textures and lookdev. We then smoothed out their features, and took advantage of different MIP levels to add a slightly more toon-style adjustment in our shaders. They were also rendered with a small shutter angle (usually 90 and sometimes 45-degrees) to give that slightly jerky video game motion.
For our environments, we took a “simple-is-better” approach for the video game look, and utilized a lot of the Digital Domain’s postviz assets as a base. We touched up, re-lookdev’d and in some cases re-ropologized a lot of the assets, then slotted them into our VFX pipeline. In Free Guy, the gameplay shots are generally displayed on smaller monitors within the film, so having lower-fidelity assets didn’t impact the quality, and it saved us a lot of overhead when it came time to render.
Can you elaborates about the proprietary tool, Charlatan?
Charlatan is a proprietary machine learning technology, created internally at Digital Domain by our Digital Human Group, that allows us to enhance our CG characters based on sets of training data. It can take existing footage and analyze an actor’s performance, then introduce a new, digitally constructed face, which can replace or augment the original performance.
With enough training data, the system is able to recreate a face, frame by frame, with stunning accuracy. That means artists can alter the facial performance to incorporate new expressions or dialogue. It opens up a lot of possibilities, and in this film it allowed the filmmakers to reshoot a pivotal scene when the actor wasn’t available.
How does this tool help you the most on Free Guy?
Towards the end of post-production, the filmmakers realized that a vital piece of dialogue delivered by the character BadAss, played by Tatum, needed to be replaced. Traditionally, this would mean costly reshoots, as well as coordinating and working around the schedule of one of the most in-demand actors in Hollywood. Instead we looked at ways to animate the digital character in order to accept the new dialogue.
We tried several methods, but the result just didn’t fit with the rest of the sequence, so we decided to try Charlatan. The result was a more realistic digital character that could mimic the actor’s facial mannerisms and movements, but could still be changed by the artists at Digital Domain. The actor later recorded the new audio, which was overlaid with his digital likeness.
Which shot or sequence was the most challenging?
The most challenging sequence for us was definitely the opening Oner with BadAss. We split off an entire team in Montreal, headed by CG Supervisor Attila Szalma, just to tackle what ended up being one of the most impressive shots in the movie.
Is there something specific that gives you some really short nights?
The scenes inside the construction building with Guy being chased by the characters Keys and Mouser was one of the more demanding sequences we completed, but it was also one of the best. There was a ton of animation required for both the characters, and the building was constantly being altered from the real world in an attempt to stop Guy. Elizabeth Bernard, Digital Domain’s animation supervisor on Free Guy, had her hands full and did an excellent job.
The building was full of CG props, glass, dust, debris and fast moving objects. Rendering all this and making it look believable was tough. The entire scene also had to be lit from outside, which means the interior is mostly bounced light, which tends to cause render times to skyrocket. CG Supervisors Bob White and David Cunningham took on this incredibly complex sequence and handled it well. It turned out to be one of my favorites.
What is your favorite shot or sequence?
I really enjoyed the sequence when Guy fights Dude on the beach. We handled the gameplay version of the fight, and it was as fun to shoot as it was to watch.
What is your best memory on this show?
Post-production for Free Guy took place right as COVID-19 started shutting down the world. My best memory from this project is how well our team was able to cope with that transition. It was such an uncertain time, and we really came together as a company to get this done.
How long did you worked on this film?
I joined the project in April of 2019, and we wrapped up around July of 2020. So overall, a little over a year.
What’s the VFX shots count?
Digital Domain delivered 348 shots for Free Guy, along with 87 separate gameplay shots that were composited into some of the VFX shots.
What was the size of your team?
It’s tough to give an exact number of people that worked on a project like Free Guy, since the team expands and contracts as necessary. That’s one of the strengths of a company like Digital Domain. Overall though, there were around 140 artists in North America and more in our Hyderabad office, 27 production members and supervisors, the previz team and more. In total, there were roughly 240 Digital Domain employees that had a hand in this film.
What is your next project?
For Digital Domain, our next project to be released will be Shang-Chi and the Legend of the Ten Rings. For me personally, I’m currently the visual effects supervisor on Spider-Man: No Way Home for Digital Domain, which is set to hit theaters later this year.
A big thanks for your time.
WANT TO KNOW MORE?
Digital Domain: Dedicated page about Free Guy on Digital Domain website.
© Vincent Frei – The Art of VFX – 2021