It’s your 5th journey to Middle-earth. How did you approach this new trip?
Unlike the last film, in this Journey to Middle-earth we get to see all new locations which brought with it new and fun challenges. With a few new characters added to the mix including Smaug and Bolg we had a very big year of design and creation which always brings with it new learning experiences.
Can explain your collaboration with Senior VFX Supervisor Joe Letteri?
Joe is the overall Weta Digital VFX Supervisor and keeps an eye on all shows. For HOBBIT 2, I was the show VFX Supervisor and worked with all the other supervisors to bring to the screen Peters vision for the film. I also took on the challenge of the Dwarven kingdom of Erebor and the dragon Smaug.
How did you split the huge workload with the other VFX Supervisors?
We split the big hero sequences out amongst a few VFX Supervisors and their teams: Jeff Capogreco took the Sluice sequence. Kevin Smith took the Dol Guldur sequence. Mark Gee took the Mirkwood sequence. Matt Aitken took Thranduil’s Realm sequence. Chris White handled Laketown.
Can you describe to us one of your typical days on-set and then during the post?
A typical day on set would start with a 7:30am call time. Every few days we would move to a new stage with a different set. We would start our day with a walkthrough of the set with Peter so we could get a rough idea of what we would be shooting for the day. While waiting for actors to get prepared we would have time to get the green screens prepared for shooting and work with the art department and grips to make sure we could get the best shots possible. Our onset crew would also take this time to survey the sets and place tracking markers that would later be used to help recreate the cameras in the virtual world. At 8:30am I would typically walk over to our pre-production dailies with the VFX producer Kevin Sherwood. Everyday we would go through any new assets for review to see if any notes were needed. Shooting would usually begin by 9:00am and go until lunch. While film was progressing we would gather lots of data about lighting, camera positions, HDRI and camera metadata. During the lunch break we typically tried to photograph the sets and when possible do a 3D Lidar scan to create a geometric model of the set. Throughout the day I would also spend as much time as possible showing Peter work in progress on the laptop to try and get feedback for all the artists at Weta. Filming would usually finish by 7:30pm. At night we would have time to finish any complex scans and get anymore reference needed for later in post-production.
Once filming stopped Kevin and myself were back at Weta where our day would consist of lots of reviews. We would start the day with a 9am asset review and go through any targets for final for the day. At 11am I would go to Erebor and Smaug dailies in the cinema and go through any new shots for those sequences. At 2pm we would get together with Joe and go through any shots we wanted to show to Peter at the daily review. The Peter review would start at 4pm and we would go through all animation, work in progress shots and anything up for approval. After the Peter review we would have a 6 or 7pm review of all the notes with artists and have a 3d stereo review of any shots ready. At this point I would typically get a chance to go to my desk and finally work on one of my own shots.
Azog is back in this movie. Have you improved his model and lighting?
Anytime we get to work with a character over multiple movies we are able to make improvements. In Azog’s case we improved the muscles in the creature rig to get a more dynamic performance. The textures were updated and detail added for bigger closeups. The biggest change was that animators and TDs had a movie to learn the best way to animate and light this character which always allows for a beSer result.
How did you use the experience gained with Azog to create Bolg?
Bolg’s underlying structure is very similar to Azog. The obvious difference is Bolg has metal bits bolted to his bones and very ripped up lips. Both of these caused challenges for the animation and creatures depts. The creatures dept had lots of extra muscle simulations to calculate for the interaction between the metal and the skin and muscles. Limiting the movement without making the skin look fake was a challenge up until the last week of production. Animation had the challenge of doing lip sync on a character with lips that are ripped and torn.
The new movie features many impressive environments. What was your approach to create so many beautiful and various locations?
Most of the new locations were designed with Alan Lee and John Howe. With a combination of artwork and working with the Environment and Layout departments Alan and John helped to design the all the locations in HOBBIT 2.
Can you tell us more about the use of miniatures?
There were no miniatures shot for HOBBIT.
The heroes are confronted by spiders in Mirkwood. How did you animate the many characters?
The spiders are all animated based on footage of real spiders. Since Peter is not a real fan of spiders the closer we could get the movement to real spiders the creepier they looked to him. Having the spiders move quickly but with enough weight to look large large creatures was a big challenge and took a lot of trial and error to get working.
How did you used previz for the river chase sequence?
We started previz on the sluice sequence in 2009 shortly after finishing AVATAR. For Peter the sequence was all about a fast paced car chase on a river. The camera, the barrels and the speed of the shots had to always be pushing forward fast. Peter worked with the previz department for close to a year coming up with gags and story points to block out the entire sequence.
Can you tell us more about the shooting for this sequence?
The filming of the barrel sequence was done in several different ways. A good portion of the shots were obviously fully digital and based on the previz envisioned by Peter. A few of the shots were done on a simple green screen with actors in dry barrels and river motion added by creative camera work on a steadi-cam. We spent a week dropping barrels into a river in the middle of the North Island and filming them as they traveled down the rapids. Then by adding enhanced env elements, dwarves and orcs into the plates we were able to get some great shots with real water.
There is a lot of destruction in this episode and especially in Dol Guldur. Can you tell us more about this aspect?
Evil emanates from Dol Guldur as the Necromancer gathers his power to wage war on all of Middle-earth. Hundreds of pieces of concept art were done by Alan Lee and John Howe in order to come up with the design that Peter approved. The main idea being that the castle is built on the edge of a cliff that many years ago had collapsed and taken with it half the structure. Now overgrown with trees only small portions of the castle emerge from the forest. Gandalf’s exploration of the imposing structure meant that it had to be photo real at every scale from wide angle establishing shots down to several large set pieces to single wall set extensions. To heighten the sinister nature of the environment, several extremely long camera moves were used to give the viewer different looks at vastly different sections of the castle, often in the same shot. This required Dol Guldur to be completed to a high level of finish everywhere. This all lead directly to another hurdle, sheer geometric complexity. The building alone was hundreds of separate pieces, each of which was high resolution and constructed of hundreds of sub pieces. Trees, vines, leaves and set dressing were added to give the griSy overgrown feel of a once mighty fortress now in decay.
How did you create the impressive sequence in which Sauron reveals his presence to Gandalf?
The fx in the Sauron sequence were all done with particle simulations. Since the fx needed to be very directed and work closely with the sound we opted to animate simple geo and get approval for basic motion before running simulations.
Let’s talk about the impressive Smaug. How did you approach this character and can you explain in details about his creation?
Getting Smaug to convincingly vocalize his speech in a way that maintained all of the richness and nuance of Benedict’s voice was a vital part of making Smaug work as character. His crocodile-shaped shaped snout and lack of humanoid mouth-shapes were an interesting challenge for animators who used reference footage of Benedict’s full body movements as he delivered his lines as a guide.
Smaug’s magnificent presence was brought to life by dozens of animators and artists using keyframe animation techniques. Twice the size of a 747 jumbo jet, Smaug has 300 individual bones, upwards of a million hand-drawn scales and 100 simulated muscles not including nine unique secondary simulation elements like the neck wattle and wing membrane wrinkles. A custom anticipatory muscle firing system was implemented to recreate the biological responses required to move a creature of that size.
Smaug’s fire was a focal point for the development of the character. Starting with the internal glow inside his chest and neck, Smaug creates a fuel-based fire that ignites deep in the throat of the dragon and bursts forth with terrifying power. Created with an in-house fire simulation tool, Odin, the output of the larger fire blasts registered just over 10 Kilotons of TNT.
How did you handle his facial animation and rigging?
As the design progressed, we created the high resolution facial puppet. We needed to maximize his range of expression but preserve the integrity of his scales as the face deformed. To deliver a full range of dialogue, we added complex lip and jaw shapes. We also had fine control over his nostrils, his eyes and brows, and his throat. We built Smaug’s crest of spikey horns to be flexible, so we could animate them to convey his mood and punctuate certain moments.
The fire is really important in this sequence. How did you create it?
Fire was another key component of the film. To cope with the complex, interacting simulations during the Erebor chase we extended Synapse, our effects development package. It is a framework for creating, integrating and enhancing fluid and volumetric sims. We used fluid and particle solvers with it as an entry point for volume and particle renders. We then used our new MIP-enabled solver ‘Odin’ in Synapse for fast, flexible and directable fire sims.
We based Smaug’s fire on a World War 2 flame-throwing tank, using sticky, visco-elastic liquid sim as fuel in a high-res sparse grid fluid solve. We used an ‘up-res’ process which could use a low-res simulation as a boundary condition and re-simulate a sub domain at a smaller cell scale. This way we could quickly shape the fire performance and sim detail only where needed.
The Erebor environment is full of gold. How did you fill it and animate so many elements?
One of the big FX challenges for HOBBIT: DESOLATION OF SMAUG was always going to be the gold which Tolkien described burying Smaug. To accomplish the simulations needed to simulate the amount of coins needed we wrote a new rigid body solver capable of solving very large data sets spread across multiple render wall machines. The Rigid body simulations were output as particle caches with geometry instancing and a new file format that allowed streaming of data in a per bucket format allowed for optimized rendering. The largest simulation reached 18 millions coins and was simulated overnight. For the the coins that were not being interacted with we wrote a procedural to generate non-repeating tiles based on height maps.
What was the biggest challenge on this project and how did you achieve it?
We knew from the start of film 1 that the water in the barrel sequence and the coins in Erebor were going to be FX challenges. In preparation for these our FX team spent a year writing new code for our water simulations and rigid body solvers.
Was there a shot or a sequence that prevented you from sleep?
The Erebor forges sequence was one the last scenes to start and we had a very short amount of time to finish. From start to finish the whole sequence was completed in less than a month.
What do you keep from this experience?
From this movie I learned allot more about camera work from Peter. Working on the mocap stage to capture cameras on pre-animated scenes, Peter was able to get the shots that helped tell the story. Going through all these with Peter was great because I gained allot of camera experience.
How long have you worked on this film?
We started really getting into this film in Feb 2013 and delivered the final shots Nov 25th just days before the premiere in LA.
How many shots have you done?
We completed around 2000 VFX shots for the film.
What was the size of your team?
We had around 1000 people working to finish THE HOBBIT : DESOLATION OF SMAUG.
What is your next project?
We have one more Hobbit movie to do before I can think about anything else.
A big thanks for your time.
// WANT TO KNOW MORE?
– Weta Digital: Official website of Weta Digital.
© Vincent Frei – The Art of VFX – 2014