Earlier this year, Glenn Melenhorst explained to us the work of Iloura on THE SPONGEBOB MOVIE: SPONGE OUT OF WATER. He now talks about his new collaboration with director Seth MacFarlane and VFX Supervisor Blair Clark for the return of TED.
How was the new collaboration with director Seth MacFarlane and VFX Supervisor Blair Clark?
We had a fantastic experience working on this film with Seth and Blair. Both were very generous with their time and feedback, and encouraging in all respects. They made the production process very smooth because of this and it was a pleasure to work with them.
What was their approach about the visual effects?
Seth’s notes across TED 1 and TED 2 were very consistent. Aside from lip-sync and general posture notes, it was all about keeping the acting subtle, being careful not to have the brows cross over the eyeballs and making sure Ted had no sense of a smile on his face while delivering a joke. Early on in TED 1 we established what came to be known as ‘the golden rules’ for Ted’s acting and they followed through into TED 2, so the animation process ran very smoothly. Blair made sure the work between the studios matched and all the technical aspects of the job were on track.
How did you approach this new episode?
In terms of Ted’s personality, it wasn’t much different to TED 1. There are the subtleties which are so important to get the emotion across, for example, how much his head tilts coupled with how his eyebrows move. Ted only has button eyes, so it’s these other movements of his head and body that help get the message across to the audience. As for pipeline it was no different. Aside from refining our tools and assets to work in our 2015 pipeline, the process of animation, lighting and compositing a Ted shot was similar. Our lighting pipeline on TED 2 was the one we upgraded the most which was necessary given we needed to complete twice the number of shots in two months less time than the first film.
What are the work done by Iloura?
Iloura was responsible for the animation of Ted, including facial animation and performance, the opening ”cosmic zoom” sequence, Ted’s wedding, modelling of Boston including cars and pedestrians, photo-real CG geese, a choreographed dance sequence and a car chase through a marijuana plantation.
Can you describe one of your typical day during pre-prod, shooting and post?
Pre-production was spent mostly on scheduling and planning our sequences, as well as working on our technical pipeline and plug-ins to ensure we were working to the best efficiencies possible. Most of the on-set supervision was handled by Blair Clark however Avi Goodman (Iloura’s CG supervisor) spent a couple of weeks on set to help supervise a particularly FX heavy scene where Ted and Co. are involved in a car chase through a marijuana plantation.
Then in post, the animation process ran very smoothly in terms of our shot production because we had planned well, and we were of course familiar with the show due to our experience on TED 1.
Can you explain in details about the creation of Ted?
For the first TED movie, we presented a character test based on what we thought a dirty-talking teddy bear would look and sound like. We had some test dialogue from Seth, and we researched some Teddy-bear looks which we worked into a character design. We then shot a scene, integrated our bear and presented it back to Seth and the producers. They loved it, it helped to get the film greenlit and we were awarded a significant amount of work on the show. Our involvement with Ted has always been very integral so it was a natural progression to move on to TED 2.
Can you tell us more about his fur?
We made a few adjustments to fur after TED 1, mainly just tidying a couple of things such as messy hair around the lips which interfered with lip-sync. For TED 2, there were no real special requirements for his fur, other than to look real. The first TED film was lit and rendered using a combination of Vray and 3Delight (Reyes). 3Delight was used for all hair renders, and Vray was used for everything else including Ted’s underlying body, eyes, nose, props, etc. For TED 2 we again chose to use 3Delight, but more exclusively this time. V-Ray would only be used for Baking Global Illumination contribution to Ted’s UV space, the Geese, and for the major environment components of the opening shot. We also moved away from the Reyes Rendering Engine and took advantage of the recent Ray-tracing improvements of 3Delights Path Tracer. This gave us the ability to render Ted within a single pass, including clothing and any scene props he may have interacted with. It also enabled us to achieve a more physically plausible result through the use of area lights.
How did you handle the animation?
Seth used a motion capture system on-set called Moven. Essentially it was a jacket that provided us with reference for his upper body – his arms and the angle of his head. We used this as a basis for keyframing; the mocap data we received went through a process of retargeting to fit onto our Ted rig controlled by our animators. The mocap system gave no facial feedback or legs or body motion, so it was often used as reference and together with captured video reference, the animators had all they needed to understand Seth’s intention for the shot. One of the common things we needed to do was tone down the size of some of Seth’s actions, particularly on the head as Ted’s head is proportionally so much larger compared to his body. Seth’s video reference was always studied very carefully and used as the main point of comparison as it contained a lot of subtle detail that sometimes the mocap did not pick up. Rarely can a mocap file be used without cleanup or retargeting. On TED, the mocap mainly provided our team with fantastic reference, and our animators were then able to add the detail needed. In that respect I feel TED is more of a key-frame show than a mocap show.
A very important part of the animation process was Ted’s face. His range of facial expression is significantly different to a human’s range so conveying the right performance lip sync without teeth and the ability to move the eyes and tongue meant we had to develop our own ‘language’ in terms of how Seth’s performance applied to Ted’s face. For example, subtle shifts in the brows and cheek area were often critical to giving Ted a ‘real life talking teddy bear‘ feel without making his face overly animated.
What are the main differences since the first movie of TED?
In terms of animation, the process was much the same, though this time we went into the show already understanding much of the nuance that Seth was after for Ted. We didn’t need to update the rig because the character in essence did not change from TED 1 to TED 2. Main changes were in the fact that this time Ted wears more clothes, so we improved our cloth pipeline, and we needed to work on a lot more on simulations and FX, natural interactions and so on. We also improved our workflows and rendering technology because we were delivering more shots in a shorter time frame.
Can you tell us more about his Tuxedo and the interaction with the other elements?
We used Marvelous Designer to model all our clothing based on real world suit patterns – just as a tailor would do in the real world. We then took this flat pattern and simulated the patches of fabric into position and ‘stitch’ them together – all in Marvelous Designer. This gave us a much more realistic result to approaches we’d used in the past – it meant the clothing was more likely to fold and buckle where it should. It was 3 years between TED 1 and TED 2, so of course huge improvements are made generally to software and hardware, but for TED 2 we reworked our cloth pipeline to improve our workflows and pipeline for greater efficiency. Ted holds a lot of props, so every interaction with food needed to be augmented with CG reaction.
How did you work with the team of Tippett Studio?
On the first TED, Tippett and Iloura collaborated very closely in order to come up with 2 separate assets that would work for the very different pipelines but would match in every way. Blair Clark co-ordinated a number of trials where iteration of the Bear from the 2 facilities would be compared, scrutinized and discussed. A shot was nominated of a room with a pool of light – and the 2 facilities would render the separate bears into this shot and comparison upon comparison was done until it was agreed we had matched almost exactly. The performance of the bear is mostly dictated by MVN performance for Seth so it was not a huge stretch to keep this consistent across the two facilities. By the time TED 2 came around the precedent had been set, the bear being a known quantity so this time around, apart from some comparison of pipeline restructure, Tippet and Iloura operated very separately.
Because both Tippett and Iloura had proprietary pipelines for rigging and fur, that tech couldn’t be shared. What we did share was his base geo and blend shapes and we bounced the asset back and forth refining his lighting and animation until everyone was satisfied that he looked the same.
The movie opens in space and move to Boston. Can you explain in details about this sequence?
As in the first film, we wanted the camera move to be a single unbroken shot, with no wipes or cheats between takes. In fact, many months of modelling, texturing animation and matte painting went into the shot. To achieve it, satellite and aerial photography was sourced and purchased and Paul Buckley, our FX lead began to block the camera move along with our match move department. As the camera approached the city we decided on what to build VS what to leave as 2D matte painted elements. Eventually we settled on building 10 city blocks of Boston and outlying skyscrapers. This build included all the vegetation, cars, pedestrians, kids on swings debris in gutters etc. We think the results look great and we’re very proud of it.
What was the main challenge and how did you achieve it?
We were very excited to work on the Main Titles scene where Seth realizes a huge Busby Berkley-style dance sequence which has Ted dancing in a tuxedo and spats. The choreography was designed by professional dancers and MVN was collected from one of the dancers to act as our guide for Ted’s motion. Often the MVN would come in later than the shots and as the schedule was so tight we opted to key frame a lot of the dance animation. It so happened we had a dancer in our animation department, David Ward, who was able to give an insight into the animation that other animators struggled with – he lead the way on this scene.
Was there a shot or a sequence that prevented you from sleep?
The shot where Ted fights the goose was probably the most challenging. It’s a long shot, we had a tight turn-around on it and Seth was actively blocking out ideas through to the last week. To achieve it in time, we divided it up between animators and spliced the acting together which helped but it had ramifications on the pipeline; we had to re-render as new animation came down the pipe, new composites etc. So I’m so proud of our team, they pulled it off on time.
What did you keep from this experience?
Probably the most valuable lesson came about by having to turn around a large volume of shots in a short time frame, without compromising on quality. We worked very hard to ensure we got everything to the client on time, and we achieved that. It’s a great testament to how well our teams and departments worked together to ensure communication was spot on and our renders as economical as they could possibly be.
How long have you worked on this show?
About 6 months.
How many shots have you done?
The award was 620 shots which quickly grew to over 800 and then by the end of the show we ended up delivering just over 1000 shots.
What was the size of your team?
120 at its maximum.
What is your next project?
Iloura is currently in production of GODS OF EGYPT, directed by Alex Proyas. We also have some other projects that haven’t been announced yet, so stay tuned!
A big thanks for your time.
// WANT TO KNOW MORE?
– Iloura: Dedicated page about TED 2 on Iloura website.
// TED 2 – VFX BREAKDOWN – ILOURA
© Vincent Frei – The Art of VFX – 2015