The VFX team at The Mill used 3D mapping cameras and software to map out the entire warehouse and then match the camera’s movements in post to replace the background behind the animated limbs.
“We recommended ARRI’s LDS lenses as The Lens Data System captures highly accurate metadata about iris, focus and zoom. We managed to pipe the focus data into our 3D camera, enabling us to render in depth of field, using real values from the actual lens. Teaming up with Efficacy4D, we captured a LIDAR scan of the set and then used the scan to ensure that we had a very accurate track. Matching the dancer would’ve been a whole lot harder if we couldn’t trust the camera track. We used the scan to project HDRI images for accurate lighting, which was also incredibly useful for making clean plates.”
Once they had captured the footage they had to animate different parts of the dancer as video progressed.
“By far the most labour intensive part of the project was the 3D tracking (matchmoving). Our team of animators had to position each limb so that it exactly matched Sonoya’s pose on all 6,798 frames. This is one of the most complicated things that we do in VFX and is a highly intricate and skilled job. How did we overcome this? Patience, lots of patience.”
Once the major animations had been made the final polishing was a feat in itself.
“There are moments in the video that Sonoya’s ‘real’ limbs and clothes are behind a mesh limb. For these parts of the image, we rendered full CG elements of her arms, legs, head and clothing so they could be used as in areas where traditional 2D painting might be too difficult…
In addition to this, we had to consider the fact that sunlight could travel through the mesh limb and then hit other areas of her body. In order to achieve the correct lighting, we create a light/shadow pass, which the compositors could use to grade up the live action shadow, to make it appear as if light had travelled through the mesh limb.”
To read their full post about the creation of this incredible video head over to themill.com.