{"id":815,"date":"2013-04-10T21:03:42","date_gmt":"2013-04-10T21:03:42","guid":{"rendered":"http:\/\/www.vfxpdx.com\/?p=815"},"modified":"2013-07-06T21:43:44","modified_gmt":"2013-07-06T21:43:44","slug":"i-have-seen-the-future-and-it-is-slow","status":"publish","type":"post","link":"http:\/\/www.vfxpdx.com\/?p=815","title":{"rendered":"I Have Seen The Future, And It Is… Slow"},"content":{"rendered":"
\"Deep<\/a>

Deep data happy dance<\/p><\/div>\n

With today’s formal announcement<\/a> by ILM & Weta Digital that OpenEXR 2.0 is finally pushed out for mass consumption, we can finally (there’s that word again!) do the VFX version of the Icky Shuffle. \u00a0Or whatever Deep data touchdown happy dance you’ve been working on since you saw the Deep data demo with The Foundry back at the Hinge Digital VFX\/PDX blowou<\/a>t<\/a> last fall.<\/p>\n

If you were in hibernation at the time and missed it, or still haven’t had much exposure to them thar Deep renders – in a nutshell, Deep finally gives us a usable Z channel. \u00a0Old depth channels have always been a little bit of a hack, plagued by nasty per pixel sampling. \u00a0Even once aliased and cleaned up, you still commonly had to split renders into pieces and\/or render holdouts to get everything jiving and edges that behaved correctly when composited. \u00a0But things that should work, like Z defocus, would instead wreck havoc and have you walking through a landmine field of broken edges, pops, sizzles, bleeps & blunders. \u00a0Deep data to the rescue! \u00a0Deep allows you to render layered CG uninhibited, in it’s full, juicy glory, and then let the Deep Z information take care of your holdouts and what layers in front of what, and it does this both correctly and (usually) flawlessly<\/em>. Simply put – on a complex film like Avatar where traditionally you may have had characters running through the forest, and had to render characters running through the trees with holdouts here and holdouts there… \u00a0and then (zing!) they change a few frames of animation in the character pass — previously you’d have to rerender EVERYTHING because the holdouts changed as well. \u00a0As of today, those days are in the rear view. \u00a0Deep compositing solves those issues, and everything now works like it should. \u00a0You rerender the changed character pass, DeepMerge it with the existing forest renders and you’re off to the races. ILM and Weta were all over this because it’s the only way they could have finished a film on the scale and scope of Avatar. \u00a0<\/em>If they hadn’t brought back Colin Doncaster & co. to finally nail down what they’d started back on ‘Rings, they’d probably still be working on Avatar here a full 3 years after release. \u00a0No jokin’. \u00a0The fact that this is finally getting pushed out into the mainstream is pretty darn exciting for everyone outside of Weta and ILM.<\/p>\n

What does this mean to us groundlings? \u00a0First of all, by no means will the words “instant gratification” come anywhere near this post. \u00a0This release means things can finally be standardized and the different workflows across software will start to come in line, given another round of point releases or two. \u00a0Deep data has been available for a while, and Renderman + Nuke paved the way, but there were still some inconsistencies as other software caught up to what Weta and ILM were pioneering.<\/p>\n

Renderers will now formalize support, some faster than others. \u00a0(Houdini’s) Mantra, Arnold, and VRay have all had support to some extent already, but then you have a look across the way at Mental Ray and they seem to be lagging far behind and (according to the guys at Hinge Digital) Deep data doesn’t appear to be a blip on the MR radar yet. \u00a0At some point in the near future, all will come around to rendering EXR 2.0 rather than dtex or whatever format was being rendered before.<\/p>\n

Nuke is the first and only compositing app out of the gate to have Deep technology, and rightly so, having developed the tools directly with Weta and ILM. \u00a0Eyeon Fusion will probably get this in there and I bet After Effects will also come around eventually, most likely with this being added to the ProEXR toolset for immediate use with plugins hot for the technology, and eventually by the stock Adobe Z tools themselves.<\/p>\n

In Nuke, other than the initial batch of Deep nodes that were released in v6, you’ll see many nodes and tools start to become Deep compatible – for example, you’ll soon see a “DeepKeymix” and nodes like that start to appear as these things pop up in production. \u00a0Even the current set of Deep nodes will change, as Dr. Peter Hillman & co out at Weta continually push things forward. \u00a0They seem to have made the perfection of the Deep workflow not only a necessity for the coming films, but it’s been elevated to almost “personal mission” status. \u00a0With the Hobbit and Avatar sequels looming, this is more than justified! \u00a0At some point it will make sense to have ALL nodes be Deep aware in Nuke and for it to be tossed around as easily as a Z channel is now, but that is a ways off and you’ll see this duality exist for a while (Keymix vs DeepKeymix, etc).<\/p>\n

\"nirvana_nevermind_adult\"<\/a>

Just like the baby in the “deep” end of the Nirvana Nevermind album cover, Z Channels are all grown up now.<\/p><\/div>\n

As far as the Deep workflow goes – I love it, but I hate it. \u00a0Your first shot with it, you’re immediately hit with the “wow that’s amazing” new car scent as you plug in that first DeepMerge and everything clicks. \u00a0But the luster soon wears off when you realize the huge amount of additional processing overhead and network traffic associated with Deep renders. \u00a0It may be sweet images, but you take the slow boat getting there! \u00a0It’ll bring your system to it’s knees quickly, and your compositing momentum will start to resemble that banana slug you almost stepped on out on your front porch this morning. \u00a0You might as well install a coffee machine at your desk, you’ll be taking so many breaks.<\/p>\n

Case in point: \u00a0on many shots for\u00a0Man of Steel,<\/em> I had volumetric cloudbox renders that were up in the territory 500mb-800mb\u00a0per frame<\/em>\u00a0. \u00a0This is not a tax bracket you want to be in. \u00a0Ultimately, whether you eventually gravitate towards a DeepMerge style of comping or flip it and go with DeepHoldouts, you’re going to want to use the Deep renders to generate your layering and then precomp them out and get them outta the stream as fast as possible, so you can return to “normal” RGBA interactivity and creative flow. \u00a0Comps are supposed to be quick – you lighters can keep your excruciatingly slow little render tile windows, thank ya very much.<\/p>\n

The hitch becomes nodes like DeepDefocus (currently unreleased, but you can use the Bokeh<\/a> plugin from Peregrine<\/a>) and others that are applied further down the tree – and for that, you’ll get to used to dialing values in and then getting them (again) out of your script – and disabling them with the $gui expression. \u00a0All in all, the workflow takes some getting used to, but it’s a small price to pay for the flexibility and power of a Z channel that actually works. \u00a0And things can only get faster & better from here as they experiment with new levels of downsampling the accuracy and compressing the renders.<\/p>\n

The Foundry Creative Specialist Deke Kincaid put out a great collection of links awhile back to help get everyone up to speed on all things Deep. \u00a0Digg ’em:<\/p>\n

original\u00a0deep\u00a0shadow paper:<\/span>http:\/\/graphics.pixar.com\/library\/DeepShadows\/paper.pdf<\/a><\/p>\n

\n
<\/div>\n
other must reads:<\/div>\n
http:\/\/www.deepimg.com\/<\/a><\/div>\n
http:\/\/www.johannessaam.com\/deepImage.pdf<\/a><\/div>\n
http:\/\/www.graphics.stanford.edu\/papers\/deepshadows\/<\/a><\/div>\n
https:\/\/code.google.com\/p\/dif\/<\/a><\/div>\n
<\/div>\n
houdini docs on it:<\/div>\n
http:\/\/www.sidefx.com\/docs\/houdini12.1\/rendering\/deepshadowmaps<\/a><\/div>\n
<\/div>\n
prman docs on it:<\/div>\n
https:\/\/renderman.pixar.com\/forum\/docs\/RPS_17\/index.php?url=deepCompositing.php<\/a><\/div>\n
<\/div>\n
videos on\u00a0deep\u00a0image compositing<\/span><\/div>\n
<\/div>\n
basic intro one:<\/div>\n
http:\/\/www.fxguide.com\/fxguidetv\/fxguidetv_095\/<\/a><\/div>\n
<\/div>\n
Johannes Samm\u2019s Vimeo channel on\u00a0deep\u00a0image tools he wrote for Nuke long before we had a toolset on doing this inside Nuke<\/div>\n
https:\/\/vimeo.com\/user3574023<\/a><\/div>\n
<\/div>\n
Rise of the planet of the apes Nuke video:<\/div>\n
https:\/\/vimeo.com\/37310443<\/a><\/div>\n
<\/div>\n
from prometheus:<\/div>\n
\n
http:\/\/www.fxguide.com\/fxguidetv\/fxguidetv-149-prometheus-visual-breakdown\/<\/a><\/div>\n
http:\/\/www.fxguide.com\/featured\/prometheus-rebuilding-hallowed-vfx-space\/<\/a><\/div>\n
http:\/\/www.fxguide.com\/quicktakes\/prometheus-in-depth-coverage-on-fxguide\/<\/a><\/div>\n
<\/div>\n
also additional\u00a0Deep\u00a0stuff from Abraham Lincoln Vampire hunter here, which Weta did:<\/div>\n
http:\/\/www.fxguide.com\/featured\/vampire-hunter-two-killer-sequences\/<\/a><\/div>\n

 <\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

With today’s formal announcement by ILM & Weta Digital that OpenEXR 2.0 is finally pushed out for mass consumption, we can finally (there’s that word again!) do the VFX version of the Icky Shuffle. \u00a0Or whatever Deep data touchdown happy dance you’ve been working on since you saw the Deep data demo with The Foundry… Continue reading →<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_links_to":"","_links_to_target":""},"categories":[5],"tags":[44,7,8],"class_list":["post-815","post","type-post","status-publish","format-standard","hentry","category-blog","tag-deep-compositing-workflow","tag-nuke","tag-pixel-nerd-stuff"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2Cfrz-d9","_links":{"self":[{"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=\/wp\/v2\/posts\/815","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=815"}],"version-history":[{"count":16,"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=\/wp\/v2\/posts\/815\/revisions"}],"predecessor-version":[{"id":1144,"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=\/wp\/v2\/posts\/815\/revisions\/1144"}],"wp:attachment":[{"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=815"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=815"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.vfxpdx.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=815"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}