Clouds sculpting & atmosphere layering for VFX and animation

Started by Maximilosc, October 15, 2021, 04:09:33 pm

Previous topic - Next topic

Maximilosc

October 15, 2021, 04:09:33 pm Last Edit: October 16, 2021, 04:42:54 am by Maximilosc Reason: Edited with the information provided by the community
Greetings Planetsiders,

As I joined the Terragang 3 days ago I am glad to find a community that commited which pushed me to invest myself into Terragen. However the learning curve is steep and I have many questions, for now only regarding animation, cloud shaping, cloud distribution, render layers and general render issues. Feel free to bring your input on whatever point nor general idea about where I might be wrong, right and how you would do it. As time will go on I will update all my progress here.

My project :

I chose Terragen for it's realistic volumetrics rendering. I have to create a scenery for a 1920*1080p animation (30sec faked sequence-shot) taking place in the sky. I only need clouds as you never see the ground. It takes place in a summer blue sky, the sun is still, high noon. Around 10 models are filmed on a green screen. One by one they are supposed to pose laying on photorealistic puffy clouds. As the camera travels, goes up, down and pan between the different characters, almost no camera rotations are involved. Also, I have transitions allowing me to cut seamlessly from one scene to another by passing through a cloud.

My Modus Operandi:

• I create my early scene in Blender so I can export my cameras animation as an .FBX to Terragen 4 Pro.
• I import preview images cards attached to Null Objects also set in Blender to preview where my models are supposed to be during my animation.
• Both my camera and my models are oversized (let's say here a human is around 100meters tall) so the clouds they are laying on have the same level of details that the one you can see from your window. It also allows to have small details in clouds shapes, main features and animation (the size of evolution noise for example).
• I create multiple Cloud Layers within my atmosphere answering different purposes:
- Background clouds field: to have a coherent background during the whole clip
- Middleground clouds field: to have some kind of parralax effect over the background clouds field during the whole clip ( I need to use a distance shader to avoid those clouds to appear to close to the camera)
- Principal clouds: which are multiple individual clouds layers grouped together, shaped and placed one by one as some clouds need to be located at specific areas (ex: where a character is sitting). I also might have to create additional cloud cards splitting some of those clouds in two to have some of the cloud passing in front of a character (ex: in front of a hand or the hips as they are sitting, maybe I can skip the cards here and create some animated cloud alpha masks to apply on a characters mask in AE)
- Foreground clouds: to enhance the depth of the scene by bringing a discrete foreground cloud card between the models and the camera
- Transition clouds: those are individual clouds shaped and placed one by one as some need to be located at specific areas. As the camera needs to dives in and out from them I will render them as cloud cards.
• I render each Cloud Layer separately with Pixelplow in 360p, standard renderer, 2d motion blur with low Anti Aliasing (2), low GI settings (Prepass at 2,2,X / GI in cloud quality at Sequence Quality x1 / Voxel Scattering Quality at 100) optimized with 10 millions of voxels, each of them is rendered in .EXR with a Render Layer Element and extra output images Cloud Alpha, Cloud RGB and Cloud Depth in .EXR. Then I assemble them in After Effect for previewing and early compositing.
• After that I finetune short sequences (30frames) on key moments in higher resolution (720p), standard renderer, 3d motion blur with higher AA (4), higher GI settings (prepass at 2,2,X / GI in cloud quality at Sequence Quality x2 / Voxel Scattering Quality at 300) and optimized with 10 millions of voxels.
• If the previous settings suits me I render each Cloud Layer separately with a 1080p resolution, standard renderer, 3d motion blur, higher AA (6) or higher GI settings (prepass at 4,4,X / GI in cloud quality at Sequence Quality x8 / Voxel Scattering Quality at 300) and optimized with 30 millions of voxels.

My interrogations:

1 - Render separated groups of Clouds with Render Layer group: Answered
I create a dedicated group to render each layer of clouds separately from another within my scene. However, the Easy Cloud 01 located into the Object Group 01 rendered into visible with all other objects set to invisible creates a black image both on the RTP and render, unless I plug it directly into the Platnet 01 athmosphere shader and add the Planet 01 into the Object group. Is there a dedicated process to render only a specific Cloud Layer without disconnecting all the other from the node tree nor duplicating a Planet in each render group?
Edit: Kevin Kipper's input: Cloud layers can not be isolated by using the Render Layer's OBJECT groups.  They are two different types of items.  To isolate a Cloud Layer you need to render separate passes, or add new planet objects to your project and create a Cloud Layer within its internal node network.  You can disable the Planet from rendering as needed.  Group the planet to itself and add it to the Render Layers group.

2 - Not that flat belly cloud: Answered
As I will explore through the nodes to sculpt better clouds I am curious to know if parameters exist to quickly create clouds with a not that flat belly and more rounded shape on their bases.
Edit: Kevin Kipper's input: For cloud sculpting please refer to the online Wiki documentation for cloud layers, specifically the Cloud Profile settings here: https://planetside.co.uk/wiki/index.php?title=Cloud_Layer_v3#Tweaks_Tab

3 - Saw in HQ renders: flickering shadows:
While rendering my previews, I know I shouldn't aim for lowest settings possible but rather for something coherent with the final result I expect. However, if i dont have noise at high resolution and settings tests, I have issues with light flickering through the belly of clouds (similar to a thunderlike effect) when rendering an animated cloud (slow drift + evolution) in 1080p, standard renderer, micropoly details at 0.5, AA 8, prepass 4,2,8 / automatic GI cache / GI in Clouds at Sequence Quality x4, max ray depth 2, Quality: rendering method 3D(volumetric), smallest scale at 0.1 and Optimisation: voxel scattering quality at 200 with 30m voxels and acceleration cache set to none. As I am writing that I just read that Oshyan suggests a GI Blur Radius between 300-500 and Richard Fraser recommands to go around 800.

4 - Cloud Cards and reverse cloud cards: Answered
As you create cloud cards for close cloud details nor animations (ex: my foreground clouds or my transition clouds), what is the opposite way to avoid clouds from a dedicated layer to appear too close to the camera? Is it by using some kind of Boolean set to difference and attached to your camera or by going into render layer/layer settings/near clipping distance and keyframe here depending of the needs? Maybe even a reverse cloud card setting somewhere?
Edit: WAS's input: The distance shader allows you to attach a camera, and use it's scalar gradient from 0-1 to mask out clouds close, or far away from the camera. This will work with an animated camera, as well. Documentation here: https://www.planetside.co.uk/wiki/index.php/Distance_Shader

5 - Render farm and GI cache:
Rendering on Pixelplow means automatic GI cache file and no blendmode for animation. Is it better to submit them a file prepared to bake only GI settings (meaning I only check write to GI cache file, set a folder then send them my .TGD), then gather them into an updated file to resend it with an interpolated blend mode set to X?

6 - Rendering optimisation for an animation containing only clouds:
There are no magical settings that makes a render looks good we agree on that. However here I only need to focus on an animation featuring clouds in 1080p with the less noise possible and rid of flickering. Feel free to share configurations that would look optimized for you as I will do the same while progressing through my project.

7 - Essential extra output images for compositing in Blender / After Effects:
Going through Pixelplow allows you to also create extra output images, for now I can only think to use Cloud RGB, Cloud Alpha, Cloud Depth.

As I will continue to go through the wiki, explore more the forum and consider each one of your inputs I will update every steps and progress I make here posting .TGD, stills and clips. I hope it will create an interesting case about using Planetside's amazing volumetrics for VFX and compositing.

In the meantime I wish you all the best and I thank you all in advance. Take care!  :)
Sincerely, MC.

Kevin Kipper

Hi Maximilosc,

Thanks for your detailed questions and description of your workflow.  I'll try to answer a couple of your issues now and the others a bit later as time allows.

First, I hope you've had a chance to watch the "Terragen for VFX" tutorial series as this addresses a visual effects production workflow.  Here is the link to the first videos in the series:
https://planetside.co.uk/terragen-tutorials-for-vfx-series-1-the-basics/

In regards to Questions #1 and #2:

1. Cloud layers can not be isolated by using the Render Layer's OBJECT groups.  They are two different types of items.  To isolate a Cloud Layer you need to render separate passes, or add new planet objects to your project and create a Cloud Layer within its internal node network.  You can disable the Planet from rendering as needed.  Group the planet to itself and add it to the Render Layers group.

2. For cloud sculpting please refer to the online Wiki documentation for cloud layers, specifically the Cloud Profile settings here:
https://planetside.co.uk/wiki/index.php?title=Cloud_Layer_v3#Tweaks_Tab


Regards,

Kevin

Maximilosc

Good day / evening Kevin!

I thank you for the time you took reading my post and addressing your answer.

As you said I am deeply commited into the "Terragen for VFX" tutorial series, which are pure gems. I am bashing them since the very first day.

Quote from: Kevin Kipper on October 15, 2021, 04:31:30 pm1. Cloud layers can not be isolated by using the Render Layer's OBJECT groups.  They are two different types of items.  To isolate a Cloud Layer you need to render separate passes, or add new planet objects to your project and create a Cloud Layer within its internal node network.  You can disable the Planet from rendering as needed.  Group the planet to itself and add it to the Render Layers group.
Copy that, I was indeed thinking about rendering separate passes but I will give a try to your new planet object grouped to itself with the planet render surface disabled (cf the visibility priority order and the 3 planes example).

Quote from: Kevin Kipper on October 15, 2021, 04:31:30 pm2. For cloud sculpting please refer to the online Wiki documentation for cloud layers, specifically the Cloud Profile settings here:
https://planetside.co.uk/wiki/index.php?title=Cloud_Layer_v3#Tweaks_Tab
I started reading this doc just few minutes before I checked your message :')

Quote from: Maximilosc on October 15, 2021, 04:09:33 pm- Middleground clouds field: to have some kind of parralax effect over the background clouds field during the whole clip ( I need to turn this one into a "reverse cloud card" to avoid those clouds to appear to close to the camera)

4 - Cloud Cards and reverse cloud cards:
As you create cloud cards for close cloud details nor animations (ex: my foreground clouds or my transition clouds), what is the opposite way to avoid clouds from a dedicated layer to appear too close to the camera? Is it by using some kind of Boolean set to difference and attached to your camera or by going into render layer/layer settings/near clipping distance and keyframe here depending of the needs? Maybe even a reverse cloud card setting somewhere?
Concerning those reverse cloud cards problematics, I just read about the Cloud function tutorial by Martin Huisman http://planetside.co.uk/docs/tg2/cloud_function_tutorial.pdf and thought about painting a map to offset (and make disappear) the clouds where my camera is supposed to move. Might be interesting to explore that lead.

Once again thank you for your interest and answers Kevin. I will look forward to hearing from you and explore cloud sculpting for my principal clouds group as to create my transition cloud cards tomorrow.

Regards,
MC

WAS

Quote from: Maximilosc on October 15, 2021, 06:59:47 pmConcerning those reverse cloud cards problematics, I just read about the Cloud function tutorial by Martin Huisman http://planetside.co.uk/docs/tg2/cloud_function_tutorial.pdf and thought about painting a map to offset (and make disappear) the clouds where my camera is supposed to move. Might be interesting to explore that lead.

Once again thank you for your interest and answers Kevin. I will look forward to hearing from you and explore cloud sculpting for my principal clouds group as to create my transition cloud cards tomorrow.

Regards,
MC


Look into the Distance Shader. This shader allows you to attach a camera, and use it's scalar gradient from 0-1 to mask out clouds close, or far away from the camera. This will work with an animated camera, as well.

Maximilosc

Hello and thank you for your help WAS,

The Distance shader is exactly what I need concerning the reverse cloud card and closes the 4th interrogation :)

Thank you again!

Regards,
MC

crisb

Wow! Your project sounds very interesting, and you're clearly on a good (and organized) path.

You'll find there are some incredibly helpful people on this forum, and is a fun community to be part of.

So, welcome aboard !

cheers,

cris.

Maximilosc

Hey hey Planetsiders,

As I have been sick and working on other projects I couldn't update lately but here I am again and will be able to show you some previews really soon I hope.

However, from the start and especially lately I have very concerning questions related to Terragen when used for VFX and especially about extracting multiple passes from a render farm such as Pixelplow.

- Pixelplow does not allow you to create extra output images (and so your render layer elements ticked) as you use their services to render your project online. Therefore, you only obtain a beauty pass when you render an animation (or even stills) there, prevening you to be flexible when aboarding the compositing and post-production stages.

It is so essential, I feel like I am stupid or missing something big... Is there a way to get around that problem using Pixelplow? If not, should I aim for another render farm that allows you to create extra output images? When using Pixelplow, is there at least a way to obtain the alpha channel (the one you are able to see and save in your render window next to the RGB channels) the same moment you are delivered your beauty pass using a render farm? As I am supposed to have 750 frames in the end I cannot do that by hand.

I asked Pixelplow and am waiting for their answer, as I will keep you updated I hope you also have an input on the situation.

In the meantime I wish you the best!
Cheers, MC.

Kevin Kipper

Hi Maximilosc,

PixelPlow does in fact render the selected render layer items.  Just set up your scene with render layers enabled and submit that to the render farm.  I agree that it's confusing in that it appears that the render layers won't render based on the PixelPlow interface, but it does work.

Do a simple test, submit your project and render a frame or two.  It's recommended that you do this anyways before rendering lots of frames.

Let us know how it works for you.

Kevin

Maximilosc

Quote from: Kevin Kipper on November 04, 2021, 01:36:09 pmHi Maximilosc,

PixelPlow does in fact render the selected render layer items.  Just set up your scene with render layers enabled and submit that to the render farm.  I agree that it's confusing in that it appears that the render layers won't render based on the PixelPlow interface, but it does work.

Do a simple test, submit your project and render a frame or two.  It's recommended that you do this anyways before rendering lots of frames.

Let us know how it works for you.

Kevin
Hello Kevin,

Thank you for your quick answer. It indeed works :') despite the fact Pixelplow saying that they do not create extra output images.

I will keep you posted later today with the previews I rendered and with more answers than questions this time, at least I hope!

Thanks again, have a wonderful day!

Regards, MC.

Maximilosc

November 09, 2021, 12:19:51 pm #9 Last Edit: November 09, 2021, 01:36:36 pm by Maximilosc Reason: Edited with the information provided by Pixelplow
Hello Planetsiders, I hope you are all good under the sun!  8)

Almost a month since I started terragen :') and every day feels like the first aha... As the project goes on, I will be able to drop some animations and previews tomorrow.

However, today I'd like to know how to use Pixelplow's render farm with Terragen 4 to render only the GI cache file from a scene.

I follow the regular pipeline to do that (https://www.planetside.co.uk/wiki/index.php?title=Terragen_4_Global_Illumination#Rendering_with_GI_cache_files), when I open the file I submit to them and lauch the Render Sequence on my computer I am able to have my .GIC, but when I send them the exact same file I only have a regular passes (beauty pass, ect). 

I don't really know what I am missing, if you have any input I'd be pleased if you enlighten me about it.

Pixelplow's answer: Unfortunately, we do not have any functionality to produce just the GI cache information for jobs, and no customer have ever requested it before. Creating a GI Cache generally is not compute-intensive, so we would not see a need to have this functionality added to the farm. Still, you can submit your own GI cache file to be used with the rendering, more information about GI and Terragen is listed on our support portal here: https://www.pixelplow.net/support/gi-global-illumination-handling-in-terragen/.

In the meantime, take care.
Regards, MC