Cloud tests

RogueNZ · October 13, 2019, 12:11:12 AM

Hi all,

Thought I'd start a thread to share some of my cloud experiments and ask a few questions along the way. It's been a few years since I last used Terragen and I'm pretty impressed with some of the improvements, RTP and the new cloud shading are game-changers.

Can anyone explain what causes the 'broken off' pieces of clouds? I'm familiar with using noise to generate a height field, which is a 2D surface, so is there something within the Cloud node that interprets the 2D noise function as something other than a continuous surface? A rougher or more contrasty noise pattern seems to lead to more broken off pieces, but this is not always desirable, and so I'm just trying to work out how I can control it.

Cheers,
Brendan

RogueNZ · October 13, 2019, 04:12:19 PM

I'm encountering issues with the renders hanging after upgrading to the latest Frontier build. First time it happened, the render appeared to have completed, except for some of the post process effects (such as bloom etc), and CPU sat running at full tilt for at least half an hour with no visible change on screen. Same deal the second time, except it was halfway through the render.

Oshyan · October 13, 2019, 04:18:40 PM

Which version were you using prior to upgrading?

- Oshyan

RogueNZ · October 13, 2019, 05:02:05 PM

The current free version, as downloaded about a week ago (not at home so cannot check the exact version number)

Oshyan · October 13, 2019, 05:05:20 PM

Hmm, ok, what are your system specs? And can you share a .tgd that replicates this behavior?

- Oshyan

RogueNZ · October 14, 2019, 06:43:15 AM

I've sent you a pm Oshyan.

This is where it's been hanging for the last hour, three hours into the render.

- Edit: disabling the cloud layer seems to ~~solve~~ remove the issue

Ariel DK · October 15, 2019, 10:32:50 PM

For an experiment, this is really a wonderful result! Specially "9_edit1_export"

Oshyan · October 16, 2019, 03:26:57 PM

Thanks! Will take a look as soon as we can.

- Oshyan

RogueNZ · October 17, 2019, 06:43:30 PM

Still struggling with this. I thought it might have been path tracing - I disabled it and thought it solved the issue (and changed a few other settings), but now trying to render at high quality again and the issue is back (with PT off). Happy to open a support thread if need be

Oshyan · October 17, 2019, 06:49:24 PM

We'll get to it shortly, just have a lot going on behind the scenes at the moment.

Thanks for your patience!

- Oshyan

RogueNZ · October 19, 2019, 03:20:21 AM

No worries, focus on that first!

I was able to build up a composite using different layers, disabling soft shadows and DoF for the cloud layer - it seems as though when either DoF or soft shadows is on it just cant cope with the clouds. Gave up with path tracing but it's unlikely that would have any affect anyway. Also forgot to turn back on some of the render filter effects like bloom :/

Might come back to this later but I'm keen to move on to other tests

Hannes · October 19, 2019, 05:59:45 AM

Beautiful image. And fantastic clouds!

Oshyan · October 19, 2019, 04:06:34 PM

I'm testing this now. I'll let it finish at the settings you provided, but I do want to note that during primary rendering it's already using about 42GB of memory. I have 128GB so this is not a problem, and I saw you mention in the PM that you have 64GB. Although memory use at the current, early stage of rendering here is not above what your machine could handle, during the end of rendering memory use can spike due to post processes like GI Surface Details/Enhance Details, DoF, etc. The reason disabling the cloud or DoF or other things would help then is simply because they add to memory use.

Of course you'd see in Task Manager if memory was an issue. Did you look there when you had issues in previous tests?

I think there are ways to optimize rendering of this scene (not least of which would be using the latest 4.4 Frontier build, if you have access) which may let you render it in a single pass even if memory is still the issue.

In any case I may see the same freezing behavior as you once I get to the end here, we'll see...

- Oshyan

WAS · October 19, 2019, 11:56:54 PM

Quote from: Oshyan on October 19, 2019, 04:06:34 PMI'm testing this now. I'll let it finish at the settings you provided, but I do want to note that during primary rendering it's already using about 42GB of memory. I have 128GB so this is not a problem, and I saw you mention in the PM that you have 64GB. Although memory use at the current, early stage of rendering here is not above what your machine could handle, during the end of rendering memory use can spike due to post processes like GI Surface Details/Enhance Details, DoF, etc. The reason disabling the cloud or DoF or other things would help then is simply because they add to memory use.

Of course you'd see in Task Manager if memory was an issue. Did you look there when you had issues in previous tests?

I think there are ways to optimize rendering of this scene (not least of which would be using the latest 4.4 Frontier build, if you have access) which may let you render it in a single pass even if memory is still the issue.

In any case I may see the same freezing behavior as you once I get to the end here, we'll see...

- Oshyan

For v3 clouds, he should be having memory scaling going on if near what's available (caching to virtual memory), which would add to render time, but I don't think it would "Hang" the render. You'll either get a unhanded memory access exception due to lack or memory management and a crash, or nothing (cause RAM isn't tapped in requests).

42gb seems to be just over 32gb (in relation to your 128gb) and seems to be right in the area your system would allocate. This seems to correlate to about 3/2.5gb of usage I'd see with v3 on 8gb of RAM and about the 6/7gb I'd use with 16gb. It seems Terragen scales well when the RAM usage is not actually a physical object that cannot be split up with system virtual memory (as I believe it would want to allocate the entire object to memory/cache; which if it's not available would crash TG; v3 RAM data is much smaller data building up).

A question: Is there any v3 cloud collision going on? This has classically caused my renders to seem to "halt" because of the immense lighting calculations I assume between the two densities overlapping (multiplied?)

Too add to this, I've never crashed with v3 clouds due to RAM, even on a 2GB netbook with adequate virtual memory (1.5 - 3+ * RAM).

Oshyan · October 20, 2019, 12:07:44 AM

From his report it sounds like it's hanging at the *end* of the render, so not necessarily during cloud calculation, but more likely a post process effect of some kind. His scene has all of them enabled, each of which is going to use more memory at the end of the render I believe. Many of these may also be single-thread effects, not certain about that though.

- Oshyan