Hi all,
Thought I'd start a thread to share some of my cloud experiments and ask a few questions along the way. It's been a few years since I last used Terragen and I'm pretty impressed with some of the improvements, RTP and the new cloud shading are game-changers.
Can anyone explain what causes the 'broken off' pieces of clouds? I'm familiar with using noise to generate a height field, which is a 2D surface, so is there something within the Cloud node that interprets the 2D noise function as something other than a continuous surface? A rougher or more contrasty noise pattern seems to lead to more broken off pieces, but this is not always desirable, and so I'm just trying to work out how I can control it.
Cheers,
Brendan
I'm encountering issues with the renders hanging after upgrading to the latest Frontier build. First time it happened, the render appeared to have completed, except for some of the post process effects (such as bloom etc), and CPU sat running at full tilt for at least half an hour with no visible change on screen. Same deal the second time, except it was halfway through the render.
Which version were you using prior to upgrading?
- Oshyan
The current free version, as downloaded about a week ago (not at home so cannot check the exact version number)
Hmm, ok, what are your system specs? And can you share a .tgd that replicates this behavior?
- Oshyan
I've sent you a pm Oshyan.
This is where it's been hanging for the last hour, three hours into the render.
- Edit: disabling the cloud layer seems to solve remove the issue
For an experiment, this is really a wonderful result! Specially "9_edit1_export" :o
Thanks! Will take a look as soon as we can.
- Oshyan
Still struggling with this. I thought it might have been path tracing - I disabled it and thought it solved the issue (and changed a few other settings), but now trying to render at high quality again and the issue is back (with PT off). Happy to open a support thread if need be
We'll get to it shortly, just have a lot going on behind the scenes at the moment. :D Thanks for your patience!
- Oshyan
No worries, focus on that first!
I was able to build up a composite using different layers, disabling soft shadows and DoF for the cloud layer - it seems as though when either DoF or soft shadows is on it just cant cope with the clouds. Gave up with path tracing but it's unlikely that would have any affect anyway. Also forgot to turn back on some of the render filter effects like bloom :/
Might come back to this later but I'm keen to move on to other tests
finalcomp.jpg
Beautiful image. And fantastic clouds!
I'm testing this now. I'll let it finish at the settings you provided, but I do want to note that during primary rendering it's already using about 42GB of memory. I have 128GB so this is not a problem, and I saw you mention in the PM that you have 64GB. Although memory use at the current, early stage of rendering here is not above what your machine could handle, during the end of rendering memory use can spike due to post processes like GI Surface Details/Enhance Details, DoF, etc. The reason disabling the cloud or DoF or other things would help then is simply because they add to memory use.
Of course you'd see in Task Manager if memory was an issue. Did you look there when you had issues in previous tests?
I think there are ways to optimize rendering of this scene (not least of which would be using the latest 4.4 Frontier build, if you have access) which may let you render it in a single pass even if memory is still the issue.
In any case I may see the same freezing behavior as you once I get to the end here, we'll see...
- Oshyan
Quote from: Oshyan on October 19, 2019, 04:06:34 PMI'm testing this now. I'll let it finish at the settings you provided, but I do want to note that during primary rendering it's already using about 42GB of memory. I have 128GB so this is not a problem, and I saw you mention in the PM that you have 64GB. Although memory use at the current, early stage of rendering here is not above what your machine could handle, during the end of rendering memory use can spike due to post processes like GI Surface Details/Enhance Details, DoF, etc. The reason disabling the cloud or DoF or other things would help then is simply because they add to memory use.
Of course you'd see in Task Manager if memory was an issue. Did you look there when you had issues in previous tests?
I think there are ways to optimize rendering of this scene (not least of which would be using the latest 4.4 Frontier build, if you have access) which may let you render it in a single pass even if memory is still the issue.
In any case I may see the same freezing behavior as you once I get to the end here, we'll see...
- Oshyan
For v3 clouds, he should be having memory scaling going on if near what's available (caching to virtual memory), which would add to render time, but I don't think it would "Hang" the render. You'll either get a unhanded memory access exception due to lack or memory management and a crash, or nothing (cause RAM isn't tapped in requests).
42gb seems to be just over 32gb (in relation to your 128gb) and seems to be right in the area your system would allocate. This seems to correlate to about 3/2.5gb of usage I'd see with v3 on 8gb of RAM and about the 6/7gb I'd use with 16gb. It seems Terragen scales well when the RAM usage is not actually a physical object that cannot be split up with system virtual memory (as I believe it would want to allocate the entire object to memory/cache; which if it's not available would crash TG; v3 RAM data is much smaller data building up).
A question: Is there any v3 cloud collision going on? This has classically caused my renders to seem to "halt" because of the immense lighting calculations I assume between the two densities overlapping (multiplied?)
Too add to this, I've never crashed with v3 clouds due to RAM, even on a 2GB netbook with adequate virtual memory (1.5 - 3+ * RAM).
From his report it sounds like it's hanging at the *end* of the render, so not necessarily during cloud calculation, but more likely a post process effect of some kind. His scene has all of them enabled, each of which is going to use more memory at the end of the render I believe. Many of these may also be single-thread effects, not certain about that though.
- Oshyan
I believe it hung halfway, as seen on his half render image a few posts up.
Correct, it's hanging part way through renders. I was able to render cropped sections (e.g. one for the foreground DoF effect, another for the soft shadows on the windmill), but seems to be when combining some of these aspects that perhaps exponentially increase render times. It crawls down and eventually stops as it gets towards the base of the cloud. The issue was present before I added the second rain cloud layer, but doing that definitely increased render times in that region of the image significantly (more than double).
I'm not 100% sure it is actually hanging, it could still be doing something, but the fact that it will render to the base of the clouds in three hours, but no further progress for at least two or three, makes me think it has. Any tips for optimizing this scene would be appreciated!
Quote from: Oshyan on October 20, 2019, 12:07:44 AMFrom his report it sounds like it's hanging at the *end* of the render, so not necessarily during cloud calculation, but more likely a post process effect of some kind. His scene has all of them enabled, each of which is going to use more memory at the end of the render I believe. Many of these may also be single-thread effects, not certain about that though.
- Oshyan
Only one way to find that out for sure, and that's a windows error event. It'll log it. That should be one of the first steps to diagnosing crashes to see if it's actually RAM related.
It's not actually crashing from the sound of it, so no Windows error is likely being reported.
- Oshyan
OK so I did a series of tests here, I wanted to get as much info as I could before replying.
I did end up being able to finish the render successfully. There is one caveat as far as this being a totally accurate test, which is that the objects weren't included, and though I located the same objects I already had, the textures apparently didn't load properly, so my grasses are all textureless and black. :D I can redo the test(s) again, but I'll note a few things.
First of all the total render time was nearly 20hrs. I limited it to 16 threads to try to more accurately reproduce the memory use you would be seeing on your machine. I have 32 threads available so I could redo it and might see 20% better performance, perhaps a tad more, since all my *physical* cores were theoretically in use in the last render. But the point is it's a pretty long render time.
Second and perhaps most usefully, the render did seem to "hang" after completing a single bucket on the right side of the lower part of the scene. I didn't see any other buckets appearing to do work once that one was finished, which was the main thing that seemed unusual, however this may have mainly been because the ground area is rather dark, so you don't see the in-progress buckets very well. That being said the lower third of the image did take a surprising amount of time relative to the upper 2/3, easily half the render time. With path tracing it's not unusual for the ground to take longer than clouds, but it still was a bit surprising given how little area of the image the ground takes in this case. And in fact I don't think it was the ground so much as the *horizon* that took a long time. I suspect, actually, that it's the lower part of the cloud rather than the distant vegetation, and this could be due to the complex cloud setup you have going on in this scene.
Anyway, without further testing I'd say it's just doing a lot of calculation (perhaps some unnecessarily) in the lower part of the cloud and/or distant horizon. This suggests that optimization may be the best option. Ideally you might find a way to optimize your creation of the cloud's shape with fewer nodes, especially whatever you're doing that affects the bottom area of the cloud. But that can be a long process. Render optimization is easier to apply and I'd say there are a relatively few things you'd want to adjust to get a much faster result for equivalent quality.
The first would be to reduce Micropoly Detail from the high level of 0.8 to 0.5 or even lower, since most (if not all) of your micropoly-rendered surfaces are covered by ray traced objects. This may not have a huge effect because of that, but it's needless calculation in any case, so worth reducing. The other most important thing if you have 4.3 or newer (and especially in 4.4) would be to enable the Robust Adaptive Sampler and go with the defaults at 1/64. I'd also recommend increasing AA to 8 just so your minimum samples ends up at 1 sample at 1/64th. Increase the Voxel Scattering Quality (GI in Clouds tab of GI settings) to 2-400 (I used 400), and also the cloud quality to 1. Then do some crop renders and reduce the Pixel Noise Threshold until the noise level is acceptable to you. I settled on 0.025, which is unusually low actually, but was needed to get equivalent noise level in this case, vs. your original.
In a 16 thread test of those changes here, render time decreased dramatically to a little less than 7hrs, about 1/3 the total render time. Not bad!
So ultimately I don't think there is any crash or even memory-related problem here. I think it's just an unusually intensive part of the scene that makes it appear that the render is hung. I think/hope that if you left it to run for ~20hrs or so, it would have finished. But try the optimization ideas above and see if you can get the scene to render as originally configured, but in a more reasonable amount of time.
- Oshyan
Sorry for the delay replying, I did read this and some of your suggestions came in handy for my latest scene, the time saving has been a huge benefit! Shows how much I still have to learn about these settings.
I also agree that it is likely something to do with the base of the cloud, perhaps the flatness of it and the camera being almost parallel might be causing some interesting ray casting situations? I definitely could have put some more effort into optimizing the node network as well.
Appreciate the effort you put in here Oshyan, cheers.