99% of final pass - Stuck

Started by D.A. Bentley (SuddenPlanet), December 20, 2021, 11:58:35 AM

Previous topic - Next topic

WAS

Yeah but if your project is ballooning that much, and it's making a GISD pass at the same resolution in 32bit or what not, that GISD pass can be around the same size in memory, that's about 150 GB, well over 128GB. It then needs added computation and no doubt memory when it comps the two together.

D.A. Bentley (SuddenPlanet)

#31
OK well I did a whole day of test renders to figure out the threshold at which renders crap out at the end because of what we thought was a memory issue with GISD.
Windows never shows in the Performance Monitor that memory is being used up, so I am curious why that is.  It feels more like a Terragen render buffer limitation.

I have determined 100% that the issue is related to having GISD turned on, but it's not merely a render resolution issue.
It seems to have more to do with how much highly detailed terrain is in the camera view (I am using 32k heightmaps/32,768 x 32,768 pixels).
I am not doing standard landscape renders pointing at the horizon with a sky and ground.  I am rendering 100% ground from a top down view.

Basically what I did was create a simple scene not much different from the default scene, but I added a Heightfield Generate node to make a terrain instead of using my 32k Heightmap.
I will attach the scene here so you can try it out on a machine with enough CPU & RAM.  I left the Heightfield Generate at 4096 so the scene will load faster, since auto generate is on.
The terrain I generated was 32km with 32768 points, so basically the same as a 1 meter DEM.  You will need to up the heightfield generate to 32768 points to experience the hang at the end of the render.

I then did some 16k renders with GISD off to get a pretty fast 16k render on my 24-core Threadripper.  My test scene renders in about 10 minutes.
I suspected that I could get a 16k render with GISD on to finish if I rendered just a small portion of the 32km terrain at 16k rather than the entire thing, and that hypothesis turned out to be right.

Then I rendered from a top down Perspective Camera with GISD turned on to see at what FOV would cause a render to hang.
Having too wide of a FOV on a highly detailed terrain (32,768 points/pixels) will capture an enormous amount of detail in the render.

I also tried a couple different resolutions in the Heightfield Generate node, and was able to get the full 32km terrain to render with GISD on.
So by reducing detail in the heightmap/heightfield it had a similar effect to reducing the FOV, and the render finished.

Incidentally, when I say the render wound hang at the end I mean it didn't finish.  It didn't actually hang Terragen or cause a crash.
It just got to 99% and stopped, but appeared to still be working on something because the "Stop" button to abort the render was still there.
I was able to close the Render Frame Buffer window and continue working in Terragen.

I guess the bottom line is I don't really need GISD on for what I am doing.  For anything high altitude you wouldn't see the details anyway.  Correct?


Derek

WAS

I wonder too if you also played with GISD values. I believe default is 24px? That may be way too big for such tight details. I don't know how it computes but maybe that has something to do with it?

But also, from such far distances, you may not need it. I never notice it actually do anything on orbital shots visibly, though I never do turn it off. But it seems it's not needed. I think this is even seen with distant terrain in haze in normal scenes.

masonspappy

I had a problem like this a while back, but don't know if it's applicable here.  My render was getting stuck on clouds (23+ hour render and it still wasn't finished) and I went back and UNticked the "receive shadows from surfaces" box. Rendering took off again, going from mind-numbingly slow to just annoyingly slow.

Dune

Yes, I also changed parameters while rendering, and it often works. I pause, then change things that won't be very obvious in the render but take a lot of time. Like reflectivity in certain parts. Uncheck and the render goes on much faster.

WAS

Quote from: Dune on January 01, 2022, 05:05:41 AMYes, I also changed parameters while rendering, and it often works. I pause, then change things that won't be very obvious in the render but take a lot of time. Like reflectivity in certain parts. Uncheck and the render goes on much faster.
I always forget the renderer is dynamic like that. Actually could be useful for stuff like what Masonspappy mentioned, get done rendering area you needed receive shadows, and uncheck it where you don't when it gets to it (pausing of course, TG finds it very rude of you to make changes while rendering and may decide to leave you).

D.A. Bentley (SuddenPlanet)

Quote from: WAS on January 02, 2022, 03:39:49 AM
Quote from: Dune on January 01, 2022, 05:05:41 AMYes, I also changed parameters while rendering, and it often works. I pause, then change things that won't be very obvious in the render but take a lot of time. Like reflectivity in certain parts. Uncheck and the render goes on much faster.
I always forget the renderer is dynamic like that. Actually could be useful for stuff like what Masonspappy mentioned, get done rendering area you needed receive shadows, and uncheck it where you don't when it gets to it (pausing of course, TG finds it very rude of you to make changes while rendering and may decide to leave you).

Wow Dune, I had no idea the Terragen renderer could do that!

I would be interested in knowing how to adjust the GI Surface Detail settings but there isn't much in the Docs.  It just explains the rendering process:
https://planetside.co.uk/wiki/index.php?title=Terragen_4_Global_Illumination#Render_GI_Settings

If there is a thread you remember that explains what the settings do please post.

Kevin Kipper

Hi SuddenPlanet,

Be sure to check out Part 23 of the "Terragen for VFX" videos for an overview of the GI Surface Detail settings.  It's about 09:24 into the video.
https://planetside.co.uk/terragen-tutorials-for-vfx-series-5-rendering/terragen-for-vfx-part-23-render-settings-global-illumination/