T4-> Rendering-> Mem maxes -> T4 crashes

Started by yesmine, January 10, 2017, 04:09:39 PM

Previous topic - Next topic

yesmine

I've been trying Terragen 4 for a few days after using and enjoying T3 for a long time, and I'm working with a scene I made in T3 long ago and that rendered successfully. However, in T4, trying to render will crash it every time.

Each time, the render window opens and things get populated as expected, but then it sits on a black screen saying "Preparing to render" and....nothing. Pretty quickly, Terragen becomes unresponsive, and on checking it's using almost ever bit of memory I have (9-10gb of 12gb on this machine). Finally it's using so much memory that Windows begins to complain and eventually Terragen itself crashes. It may take 15-minutes to finally die, with nothing ever showing in the Render window.

As a check, I reopened Terragen 3 and loaded the same scene. There are a lot of warnings that show up (I presume having saved the scene in T4, there are parameters T3 doesn't recognize), but the scene loads completely and the render starts quickly and then completes without any trouble. Memory usage in T3 doesn't seem to go past about 4gb usage while rendering, and cpu usage is about as expected.

So, I'm curious if there's something I don't know about T4 that may be causing it to blow out memory usage? Any suggestions are appreciated.
Thanks...

zaxxon

Are you using the RTP (Real Time Preview) to visualize your scene? If you are, you might want to test the scene with a fresh reboot of TG 4 and not use the RTP to see if that helps.

Oshyan

Terragen 4 does use more memory than Terragen 3, particularly for the Ray-Traced Preview, but also just for general rendering. This was a necessary trade-off to enable much faster rendering of objects and higher quality clouds. That being said, it sounds like you're saying it's using at least twice as much memory as TG3, which I would not necessarily say is typical or expected. Can you show an image of the scene or describe it better (what elements are in it, what settings)? Or better-yet, provide a TGD?

I also want to note that the comparison is of course not really useful if you have made any changes in TG4 that are not reproducible in TG3, especially adding new clouds. If you haven't added anything and have simply loaded the scene and attempted to render in TG4, and you are doing the same in TG3, then it's certainly worth investigating further.

- Oshyan

yesmine

The scene I'm testing with is a desert terrain fly-over animation, and it does have a lot of bushes, cacti, grass, etc, with very distant background mountains with pines. It was all done in T3 originally and has nothing new added. Presently, I'm trying to render GI cache in T3 and it's moving along well enough on this older i7-920 w/12gb ram and an Nvidia card, w/Windows 10. It does vary in ram usage, about 7gb used and moves steadily up to about 9.5gb, but can drop back to 3gb on some parts--but it keeps going. Rendering images works about the same too.

Last night I temporarily switched to T4 again, opened the same scene, and tried rendering GI cache. Nothing. It populated things, but never even began rendering the first cache file..just sat on black screen with "Preparing to render" as before. 5-10 minutes later, still nothing, and the entire system froze a while, then Windows complained about memory being too low, and I finally had to end-task on T4 to get back to Windows. It doesn't do that in T3.

I'm new to T4, and it may be something to do with the RTP. I'll look at it again when the GI cache finishes in T3.
Thanks..

Oshyan

Given that you report the scene uses up to 9+GB of RAM even on TG3, I am not surprised it is at failure point on TG4. TG4 does use more memory, as I mention above, and since your operating system also uses some memory, and any other resident processes, ~10GB is about the maximum you can expect any one application to have available to them on your system. As it happens you seem to have been just about at the maximum of your RAM with this scene in TG3. With the higher memory use in TG4 it seems like it just pushes it over, and it wouldn't take much as I illustrated a moment ago.

I know this is frustrating since it worked on the previous version, but it seems like you may have to work on optimizing the scene a bit before rendering in TG4. On the bright side it should render at least a bit faster in TG4, if not a lot faster.

- Oshyan

Dune

Stick some extra memory in, it's not very expensive. If possible of course.

yesmine

Bit of an update:

I decided to try T4 on a better machine (i7-4770K, 32gb ram, GeForce GTX 760):

It comes up fine and loads the same files as before without any apparent issues. The RTP button is 'off' initially, so it defaults to the usual view.

However, if I press the RTP button...uh-oh. The system starts to hang as memory shoots almost to max, then everything goes totally unresponsive, virtually frozen for literally 22 minutes. I'd opened Task Manager so I could see what was happening, and oddly the cpu usage goes to -zero- and memory was suddenly down to 71mb (yes, mb), all frozen for about 5 minutes. Then memory went to around 15gb with cpu at about 20%, but things are still frozen and stayed about the same for more minutes with some small changes in mem/cpu, Finally, at about 22 minutes, everything came to life and the RTP preview began to render (and it's very nice if I may say so!)

After that things seemed workable, and I can change to/from RTP without anything but a little delay while it updates. Also, I rendered an image to test that part, and it came through in about 52 minutes (which is great imho since it takes 2.25 hours on the other machine, though I don't know how much to credit solely to the faster cpu and more RAM. Still, much better.)

Unfortunately, if I close down and restart everything and touch RTP, the 20 minute freeze happens all over again. Frustrating at best..

I'll test T3 on this machine and see how fast the same scene renders and will report back shortly.
Later...

Oshyan

Wow, that is some very strange-sounding behavior. Would it be possible for you to share the scene with us, complete with any objects and other assets in it? I am guessing there is some asset (object, texture, etc.) in the scene that is causing the RTP some trouble.

- Oshyan

yesmine

Thanks, Oshyan...and you got me thinking, so I tried a few tests that have given me a clue to what may be my issue (emphasis on 'my'). I'll consider sending the whole scene, but I'm a little reluctant to show it all at the moment since it's still a work in progress...however, I'll explain what I've found after your last note.

As of now, I think it's related for the most part to a couple of populations of grasses that I made when I was creating the desert initially. It is a huge area of desert terrain populated with bushes of different sizes, pine trees on mountains about 5 miles away in the background, and two different grasses. I think it's the number of grass objects in those populations that may be too much to handle initially...here's why:

I opened the scene and disabled everything initially, except for some basic shaders. Then one by one I enabled things like the terrain and atmosphere without any real problems. I left the objects for last, then enabling-disabling as I checked them one by one, each time trying RTP. With the bushes and pine tree type objects enabled, the RTP worked quickly enough and I could start a render fast enough to be okay with it. But when I enabled the first grass population (no other objects enabled) I saw the first big delay as the preview went black, and things seemed to not respond and freeze--but after several minutes it finally showed the RTP. Same thing when I tested the second grass population. Ah..so..

I then did a quick test with just the bush populations one by one, and while each would load up fairly quickly, with the RTP going, it was very clear that the higher the number in the population, the more it too for RTP to crunch the numbers, so to speak. As of now, my guess is that it isn't actually the size of the object or how large the area that population covers, but the density of the population (which I suppose means too many individual objects over too much area = just too much to handle with RTP enabled.)

I've just tested enabling all the bushes, trees, cacti, etc... --but not the grasses-- and RTP will load it all (in due time) and though it may take 10+ minutes for a render to actually show progress, the render does finally begin. But if I have everything enabled, grasses included, the whole system may eventually crash, if I can't get enough control to open Task Manager and terminate it. Had to press 'reset' once yesterday when the screen went dark. Oh, I might add that this testing was on my slower machine, so that is a factor.

To stop for now, I'd say that as long as I Resist Temptation to Press the RTP button in this scene, I'm probably okay in T4. I'm going to try turning RTP off in the preview and then hit Render just out of curiosity to see if the render starts more quickly if RTP isn't on in the preview. Will report back.
Thanks again...

Oshyan

Thanks for looking into this more in-depth. I think your preliminary conclusions about the culprit(s) seem quite plausible. Can you report the specific numbers of instances you are working with in each population?

- Oshyan

Dune

To be sure I always render without RTP on. No use for it then, anyway.