Terragen Render Showing No Progress, CPU Usage Regularly Below 10%

Started by Chaz, August 30, 2013, 09:28:25 am

Previous topic - Next topic

Chaz

I'll try to keep this short and concise (I timed out and don't want to try to rewrite everything.) I got Terragen 3 yesterday. I have a fairly outdated system (Intel Core 2 Duo E4500 processor running at 2.20GHz, with 2GB DDR RAM - two sticks of 1GB each.) Other basic details can be found in the second screenshot. I also run on 32-bit Windows 7, though I've heard that 64-bit is better. My expectations for system performance were pretty low to begin with. First thing I did, I made a basic mountain with a lowland in the horizon, midday, virtually no special effects or anything, no water, no clouds... and I rendered it at 1280x800 pixels, and after an hour or two, it popped out clean, quite realistic, and even astonished my brother (who majored in graphic design and loves 3D landscapes.) Admittedly, he might've been more astonished that I made it on this computer than anything else. Still, came out well, and it didn't even take as long as I expected.

Do note that I had worked a little with Terragen Classic before graduating to Terragen 3 (Free Version.) I tried adding water to that same first landscape, as well as some sun and atmosphere effects, tried rendering it, and I got the 'Not Responding' at the top of all the windows. I tried waiting it out, and after several hours of no response, I closed it and lost all progress. I started anew, did similar things, and it crashed. Turned out that it kept popping up with 'Not Responding' because I raised the memory subdiv cache from 400 to 800. I lowered it to 600, got the same result, so then I lowered it back to 400. Well, no more 'Not Responding' messages at the top of each window! However, there are still some concerns...

I made a new picture, one medium sized island in the middle of an ocean, sunset, some cloud coverage (volumetric cumulus and 2D cirrus), the clouds are set to max quality (my mistake, as I've read this wasn't necessary), have max reflectivity, almost perfectly still waters, a modest array of shaders, atmospheric effects, and lighting effects. Resolution is 1280x800 like the first render, the subdiv cache is back to 400MB, it's set to use 1-2 threads (maximum one thread per processor core), and many of the qualities are set highest or normal. Clouds are set to be 'Optimised.' The render preview was a burden on the CPU and memory, frequently maxing them out, and taking upwards of 30 minutes to come up to 80 quality. The terrain geometry is set to polygons. With how tough the render preview was on the system, I expected the final render to take quite some time and stress both the CPU and memory.

Well, over 15 hours into the render, now, and it looks the exact same as it did only 10 minutes in. Still lots of orange, sunset colored dots at the top... and pitch black rectangular patches at the bottom. I've paused and unpaused multiple times with no chance... The timer just continued from where it left off. Everything on the computer otherwise seems to be running smoothly. The processor seems to rarely get above 10%, usually hanging around 0%, usage when no one is actively using the computer, even if other programs are going like Vuze downloading a torrent, or Firefox just left on, or some other miscellaneous program going. The memory usage rarely goes above 60% usage, even when the computer is being actively used by someone. Perhaps more details could be discovered when looking over the first two screenshots I've attached. I'm also attaching a third screenshot showing the render advanced settings. Nothing special, really.

So, please, if someone has even an inkling of what might be going on, I would appreciate the feedback. I just don't understand why Terragen doesn't seem to be using hardly any CPU or memory, and doesn't seem to be making any progress, and yet nothing else seems amiss!


Side Note: I did get two error messages, "Unknown error occurred in a render thread," but these seem pretty generic and easy to ignore. I got four or five of them on the first render I mentioned that came out just fine. No warnings or further errors have come up since the very, very beginning of the render.

jaf

I would guess something memory related, though the task manager indicates it looks okay (as far as amount free.) However, there is a lot of time that has passed by that is not shown on the task manager performance graphs, since the render is +14 hours.  Then there's the pages and nonpaged.

I would try this. Bring up the task manager before you start another render and watch it and the "red" dots and see how long it takes to get the dots to about where they are in your screen capture.  And watch the cpu and mem usage.  Do this for a reasnable amount of time, say 30 minutes most.

If that doesn't get you anywhere, try killing the render and deselect "Atmo/cloud visible" on the render panel and give it another go (render.)

Good luck and let us know what happens.  Maybe you could upload your scene so one of us could render it and see if there's problems (at least that may narrow the problem down to you computer.  :)
(28aug19) Ryzen 1800x, 970 EVO 1TB NVME-M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 436.15 (27aug19), Win 10 Pro x64, Terragen Frontier 4.4.18 BMark 0:10:18

Dune

So what are your detail and AA settings? Posting your tgd is probably the fastest way to an answer, indeed.

Chaz

After looking at the render and the preview, comparing the two, I realised that the areas that are still just black rectangles are all water, whereas the parts that have dots are mostly land and sky. I'm figuring that it's trying really hard to get the water figured out, even if it doesn't seem like it's doing much. Also, I can definitely post the .tgd. Also, it's now over 36 hours in, and still no clear change. For anyone who wants to try it out, or at least see the details of the render, it's attached below.

Dune

Testing a crop render now, but the first thing I noticed is that you have soft shadows on. Not really necessary in a render like this. It will increase render time. Your clouds are all 2D, you knew that? For the cirrus that's normal, but for other clouds 3D is better.
You have a very large atmo quality (64): I'd set it to 24 (or even 16), and shadows from surfaces adds to render time as well. Unless really needed, uncheck this!

EDIT: if you change the clouds to 3D it's becoming very dark I noticed, so you might want to stick to 2D or change the sun.

Chaz

I originally had the cumulus set to 3D, but I guess I must've changed it to 2D last minute and forgot. I believe I was attempting to lower render time (I read that, unless clouds are the focal point, it's not important to make them 3D.) Most of the settings I was honestly winging. I'm still quite new to fractal terrains, let alone the programs to make them, and quite a bit of it went over my head, so I mostly just experimented. I was also somewhat hoping that, if I simply provided enough time, that I could get a high quality render even on this old machine. Suppose I was reaching a bit too far beyond my grasp. I was also trying to balance cloud coverage and darkness, so I messed with the clouds and atmosphere a bit to attempt to have the right lighting. Since I couldn't see if the clouds were even in the frame from the render preview, I almost took them all away, but I was hoping that you could at least notice them in the final render.

Edit: I just paused the main render to let the render preview get passed 'Render quality 80,' and now I see that the clouds are visible, which makes me want to keep them more. Since the render doesn't seem to have finished even making the dots for the water, I'm suspecting that the reflections and dynamics of the ocean are most likely what's causing this to hang most. But I'm certainly no expert, so that's just a guess. Also, as I said before, the preview does drain the CPU and memory like I thought the final render would. During most all of the render preview, it maxed out CPU usage and drained almost 90% of the memory for a significant portion. And yet, still, the final render uses almost no CPU whatsoever, and hardly dips into the memory.

jaf

Yes, water will slow things down.  I rendered out your scene and it took app. 10 minutes (I used 8/64 threads and I'm on a 64 bit system.)  I disabled the water and it took 3.25 minutes.  I then rendered without the water, atmosphere and clouds and it took 1 minute.  So you can see the water and the atmo/clouds take a bunch of the render time.

I realize now that tg3 does not release memory, so my statement about "maybe you were using more memory earlier in the render and it wasn't shown on your screen capture" was wrong. What Dune suggested should help.  I haven't investigated the TG3 layer features -- maybe there is something that could be done that way. 

I remember some of those old Bryce renders that would take me several days, but the advantage Bryce had, you could save and resume renders.
(28aug19) Ryzen 1800x, 970 EVO 1TB NVME-M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 436.15 (27aug19), Win 10 Pro x64, Terragen Frontier 4.4.18 BMark 0:10:18

Chaz

I imagine that my hardware can still drastically be affecting the render time. This old thing barely handles video games from approximately 2004, which anything before is essentially too outdated to be compatible, and anything newer is too labor intensive for the machine. This computer has never been able to handle even moderately intensive programs and whatnot from this time, and only did somewhat decent for games and such when this machine was new. Can't recall how old it is, but it certainly wasn't top-of-the-line in its day, though it usually does pretty well with things that mostly just require the transfer of information and processing. While I know that Terragen doesn't utilise graphics cards much, that's probably this system's weakest point. However, I did check the bus speeds of the memory and processor, and they're set optimally, so that's probably why processing power in general isn't too poor. It's just very outdated at this point. It's like and old ox... Can stand on its own two feet, haul some if it needs to, but can't keep up at all with the newer, younger oxen. I assume that the 'Islands_modded.tgd' just has the changes suggested by Dune. I actually already input those changes in case I decide to just ditch the current render and try those changes out. I have a feeling that it might be hanging, won't ever make progress, despite Terragen being completely functional otherwise. System? Hardware? I honestly think that Windows 7 is a poor build, to begin with (had far more issues with it, in my experience, than even the 'infamous' Vista.) 32-bit also seems to work against me. Might be a cluster**** of various factors. I just really don't want to stop the current render on the of chance that it might work out, should I just give it enough time.


Side Note: I haven't said this yet, and I should've before, but thank you guys for all your help so far. I feel a bit naïve and very green amongst those much more experienced and knowledgeable than me. To be honest, I'm the sort of person who hates knowing any less than someone else, but it's great to have support when such things arise, as they inevitably do.

jaf

I know what you're saying -- I went through a lot of years with a slower than average system.  I suspect you are right on the edge as far as minimum memory needed, but my main concern would be cooling.  Terragen will really push your CPU and raise the temperatures rather quickly.
(28aug19) Ryzen 1800x, 970 EVO 1TB NVME-M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 436.15 (27aug19), Win 10 Pro x64, Terragen Frontier 4.4.18 BMark 0:10:18

Chaz

Quote from: jaf on August 31, 2013, 06:39:09 pm
I know what you're saying -- I went through a lot of years with a slower than average system.  I suspect you are right on the edge as far as minimum memory needed, but my main concern would be cooling.  Terragen will really push your CPU and raise the temperatures rather quickly.

Which is all the more reason why I'm surprised that it's seemingly not at all using the CPU for the big render, with the fan hardly making a sound. When I rendered the preview, using almost consistently upwards of 100% CPU, the fan was a lot louder, but temps seemed to stay consistently within reasonable limits. So, in terms of temperatures, I think it's actually faring quite well.

Dune

You'd better start saving for a new machine then, Chaz. They're not that expensive anymore. It's very frustrating to have machine trouble, or have to wait indefinitely. The newer Windows also takes more, and the growing complexity of TG3 as well. I have an 'old' (what, 5 years maybe) machine with XP running for daily work, but a one year old i7 with Win 7 to render stuff. There's an incredible difference between these 2.

Kevin F

I agree with Dune Chaz. I just rendered your original .tgd and it rendered in 5 min 57 secs, I then made the mods that Dune suggested (left the 2D clouds) and it rendered in 2 min 2 secs! No noticeable  differences really.
This was on a 12 core  i7 with lots of ram.
Here's the pic:

Chaz

I shut the computer down after aborting the render. It had been over seven and a half days since this computer had been turned off, and I let it sit for three or so hours before turning it back on. I then took the 'Islands_modded.tgd' file, then did a render (it was already set for a crop render at 800x450 resolution.) Took me about 3 minutes. I then decided to go back to the original file and made most, if not all, of the suggested changes. Well, the render is going right now, been about 18 minutes, and it seems to be rendering quite decently (to this system's standards, anyhow.) It, so far, looks a whole lot like the top half of the image of the render you posted, Kevin. To be honest, I'm incredibly disappointed. The land looks horrible (except for the little sandy part to the right.) I'm thinking of taking the land out entirely, messing with the water and atmosphere some more, and then doing a picture of the reflection of clouds in the open water. I'm thinking that keeping the machine on so long could've at least in part affected my difficulties with the render. I'm also quite certain that making the changes suggested by Dune make a significant impact in how it's currently performing. I just wish I could more clearly see how the picture would turn out far before I actually go to render it, because - like I said - I'm really disappointed, now. Looks like it's from a... very outdated video game. The clouds are still fairly nice, and the water isn't horrible, but I see that my terrible lack of expertise probably shows heavily in how this turned out. In fact, my first image of just a desolate mountain, having been almost unchanged and unmodified from the initial terrain which Terragen popped out upon opening the program, turned out better looking than this.

Thanks again, everyone. I'll try to give this computer a little more time to rest here and there, and also try not to over-extend my grasp or expectations. I may also go back to experimenting and working more with Terragen Classic. It made some decent landscapes for me, and perhaps I can take some of my experiences with TG3 and try applying them with TGC.

Also note that my current render is at 1280x800, not 800x450. I also turned the FoV 3D back on.

Chaz

I have what I believe is a fairly major update! I did just as I said I would: I took away all the land, made it just open ocean, and focused more on the clouds and water. While this may seem counter-intuitive ('Weren't you just having problems with the clouds and water?') However, I was a little more careful and modest with certain settings, and made sure not to overdo things by essentially setting everything to the highest possible quality or setting (major faux pas on my part.) However, I made a fairly nice sunset seen over open ocean, and did a couple quick render test runs, as well as made more modifications after aborting a few render attempts. There are a few reasons why I aborted. The first time was aesthetic - I didn't like how the sky was turning out from the first frame rendered. The second time, I slightly increased the subdiv cache (450MB instead of 400) and set it to 'Preallocate subdiv cache.' Another faux pas, as it simply gave me an error that it couldn't preallocate and then didn't do anything, despite the timer running. The third time, I changed the thread count (1-8 instead of 1-64), but I disabled 'Preallocate subdiv cache,' and kept the cache at 450MB. Well, I didn't get any warnings regarding memory, but it did the 'Calm of Doom,' as I'm going to call it. Why call it that? Everything get's quiet, virtually no memory is used, and the CPU seemingly goes to sleep with not so much as a little 'F-f-f-f-f' sound of the fan gently rotating at a low RPM. Even though it creates all the dots, it stops there... and never does anything again. Finally, I switched the thread count back to 1-64, and then hit, 'Render Image.'

What happened? It ran smoothly - and still is. I changed a few settings, like switching on almost every Ray Trace option, except for 'Ray trace everything.' I also lowered AA to 3 instead of 4 (hopefully lower the render time slightly, at least), and turned on atmospheric shadows (hoping to get an effect from the sun behind some clouds.) I also made all the clouds 3D volumetric, which looks much nicer. I just had to make the sun shine a little more, and increase the sun's lighting effect on the clouds. Turned out brighter than I expected, even. But it's rendering! The CPU is now maxed at 100% usage, RAM is a littler higher than normal with 66% usage (it's normally around 40%, even without Terragen doing anything.) The image is slowly coming together piece by piece, and so far, it's looking good. I did 90° Field of View, causing a nice fish-eye lens effect, warping the environment around it slightly, and giving a more expansive feeling. The sky is a bold purple, the clouds various shades of pink. I have yet to see the water, but the preview render made it look pretty nice.

I'm thinking that the thread count is pivotal in this whole 'hanging' issue, where the render just goes on and on without seeming to do anything, or use the computer's resources in any way. By increasing the thread count (back to default, actually), the processor gets a workout, and the memory is at least accessed and obviously utilised to some degree. And the picture Terragen is painting right now? Looks like a surreal photograph, so far. Can't wait to see the rest! Maybe I can still get some use out of this machine when it comes to Terragen 3.

Dune

If you're quite new to TG3 don't change the defaults (cloud settings, quality, AA, bucket size, GISD, that sort of thing) too much, until you know what you're doing. Try 'ordinary' setups, nothing extreme. For test renders you can easily decrease the now default quality of 0.5 to 0.4 or even 0.3, just to see what's going on. If you're doing water, try it without transparency (set it to zero, very easy), saves a lot of render time, especially for testing purposes. Some waterscapes don't even need transparency!