Terragen Render Showing No Progress, CPU Usage Regularly Below 10%

[jo]

Hi,

Can you hear your hard disk working when TG is rendering slowly? My suspicion is that you've used up the available RAM and virtual memory is being used. It does look as if you aren't using all the available memory, but I have my doubts. The "Unknown error occurred in a render thread" message is usually due to running out of memory. The problems you were having with preallocating the subdiv cache also point to being low on memory.

The changes you were making to the thread count wouldn't be having any effect. Even if you change the maximum threads used to some high number it will alway use the number of cores reported on the splash screen or About box. You could make it use more threads by changing the minimum number of threads but I wouldn't recommend that in this case.

I've run TG on older machines than yours and I wouldn't expect it to be as slow as you're describing, unless all the RAM had been used up and virtual memory was being used.

Regards,

Jo

[Chaz]

No, I couldn't hear hardly anything. The computer was completely silent. And I did notice drastic changes to an otherwise identical render just by changing thread count. With the low thread count, absolutely nothing would occur - my 'Calm of Doom.' However, up the thread count, and the computer starts working its butt off, and can eventually get a full render out. I'm really quite sure that the issue is related to thread count (my first render, which I had previously said was successful, and came directly before the render I was having trouble with, had the default 1-64 thread count.) I can't explain why, but when I make the thread count low, the Calm of Doom happens, and when I raise it to default, the render works fine. As far as I can tell, it doesn't dive into the virtual memory, because I didn't even hear the hard drive.

[jo]

Hi Chaz,

The problem is that changing the maximum thread count, unless you change it to lower than the number of cores detected for your computer, isn't going to make any difference. When TG starts up it tries to detect the number of cores on your machine. It shows this number on the splash screen when it starts up, as well as on the About Box.

When you start a render TG looks at the number of cores detected and the maximum number of threads set in the render node. The number of threads it creates for rendering will be the smallest of those two numbers. For example, if your machine has 4 cores and maximum threads is set to 8 then 4 render threads will be created. If the maximum threads is set to 64 then once again 4 render threads will be created.

You can set the maximum threads lower than the number of cores, and that will have an effect. Some people do this because they want to stop TG using all their CPU while rendering so they can do other things. For example if your computer had 4 cores and you set maximum threads to 3 then 3 render threads will be created.

So that's how the maximum threads setting works. That's why I'm doubting that your changing that setting was having any effect, it is much more likely to be something else.

How many cores are detected for your computer?

Regards,

Jo

[jaf]

Ah, I just learned something.  I thought with multi-threading a single core could have two threads (or more.)  So in my case, there no use setting threads to more than 8?

[jo]

Hi jaf,

You can use the minimum threads setting to make TG use more threads. For example if you have a single core machine and set the minimum threads to 2 then 2 render threads will be created. I didn't want to mention that because we don't really recommend that you change it unless you have a very specific reason, and I thought it would only could the issue.

By default TG will create one thread per core because that is the most efficient way to do it in most cases. In a technical sense the number of threads you can create isn't limited to the number of cores you have.

Regards,

Jo

[jaf]

Thanks Jo,

I did a little testing and made a table like this:

      800x450

1  8/8/1200   2:52           256x256   Allow Auto Reduction
2  8/64/1200   2:54           256x256   Allow Auto Reduction
3  8/8/400   2:52           256x256   Allow Auto Reduction
4  8/8/800   2:52           256x256   Allow Auto Reduction
5  8/8/800   2:49           128x128   Allow Auto Reduction
6  8/8/800   3:18           64x64   Allow Auto Reduction
7  8/8/800   2:51           512x512   Allow Auto Reduction
8  8/8/800   6:51           512x512   Allow Auto Reduction - No
9  8/8/800   2:50           800x450   Allow Auto Reduction
10 8/8/800   13:04   800x450   Allow Auto Reduction - No
11 8/8/800   3:55          128x128   Allow Auto Reduction
12 8/8/800   2:54          128x128   Allow Auto Reduction (used "Render Image")
13 8/8/800   2:50          128x128   Allow Auto Reduction (used "Render")
14 8/8/800   2:48          128x128   Allow Auto Reduction (used "Render")
15 8/8/800   2:48          128x128   Allow Auto Reduction - No (used "Render Image")
16 8/8/800   2:52          128x128   Allow Auto Reduction - No (used "Render")
17 8/8/800   12:56      800x450   Allow Auto Reduction - No (used "Render")
18 8/8/800   2:49          128x128   Allow Auto Reduction (used "Render")
19 8/8/800   13:00      800x450   Allow Auto Reduction - No (used "Render")
20 8/8/800   2:52          128x128   Allow Auto Reduction (used "Render Image")

I was a bit "alarmed" when after I rendered using the settings from line 10 and went back to line 5 settings (line 11) and it took and extra minute (25% increase) so that's why the "used "Render Image" and used "Render" notes were added.  I was wondering if it made a difference using the render view render button or the render button on the render tab panel.  I couldn't duplicate that, so I don't know -- probably a mistake on my part.

I'm sure the render size, populations, atmosphere, shaders, and all that would really determine the optimal setting, but it is interesting seeing what the different options do. 

[Chaz]

At one point, I did set the minimum to 2 instead of one (this computer has 2 cores), and I think it just froze, so I had to restart the program. However, it didn't cause the 'Calm of Doom.' The only time that I could replicate it was when I lowered the maximum. It could've been just coincidence - I don't know. It's the only specific change that I can note as occurring each time that the low CPU/RAM usage plus no progress did.

[Oshyan]

You sir are a brave soul for sticking with the use of your old hardware! I can't help but respect your tenacity. I've been through my share of struggling with old hardware and being willing to wait hours or even days for even the simplest results, so I can sympathize. I hope new hardware is in your future. In the meantime, let's try to make it a bit more pleasant for you! I think your biggest problem is probably not your CPU so much as your memory and 32 bit OS. If you can remedy those two things, I suspect it will be a lot nicer. Until then...

Simple recommendation: do not change the number of threads used, and since you have only a 2 core setup and low memory, you want to actually *reduce* your cache size, if anything (in general best to leave settings like this alone and *not* think of them as being in any way equivalent to TG Classic settings, there is virtually no relation). I would suggest a cache size of 200, this will still give plenty of memory *per render thread* (which is what matters, i.e. on a machine with 8 or 16 render threads, you *need* a bigger cache to keep all the render threads happy as each thread uses some of that total memory pool). A 200mb cache will leave you more memory for other things, which I would guess is a big part of your problem.

The rest of my suggestion would be echoing Dune, don't bother changing all kinds of *render settings* and cranking settings to high. As you may have already found, very high render settings won't make non-ideal scene setups, texturing, etc. look good, they'll just take a long time to render. It is by far best to spend time with default settings making something that "looks promising but grainy or not detailed enough" and *then* increase *some* render settings by moderate amounts. For an example, the increase of detail from 0.5 to 0.6 will actually be noticeable, even though it's a small change. I would strongly suggest ignoring most other render settings and "raytrace" type options until you know more what they mean as they all, as a general rule, increase render time a lot, and are *only* useful in specific situations, likely having literally zero effect on the actual render output, and simply costing you render time. This is why they're off by default.

An example of a setting to avoid is "receive shadows from surfaces" in the cloud and atmosphere nodes - this only has any effect if the *terrain* is actually casting shadows onto/into the clouds/atmosphere, which does happen of course, but is not as common as you might think, and even where it might be visible, it's questionable in many cases whether the subtle end result effect is worth the added render time. Even if it is, it's a "finishing touch", a nice improvement to a scene you have otherwise brought to a point you're happy with, rather than something to enable early on in the hopes of getting a nicer result. It will improve a good result, but will do little or nothing for an inherently bad one, i.e. it won't make you more satisfied with your scene output by itself.

- Oshyan

[Chaz]

Okay, so I understand that this is probably a dead thread by now, but I really wanted to post this reply. First off, I'll just say that I haven't replied in so long because I've essentially had no internet at all since September. However, that did give me a few months to play around with Terragen a lot more, and I have to say... I'm quite pleased with it, now!

I ended up using Terragen 3 on the laptop rather than my brother's desktop computer because of noise, primarily, but portability (so that I can work on things while on the move, for instance.) However, what I ended up finding out is that, so long as I keep the laptop adequately ventilated, I can render things quite a bit better. I looked at the hardware to see if there was a distinct reason, and at first I was puzzled. Why? Well, this laptop's CPU only had one core! So then I looked at the RAM. Only 2GB! And then I saw the truly differentiating factor - the laptop uses Windows 64-bit, whereas my brother's desktop used Windows 32-bit. The laptop's also a little newer, so I think that the hardware's slightly more powerful, despite being a single-core CPU.

Now, I've been tinkering with the setting, practising and experimenting with all the little options for months, like I said. I've gotten pretty good at knowing what's what, and how to properly adjust the various settings to get the results that I want. I've even made quite a variety of scenes, even ones that I can now say that I'm actually quite proud of! Quite a few even look professionally done, I think (and I don't believe that to be boasting.) I just wish I had the money to get the commercial edition of Terragen 3 so that I could expand my capabilities, or even sell some prints of my renders. I've become that confident in my capabilities with this program (and I'm not the kind of person who ever has much confidence in himself regarding just about anything at all.)

I've found that, when I do raise settings that make a render very memory-intensive, I can just reduce the bucket sizes (128, 64, 32... for instance) while keeping the cache size the same as I would've for the original bucket size (256), and it will render even some of the most memory-exhausting renders, just with a little more time and effort. I've made incredibly complex and intricate renders that I would've never expected this laptop to make just by reducing bucket sizes to 32/32 and setting the memory cache to 800Mb. Having a single processor, this means that the single thread can invest 800Mb into a single rectangular spot that it is rendering at a time, while also being able to devote virtually 100% of the CPU - if I understand all this right. For a bucket size of only 32, this allows me to do even some of the most embellished renders on this very, very outdated system. I've tinkered with bucket sizes and memory caches, and it seems to come out this way every time: complex renders work best when I use small bucket sizes and an 800Mb cache, while simple to moderate renders work best with default, or slightly below default bucket sizes and the same 800MB cache (800Mb has just become my standard, as when I render, I turn off all other unnecessary programs to dedicate nearly the entirety of the RAM and CPU to the render.) With 2GB of RAM, and accounting for background processes that must keep running, I find that I frequently use 100% CPU and about 60-80% RAM for as long as 36 hours for the most complex renders and about a single hour for more simplistic renders.

Also, I think that the issue I was having before had to do with an attempt at using Depth of Field improperly, and having certain settings too high or too low. So, yes, I definitely agree that I was having memory issues and was just far too green to know what exactly was going on. When I turned off DoF and adjusted a few settings regarding memory and bucket sizes, all my problems were remedied! I haven't had that problem ever since. I've learned how to use DoF properly, now, but I don't especially favor it... I suppose I just tend to make images that have no need for DoF, but rather for broad, clear views with sharp focus throughout.

Again, thank you guys for all your help and sorry I just kind of fell out of the conversation. If it weren't for my internet troubles, I wouldn't have waited months to reply! I really appreciate everything you've all done for me, and how kind you've all been.

[Oshyan]

Thanks for the update and glad to hear you've had a lot of success since then! Now that you're back online, you should post some if your images.

- Oshyan

[Chaz]

My internet is really sub-par, so I just made a single image that's an amalgam of all the renders I made and reduced in resolution size to upload easier. Don't mind the watermarks, too. ^^; I get paranoid whenever I make anything I'm even remotely proud of. The final and biggest image I actually made for my little sister. She's nearly a generation younger than me, and I'm not especially good at connecting with her, so I wanted to use it to do something nice for her. Still need to go to Kinkos and get it printed on something nice and framed up. It also happens to be, in my opinion, my best render.