v1.9.88.1 only detecting 2 of 8 cores

Started by Pyronious, April 28, 2008, 07:24:49 PM

Previous topic - Next topic

Pyronious

Thanks for the new multi-threaded release!  I'm curious to understand how many cores this new build is supposed to be able to recognize / utilize.  On my box, which has 2 4-core Xeon processors, the terragen startup window only detects 2 logical cores:

[attachimg=#]

Upon rendering, my process manager shows that indeed 2 cores are being maxed out, while the other cores are just kind of sitting around:

[attachimg=#]

Is this the intended functionality for the current build, or should it be able to detect and use more cores if they are available?

Thanks,

Patrick

Matt

This is a known problem with some multi-processor configurations. You can force a render to use more cores by increasing the "minimum threads" parameter in a render node's "More settings" tab to however many cores you want to use.

Unfortunately in this build memory use can rise considerably with the number of threads used, so there may be some scenes where you will need to avoid using too many threads. I will work on a solution to that.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

Pyronious

#2
Thanks for the tip.  It's using all 8 cores now.  I'll keep an eye on the memory situation -- it's a 64-bit box with 6 gigs of RAM so hopefully it will have enough room to breathe.

Thank you,

Patrick

p.s. Suddenly I feel like this guy:

[attachimg=#]

Pyronious

Interesting.  It didn't seem like I ran out of memory (Task Manager showed 3 gigs available) but about 3/4 of the way through the render the following error occurred and the render stopped:

[attachimg=#]

Any thoughts?

Oshyan

Although your total system memory may be adequate and still have available memory, it's likely that the TG2 process itself is running into the 2GB per application limit of 32 bit programs. Unfortunately this has not changed either in later 64 bit versions of Windows (although I would think it would be possible, given the /3GB switch of XP). In other words the problem here is likely to still be memory, and it is the utilization footprint of the TG2 process itself (virtual memory included) that matters. If that reaches 2GB you're likely to crash. Reducing number of render threads used should help as each additional thread increases memory use.

Also note that utilization of multiple cores has diminishing returns as far as performance increases. After 4-6 cores the gains start to get pretty low given the memory overhead. Hopefully this can all be improved as we go forward.

- Oshyan

Pyronious

Thanks for the rapid response.  I'll do some experimentation to find a stable configuration.

Patrick

nikita

btw: I recently read that the program has to be compiled with a special flag (largeaddressaware)  to make use of the additional gigabyte the /3GB switch provides.
http://support.microsoft.com/?scid=kb%3Ben-us%3B294418&x=13&y=12

Maybe you can use that in future builds?

Moose

Quote from: nikita on April 29, 2008, 06:22:13 AM
btw: I recently read that the program has to be compiled with a special flag (largeaddressaware)  to make use of the additional gigabyte the /3GB switch provides.
http://support.microsoft.com/?scid=kb%3Ben-us%3B294418&x=13&y=12

Maybe you can use that in future builds?

This has my vote. :)

Just to clarify nikita's post... on x64 systems the /LARGEADDRESSAWARE option allows 4GB per application. This works well with ZBrush, for instance, which is still a 32bit app but users of x64bit systems can access 4GB. Please PS, if it's easy enough to do. :)

Oshyan

Very interesting indeed. Since the /3GB swtich is not well supported by MS there isn't a lot of development-side documentation on it. If it's easy to just switch it on it may provide a nice boost for those of us (like myself ;D) with larger amounts of RAM, at least until a 64 bit version is possible. :)

- Oshyan

rcallicotte

This is the same error I was getting last night and this morning.  I was using 6 Min threads last night, but only 1 Min thread this morning.  I was pushing memory around 1.8G.  Maybe this way of taking advantage of the /3GB would be helpful.

Should any of us try to run it without this /3GB switch to see if anything improves for this error?



Quote from: Pyronious on April 28, 2008, 10:18:52 PM
Interesting.  It didn't seem like I ran out of memory (Task Manager showed 3 gigs available) but about 3/4 of the way through the render the following error occurred and the render stopped:

[attachimg=#]

Any thoughts?
So this is Disney World.  Can we live here?

Oshyan

Since we have not enabled large address support it's unlikely that the /3GB switch will help. I'm running on 64 bit Vista which ought to be able to allocate up to 4GB of memory to 32 bit applications which have large address support and TG is still running into the 2GB limit. We are looking at enabling large address support however.

- Oshyan

Moose

Quote from: Oshyan on April 30, 2008, 10:45:23 PMWe are looking at enabling large address support however.

Thanks. I look forwards to what you guys can come up with. Hope it's pretty simple to do... :)

lightning

i noticed this as well i have 2 dual core 3.2 ghz AMD processers which should be 4 cores shouldnt it? but when i start up terragen it says exactly what Pyronious's does 2 logical processors found except with my name of coarse ;D

Oshyan


lightning

Quote from: Oshyan on May 02, 2008, 01:14:30 AM
You've got an AMD "4x4"?

- Oshyan
i have no idea what your talking about ;D

i have 2 processers in my computer and each processer has 2 cores meaning i should have 4 cores
but it says in the start up screen i only have 2 cores
heres a screen shot