Planetside Software Forums

Support => Terragen Support => Topic started by: Fluss on March 06, 2018, 04:41:50 AM

Title: Does terragen supports NUMA nodes ???
Post by: Fluss on March 06, 2018, 04:41:50 AM
Hi,

I'm currently testing the free version to see if Terragen could be part of my workflow. The only complaint I have is that Terragen is using only one core of my workstation. I tried to override core count with no luck. On High core count systems, Windows is creating processor groups called numa nodes. Does Terragen support it ?

My system :
_win 10 pro x64
_dual xeon 2696V4 total : 44cores/88threads
_64Gb RAM
Title: Re: Does terragen supports NUMA nodes ???
Post by: Matt on March 07, 2018, 06:02:04 AM
At the moment there is an issue on these high core-count systems. We hope to fix this in an update soon. However, you can suggest Terragen to use more threads than it detects automatically, and the OS will distribute those threads without Terragen needing to be aware of the hardware. One way to do this is to enable an override on the Startup panel in Preferences (set the desired number of threads, although it calls them cores). Thats a per-machine (per user) setting. If rendering on a network where you want to control the number of threads for each computer there's a command line option, e.g. "-threads 88". As a last resort if you need to force the number of threads on many computers but can't change the command line, it can be set in the render node on the Advanced tab. There you can set both min and max to the number you want. This is stored in the project (scene file).

Bear in mind that TG might not be as efficient at 88 threads. In some cases it might be slower than at 44. This can depend on the contents of the render.

Matt
Title: Re: Does terragen supports NUMA nodes ???
Post by: Matt on March 07, 2018, 06:10:27 AM
Actually, regardless of what other steps you take, you'll need to increase the max from the default 64 in the render node's Advanced tab. So you can either set both min and max to 88 (if this project will only be rendered on this computer), or set only the max to 88 while leaving the min at 0 (to make the project portable to other computers) and then set the core detection override in Preferences to 88.

Matt
Title: Re: Does terragen supports NUMA nodes ???
Post by: Fluss on March 13, 2018, 01:28:49 PM
If I do that, I got a prompt that too many threads are set coz it's running only on one CPU (so half the threads). We were able to change some BIOS settings on Xeon V3 to allow software that does not support numa nodes to take advantage of all cores, sacrificing some performance. But with Xeon V4, this setting has gone so we're condemned to use only one core unless the software support NUMA nodes... That's quite annoying that most software does not take care of that, especially in CGi industry which relies heavily on computation performances... Looks like only renderers are supporting it ATM which is a shame IMHO.
Title: Re: Does terragen supports NUMA nodes ???
Post by: Fluss on January 23, 2020, 10:30:41 AM
Quote from: Matt on March 07, 2018, 06:02:04 AMAt the moment there is an issue on these high core-count systems. We hope to fix this in an update soon.
Any news on this issue?
Title: Re: Does terragen supports NUMA nodes ???
Post by: Matt on January 23, 2020, 09:59:45 PM
This is on the roadmap for v4.5. I'll start work on it soon.
Title: Re: Does terragen supports NUMA nodes ???
Post by: lemon_mirage on October 01, 2021, 07:01:37 AM
Greetings,

I'm new to Terragen. I have a 64core/256Gb node, and Terragen is using only a maximum of 50% of my cores/threads. Is it an actual limitation on Terragen ?
I tried the startup overide and to put 128 threads in the renderer. Nothing helps.

Best,

Olivier
Title: Re: Does terragen supports NUMA nodes ???
Post by: WAS on October 01, 2021, 12:18:22 PM
Quote from: lemon_mirage on October 01, 2021, 07:01:37 AMGreetings,

I'm new to Terragen. I have a 64core/256Gb node, and Terragen is using only a maximum of 50% of my cores/threads. Is it an actual limitation on Terragen ?
I tried the startup overide and to put 128 threads in the renderer. Nothing helps.

Best,

Olivier
Yes, Terragen is not NUMA aware. Split your scene up between two instances of Terragen (crop render).
Title: Re: Does terragen supports NUMA nodes ???
Post by: crisb on October 01, 2021, 06:12:58 PM
Hello lemon_mirage,

This limitation is something that we're working on. It's not as simple as it sounds though.

cheers,

cris.
Title: Re: Does terragen supports NUMA nodes ???
Post by: WAS on October 01, 2021, 06:50:39 PM
Curiously, maybe you can answer @crisb . Is it possible to have a divided sequence? Like, it only renders every other frame by a defined offset or what not, so you could split the task? That may help as well and be less to set up?