CPU usage

[mayaman]

Hi everyone

I've just started experimenting with the trail version of terragen2, but the fun is taken away a bit by the excruciatingly long render times.
I have a workstation with two Xeon quad processors inside, but when I open up the Task manager, I see that the preview makes it work at 12%, and when I start rendering, it goes up to 25%. This made me think that Terragen had a weak renderer that did not take full advantage of multicore processors not to mention dual processors.
But then a friend of mine installed the trail version with one quad core processor, and his processor started working at 100% during rendering.
What's going on here, and what do I have to do to improve my render performance.

[red_planet]

Try looking at advanced settings in the Renderer settings page.

You can set max and min threads here for those occasions when, for whatever reason, TGTP doesn't accurately determine how many processor cores are available.

Hope this helps.

Chris

[reck]

Also I believe that the threaded preview window had to removed for stability reasons. Hopefully this will be back shortly after the first release.

[mayaman]

Thanks, that makes a huge difference. Still jumps up and down very erratically, but performance is much better now.

[dandelO]

You can determine how many processors TG uses in the preferences 'startup' dialogue. Choose 'user overide' and set the number of cores to your preference. It seems to be only recognizing one of your cores by default(your 25% cpu usage when rendering). Some people have problems with this on their systems, I didn't for some reason. I don't know why TG reports the correct number of processors on some machines and doesn't on others.

EDIT: Found this thread... http://forums.planetside.co.uk/index.php?topic=3926.0 
Jo posts a possible answer to why TG doesn't recognize your 'cores' at startup at the top of page 2.

[rcallicotte]

I was fine with XP32 (4 cores) and now only show 1 core in Windows XP 64.

[Matt]

I've uploaded a very simple program which reports CPU layout information on Windows: the relationships between NUMA nodes, cores, logical processors.

http://www.planetside.co.uk/terragen/downloads/cpulayout.exe

The problem is that some non-HT architectures are reported as using HT (i.e. sharing multiple logical processors on the same core). In order to be efficient on HT machines, TG needs to ignore HT logical processors and only look at the number of true cores. Unfortunately there are many machines that have true cores which Windows (or our layout detector) thinks are shared (i.e. HT), and so we don't utilise them fully.

If any coders out there have a solution to this, please help us!

(The way to get around this problem, as a TG user, is to override the detected number of cores in Preferences "Startup" panel, or modify the thread settings in render nodes.)

Matt

[PG]

Well the only way to actually disable hyperthreading is in the bios, otherwise the system would become massively unstable. As far as I'm aware only Vista has the default ability to report whether a core is physical or logical with some of the new additions to WMI. There are hotfixes for WMI in XP that allow this though. http://support.microsoft.com/kb/936235/
This is a code excerpt I found that uses the new properties

Code Select Expand

<rs:data>
<%foreach wmiobject="o" namespace="root\CIMV2" wql="SELECT * FROM Win32_processor" %>
<%set numberofcores = "wmi:o.numberofcores" %>
<%next%>

<%foreach wmiobject="o" namespace="root\CIMV2" wql="SELECT * FROM Win32_ComputerSystem" %>
<z:row
c0 ="<%writexml "%numberofcores%"%>"
c1 ="<%writexml "wmi:o.NumberOfProcessors"%>"
c2 ="<%writexml "wmi:o.NumberOfLogicalProcessors"%>"
/>
<%next%>
</rs:data>


Code Select Expand

<rs:data>
<%foreach wmiobject="o" namespace="root\CIMV2" wql="SELECT * FROM Win32_Processor" %>
<z:row
c1 ="<%writexml "wmi:o.Description"%>"
c2 ="<%writexml "wmi:o.Manufacturer"%>"
c3 ="<%writexml "wmi:o.Name"%>"
c4 ="<%writexml "wmi:o.NumberOfCores"%>"
c5 ="<%writexml "wmi:o.NumberOfLogicalProcessors"%>"
/>
<%next%>
</rs:data>

This program was developed by intel to help developers determine whether a system has logical processors. http://software.intel.com/en-us/articles/detecting-multi-core-processor-topology-in-an-ia-32-platform

[Matt]

I am not sure but I think this might be the same hotfix that allowed XP to count logical processor and number of cores separately, bringing it up to date with Vista. That is what TG uses, and is how TG thinks that the number of cores is less than the number of logical processors. It just happens to be misleading on multi-proc, multi-core computers.

I thought that the functions on that hotfix page were all related to the hotfix I am talking about, but maybe it is different. I scoured a lot of info on the subject, including the Intel work you linked to. Everything I've found seems to be flawed in some way even when it is not acknowledged by the authors. Perhaps the Intel work is the best thing to try next. I would really like to know what other rendering software uses. I just don't know why it's so difficult to a get a definitive answer to this problem.

Matt

[PG]

Well it's as intel themselves say, an inherent problem with hyperthreading is that you have to trick the computer into thinking there is more than the number of physical cores so that instructions are sent properly.
Speaking only in terms of an idea, I've no idea whether it would work in practice but, if the operating system is able to derive information about the processor from the part of the processor that stores the architecture information, and this part of the processor is used in hyperthreading to handle the extra instructions, then perhaps it would be possible to do a query to find out whether an instruction is being sent to that part of the processor or not. Send one piece of data to collect some information about the processor then send another piece of information on the each core in turn to see which core returns that data. This is obviously not something that Windows accomodates for but perhaps if it was done in a low level language (Assembly) then it could be possible.