Planetside Software Forums

Thanks for the new multi-threaded release!  I'm curious to understand how many cores this new build is supposed to be able to recognize / utilize.  On my box, which has 2 4-core Xeon processors, the terragen startup window only detects 2 logical cores:

[attachimg=#]

Upon rendering, my process manager shows that indeed 2 cores are being maxed out, while the other cores are just kind of sitting around:

[attachimg=#]

Is this the intended functionality for the current build, or should it be able to detect and use more cores if they are available?

Thanks,

Patrick

---

This is a known problem with some multi-processor configurations. You can force a render to use more cores by increasing the "minimum threads" parameter in a render node's "More settings" tab to however many cores you want to use.

Unfortunately in this build memory use can rise considerably with the number of threads used, so there may be some scenes where you will need to avoid using too many threads. I will work on a solution to that.

Matt

Thanks for the tip.  It's using all 8 cores now.  I'll keep an eye on the memory situation -- it's a 64-bit box with 6 gigs of RAM so hopefully it will have enough room to breathe.

Thank you,

Patrick

p.s. Suddenly I feel like this guy:

[attachimg=#]

---

Interesting.  It didn't seem like I ran out of memory (Task Manager showed 3 gigs available) but about 3/4 of the way through the render the following error occurred and the render stopped:

[attachimg=#]

Any thoughts?

---

Although your total system memory may be adequate and still have available memory, it's likely that the TG2 process itself is running into the 2GB per application limit of 32 bit programs. Unfortunately this has not changed either in later 64 bit versions of Windows (although I would think it would be possible, given the /3GB switch of XP). In other words the problem here is likely to still be memory, and it is the utilization footprint of the TG2 process itself (virtual memory included) that matters. If that reaches 2GB you're likely to crash. Reducing number of render threads used should help as each additional thread increases memory use.

Also note that utilization of multiple cores has diminishing returns as far as performance increases. After 4-6 cores the gains start to get pretty low given the memory overhead. Hopefully this can all be improved as we go forward.

- Oshyan

Thanks for the rapid response.  I'll do some experimentation to find a stable configuration.

Patrick

btw: I recently read that the program has to be compiled with a special flag (largeaddressaware)  to make use of the additional gigabyte the /3GB switch provides.
http://support.microsoft.com/?scid=kb%3Ben-us%3B294418&x=13&y=12

Maybe you can use that in future builds?

Quote from: nikita on April 29, 2008, 06:22:13 AM
btw: I recently read that the program has to be compiled with a special flag (largeaddressaware)  to make use of the additional gigabyte the /3GB switch provides.
http://support.microsoft.com/?scid=kb%3Ben-us%3B294418&x=13&y=12

Maybe you can use that in future builds?

This has my vote. :)

Just to clarify nikita's post... on x64 systems the /LARGEADDRESSAWARE option allows 4GB per application. This works well with ZBrush, for instance, which is still a 32bit app but users of x64bit systems can access 4GB. Please PS, if it's easy enough to do. :)

Very interesting indeed. Since the /3GB swtich is not well supported by MS there isn't a lot of development-side documentation on it. If it's easy to just switch it on it may provide a nice boost for those of us (like myself ;D) with larger amounts of RAM, at least until a 64 bit version is possible. :)

- Oshyan

This is the same error I was getting last night and this morning.  I was using 6 Min threads last night, but only 1 Min thread this morning.  I was pushing memory around 1.8G.  Maybe this way of taking advantage of the /3GB would be helpful.

Should any of us try to run it without this /3GB switch to see if anything improves for this error?

Quote from: Pyronious on April 28, 2008, 10:18:52 PM
Interesting.  It didn't seem like I ran out of memory (Task Manager showed 3 gigs available) but about 3/4 of the way through the render the following error occurred and the render stopped:

[attachimg=#]

Any thoughts?

Since we have not enabled large address support it's unlikely that the /3GB switch will help. I'm running on 64 bit Vista which ought to be able to allocate up to 4GB of memory to 32 bit applications which have large address support and TG is still running into the 2GB limit. We are looking at enabling large address support however.

- Oshyan

Quote from: Oshyan on April 30, 2008, 10:45:23 PMWe are looking at enabling large address support however.

Thanks. I look forwards to what you guys can come up with. Hope it's pretty simple to do... :)

i noticed this as well i have 2 dual core 3.2 ghz AMD processers which should be 4 cores shouldnt it? but when i start up terragen it says exactly what Pyronious's does 2 logical processors found except with my name of coarse ;D

You've got an AMD "4x4"?

- Oshyan

Quote from: Oshyan on May 02, 2008, 01:14:30 AM
You've got an AMD "4x4"?

- Oshyan

i have no idea what your talking about ;D

i have 2 processers in my computer and each processer has 2 cores meaning i should have 4 cores
but it says in the start up screen i only have 2 cores
heres a screen shot

---

Hi,

I think the problem is that what we use to detect the number of processors on XP perhaps doesn't tell us the correct number of cores, only the number of actual processors. So, for a machine which has 2 processors but 4 cores, it will only tell us that there are two processors available. It will also give different results if your processor(s) use HyperThreading. For Server 2003 up, including Vista, we do it another way which is accurate. There is another way to do it on XP but it is much more complicated.

Regards,

Jo

Hi again,

I should add that if you're using Vista and getting the incorrect number of cores reported it would be very interesting to hear from you.

Regards,

Jo

Quote from: jo on May 02, 2008, 06:45:06 AM
Hi again,

I should add that if you're using Vista and getting the incorrect number of cores reported it would be very interesting to hear from you.

Regards,

Jo

Hi Jo,

I was not experiencing any apparent issues with max cores set to 4, but after reading your post decided out of curiosity to see what times the other settings would yield. They look a off.

Intel quad core
Vista x64

Min   Max   Time   CPU usage
1      1       2:45   25%
1      2       1:34   75%
1      3       1:00   100%
1      4       0:47   100%

Hi Moose,

Quote from: Moose on May 02, 2008, 04:04:14 PM
I was not experiencing any apparent issues with max cores set to 4, but after reading your post decided out of curiosity to see what times the other settings would yield. They look a off.

I think we may have crossed wires here. I'm talking about the number of logical processors reported on the splash screen and in the About box. That's the number TG2 uses to decide the maximum number of threads it will use, if the max threads setting is greater than the number of cores TG2 has detected. I'm not sure what you think I was talking about :-).

In any case, I don't really see a problem with your results. You're getting a good increase in performance each time you allow a new thread, so that's ok. I don't know that the CPU usage numbers are anything to worry about, threads don't necessarily run the whole time on the CPU/core they started on and they can bounce around a bit. It's up to the OS to decide where a thread runs, so your numbers might just be saying that the OS was bouncing the threads around 3 cores when running 2, and all the cores when running 3. I'm not really sure, but I don't think it matters too much.

Regards,

Jo

I imagine some of the optimizations we're doing now might also improve the expected utilization of CPU cores with 2 and 3 threads vs. 4 (you would expect 2 to use 50% for example). The important thing though, as Jo said, is there is still a decent speedup with more and more cores.

- Oshyan

According to MS documentation, the function I'm using should be available on the following:

Windows Vista, Windows XP Professional x64 Edition, or Windows XP SP3.
Windows Server 2008 or Windows Server 2003.

So XP with SP3 should be OK. However we've had reports that TG is not detecting all the cores on some Vista configurations, so I think there might be other reasons why it's not detecting properly.

If you are running any of the above versions of Windows and it's not detecting the correct number of cores, can you please run tgdcli.exe from the TG2 program folder and tell us what the first 5 lines say in the console window? (You can omit license information, of course.)

Matt

I was using some sample code from MSDN to count the number of cores. I'm looking again and it appears that MS have made some changes to the sample code which may help. Or perhaps I did something stupid when I first adapted it. Either way, I think I have an idea what the problem might be.

Matt

In case you still need it, I'm running XP Pro 64:
<<<### APP RUN STARTED ###>>>
Terragen 2 Technology Preview 3 (build 1.9.88.1)
Licensed to CrackedByAppzM4str83
Found 1 logical processors (cores).
AddClass: added class group (268611520 268611616)
//etc.

TG2 detects the number of cores correctly on my notebook. It runs XP Pro (32).

PS: ;)

When I'm booting XP 32 instead of XP 64, TG2 detects both cores.

i run on xp pro 32bit

I'm showing only 1 core in xp64 and was showing 4 cores in xp32.