Planetside Software Forums

General => Terragen Discussion => Topic started by: ajoe on September 08, 2008, 09:22:35 AM

Title: Mac/Win Performance Tests
Post by: ajoe on September 08, 2008, 09:22:35 AM
I have not been very satisfied with render times on my Mac using TG2 Deep, so I decided to do a test and see how performance compared using Windows installed on the same machine:

Mac Pro 2 x 3.2 GHz Quad-Core Intel Xeon, 8GB RAM
Mac OS: 10.5.4
Win OS: Vista 64-bit

I created a single Test.tgd file with only an initial terrain generated. Full render settings as follows:

1024 x 768
All other settings at default values, with Advanced settings as follows:
Min threads: 1
Max threads: 16
Size of subdiv cache in Mb: 2048
Preallocate subdiv cache: checked

Mac render time: 13:57
Win render time: 1:23


In both cases, the render was done immediately after booting the respective OS with no other apps running.

Can anyone explain the huge disparity in these results?

Thanks,

Joe
Title: Re: Mac/Win Performance Tests
Post by: cyphyr on September 08, 2008, 10:26:49 AM
That is a HUGE discrepancy. I would recommend doing the test again (and maybe again after that) so you can get an average. Make sure there's no scheduled tasks running in the background and your not connected to the internet, just because you've rebooted don't mean that the system is not busy doing something in the background.
If anything I would have expected the results to be the other way round but as you on an intel chipset both operating systems (I believe) run natively.

I notice that your Size of subdiv cache is 2048MB, thats also HUGE, is there a reason why you went for such a large number?

There is a site that has benchmarks (a bit dated now) with a default scene here (http://tg2bench.kk3d.de/index.htm), might be worth a look.

Sorry I cant come up with an obvious answer.

Good luck

Richard


Title: Re: Mac/Win Performance Tests
Post by: old_blaggard on September 08, 2008, 10:35:59 AM
It has been noted before that for some reason the Xeon Mac Pros seem to be really nefficient at rendering TG.  I believe the best results were achieved with Max theads set to 2.
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 08, 2008, 11:15:33 AM
Quote from: cyphyr on September 08, 2008, 10:26:49 AM
I notice that your Size of subdiv cache is 2048MB, thats also HUGE, is there a reason why you went for such a large number

Because set at the default, it produces the Warning: "Size of subdiv cache per thread is only 50 Mb (400/8), which could reduce performance...", so I figured to give it some gas. With 8G, giving it 25% of available RAM seemed conservative.

Quote from: cyphyr on September 08, 2008, 10:26:49 AM
There is a site that has benchmarks (a bit dated now) with a default scene here (http://tg2bench.kk3d.de/index.htm), might be worth a look.

Thanks, and I followed your advice. Here are the average results based on three renders for each setting:

Default settings as supplied in tg2bench_v20061227.tgd:

Mac = 2:46s
Win = 0:30s

With Size of subdiv cache in Mb set to 1200:

Mac = 2:47s
Win = 0:30s

With Preallocate subdiv cache checked:

Mac = 2:47s
Win = 0:30s

The interesting thing here is that the Mac renders seem to be in line with most of the data at the benchmark site, it is the Win times that are far superior to anything reported! Very curious.

Thanks again,

Joe
Title: Re: Mac/Win Performance Tests
Post by: rcallicotte on September 08, 2008, 11:17:37 AM
@ajoe - Out of curiosity, are you using the /3Gb argument for your Windows run?
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 08, 2008, 11:24:25 AM
Quote from: old_blaggard on September 08, 2008, 10:35:59 AM
It has been noted before that for some reason the Xeon Mac Pros seem to be really nefficient at rendering TG.  I believe the best results were achieved with Max theads set to 2.

Well, you seem to be correct!

I reduced Max threads to 2 and produced Mac renders (tg2bench_v20061227.tgd) in 1:42s.

So I went back to my original Test.tgd file and set Max threads to 2, and achieved a renders time of 4:05s, versus the 13:57s originally reported.

Still not in line with the incredible Win performance, but definitely better.

Thanks!
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 08, 2008, 11:25:09 AM
Quote from: calico on September 08, 2008, 11:17:37 AM
@ajoe - Out of curiosity, are you using the /3Gb argument for your Windows run?

No. I am not familiar with that.
Title: Re: Mac/Win Performance Tests
Post by: rcallicotte on September 08, 2008, 11:29:44 AM
Since you have 8G of RAM - http://forums.planetside.co.uk/index.php?topic=3878.msg40610#msg40610
Title: Re: Mac/Win Performance Tests
Post by: RArcher on September 08, 2008, 11:43:22 AM
The /3GB switch is not used with 64 bit operating systems.  Vista 64 will automatically use whatever RAM is available up to around 4GB (which is the current limit until TG2 is rewritten into a native 64 bit application).
Title: Re: Mac/Win Performance Tests
Post by: rcallicotte on September 08, 2008, 11:44:58 AM
Ooops.  I didn't see Vista.  Sorry.

But, this might explain the fast Windows times.
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 08, 2008, 11:49:24 AM
Quote from: calico on September 08, 2008, 11:44:58 AM
Ooops.  I didn't see Vista.  Sorry.

But, this might explain the fast Windows times.

Yep, I was doing a little research after reading your post and found this about Vista:

http://msdn.microsoft.com/en-us/library/aa366778.aspx
Title: Re: Mac/Win Performance Tests
Post by: old_blaggard on September 08, 2008, 02:45:23 PM
I really think that it's an inefficiency with the Mac side of things, not a RAM thing - TG2 will take advantage of the full 4 GB available under both Vista 64 and Mac OS X.  This problem also seems limited to the Xeon processors - Macs with Core 2 Duos under the hood seem to be perfectly on par with their Windows counterparts, and even slightly faster.  Perhaps it's a problem with the Mac kernel when running on the Xeon processors, in which case it may be fixed in the next version of OS X.
Title: Re: Mac/Win Performance Tests
Post by: jo on September 08, 2008, 09:29:18 PM
Hi,

This is really interesting. I only have dual core Macs to test on unfortunately. I had no idea things were so bad with the greater number of cores. Did you try it with 4 threads as well?

I did have an inkling things might be slower with what an 8 core Mac. OS X is considerably slower than Windows or Linux for memory allocations and thread synchronisation, and this is causing problems for quite a few developers.

We'll have to look into this during the beta testing period.

Regards,

Jo
Title: Re: Mac/Win Performance Tests
Post by: jo on September 08, 2008, 09:37:13 PM
Hi Joe,

Quote from: ajoe on September 08, 2008, 11:15:33 AM
Because set at the default, it produces the Warning: "Size of subdiv cache per thread is only 50 Mb (400/8), which could reduce performance...", so I figured to give it some gas. With 8G, giving it 25% of available RAM seemed conservative.

With more complex scenes involving more populations and such you may find that subdiv cache size is too great. Although you have 8 GB RAM TG2 is a 32 bit application and it can only use up to 4 GB. There is some overhead in there for the OS, so the maximum may be closer to something like 3.2 GB. By using such a large subdiv cache, especially if it's preallocated, you may run out of memory for other stuff. I'm not sure what would be the ideal size if you were using 8 threads, 800 MB perhaps ( giving 100 MB per thread ), but if you're using 2 threads because of the performance issues the default 400 MB should be fine.

BTW, if you want to stick with 2 threads in the Mac version for time being, you can use the Processor core detection prefs in the Startup preferences panel to set that as an override. See here (http://forums.planetside.co.uk/index.php?topic=2798) for more information.

Regards,

Jo
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 08, 2008, 09:59:15 PM
Quote from: jo on September 08, 2008, 09:29:18 PM
Did you try it with 4 threads as well?

I did some more tests (Mac OS X) and here are the results:

Min threads was always set at 1 and subdiv cache at 1200 with Preallocate off.

Max 1 thread: 2:58
Max 2 threads: 1:41
Max 4 threads: 1:18
Max 6 threads: 2:46
Max 8 threads: 2:47
Max 12 threads: 2:49
Max 16 threads: 2:49

Thanks for the help.

Joe
Title: Re: Mac/Win Performance Tests
Post by: inkydigit on September 09, 2008, 06:54:33 AM
using the benchmark from http://tg2bench.kk3d.de/ (http://tg2bench.kk3d.de/)
when I tested 1.9.98.1 on my iMac -intel core 2 duo - 2.4 ghz - 3 gb Ram
took 2minutes 25 seconds
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 09, 2008, 10:09:56 AM
Now that we have some of this sorted out--at least as much as possible at this stage of development--I would only add that it is a bit frustrating to have all this cpu power and RAM be underutilized while having to wait for renders.

My activity monitor shows that TG2 typically only uses 1/16 of available RAM and about 1/2 of available CPUs, at least with the render settings I have tried to exploit.

If there is a way in future development to give the user the option to have TG2 use the maximum resources available, I would certainly appreciate shorter render times and improved 3d Preview performance.

Just my 2 cents.

Joe
Title: Re: Mac/Win Performance Tests
Post by: RArcher on September 09, 2008, 12:25:43 PM
There are options right now to use the maximum resources available, but not all scenes are going to use the maximum amount of RAM that you have.

In my experience, having large amounts of RAM is only useful when you are using large amounts of objects and populations.  For scenes without populations of objects you will not see much of your RAM being used. 

In terms of your cpu usage, you are directly controlling that by reducing the threads you are using.  By restricting your threads to a maximum of 2 or 4, what you are doing is limiting your computer to using either a max of 2 or 4 of your cores instead of the full 8. 

Right now your best bet seems to go with default settings using Vista 64 which will use all 8 cores.  However if you are determined to use OSX you could try to experiment with running two separate instances of TG2, each one restricting the threads to 4. Theoretically this should max out your cpu usage without running into the current problems you are facing when trying to use all 8 cores within a single instance.

Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 09, 2008, 12:42:57 PM
Quote from: RArcher on September 09, 2008, 12:25:43 PM
There are options right now...

Thanks for the suggestions. I am continuing to experiment. My problem is that I am Mac guy at heart, having used Macs since '85, so that is where I am most comfortable and where I do my primary work.

However, TG2 is not the only app I have that likes the Windows side, so it is looking more and more like I need to get comfortable living in Windows.

Incidentally, although I only have cursory observations at this point, it does appear that the quality of the final renders are superior on Windows. I'm going to do some more tests on that aspect of things and will report back.

Joe
Title: Re: Mac/Win Performance Tests
Post by: jo on September 10, 2008, 08:55:44 PM
Hi Joe,

TG2 is using only half your CPU power because you've set it to use 4 cores. As you've found, right now TG2 doesn't scale well beyond 4 cores and gets slower once you go over that. This is probably largely because of overhead incurred by the OS, relative to Windows and Linux. As I said before, this is effecting a number of Mac developers. Hopefully Apple will resolve some of these issues, but at the moment they're suggesting workarounds which are quite a lot of work which is especially annoying when then Windows version, for example, requires no changes to perform well.

TG2 is currently at alpha level. When we move to beta we will be able to spend more time investigating performance issues and may be able to optimise things so the renderer scales better with more cores.

Part of the reason the 3D Preview is relatively slow is that it isn't multithreaded yet. Originally we did have it multithreaded but it lead to stability issues. I believe at some point we'll look into the changes required to make it stable when threaded, we know what's required but it will take some time. It may not happen prior to the final release.

TG2 only uses as much RAM as it needs. Sometimes that means it wants far more than you have available, sometimes it needs far less. There is no particular benefit to just grabbing all your RAM if it doesn't need it, in fact that may lead to your system as a whole slowing down. At the moment TG2 Mac will take up to about 1/2 of your RAM max, I think you said you have 8 GB and TG2 Mac can only use a maximum of 4 GB. This is the same for every application you use which is 32 bit and not 64 bit. After the final release we will be working on making both the Mac and Windows versions 64 bit and then TG2 Mac will be able to use all your RAM if that's is what is required for your scene.

Regards,

Jo
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 10, 2008, 10:26:05 PM
Jo,

Thanks for the feedback and your participation in this thread. I respect the fact that TG2 is early in the development process and there is much to be done. But, since this is in public preview, I hope that Mac users don't get turned off by the performance. The render qualities are superb but if I was on a lesser machine, I don't know that I could handle the 3d Preview and render performance.

But it sounds like you folks are on top of it.

Thanks,

Joe
Title: Re: Mac/Win Performance Tests
Post by: efflux on September 13, 2008, 11:00:19 AM
I don't have an Intel Mac but I have a G5 dual core. I was a bit disappointed with render times on My Mac with the multi core TG2 compared to my Windows HT PIV machine.

I've moved nearly all of what I do to Linux now. Even Windows apps running under Wine are outperforming from a Windows system.

TG2 runs out the box now on Linux Wine but unfortunately rendering performance is not so good. The MSI installer works and I didn't need to pull any .dlls from Windows.

My Mac is heading for eBay because Linux is way way ahead of both Windows and OSX now. I'd have said Apple were top dog at one point but now I'd go as far as to say they are below Windows (XP that is). OSX is simply too bloated now. I've moved all my audio production off the Mac and Linux's audio performance completely destroys my Mac. This is on a 6 year old PC!

I'd say a TG2 Linux rendering engine is a must.
Title: Re: Mac/Win Performance Tests
Post by: plugsnpixels on September 13, 2008, 03:48:41 PM
efflux, I'm interested in more details about your Linux migration and your comments about OS-X. I'm a Mac user that runs Windows via Parallels as necessary; I've played with Ubuntu Linux in the past and while I like the concept (everything is free and the community seems to really care about its work), I can't see using it full-time, especially after getting used to all the goodies available for OS-X. Perhaps if I were a Windows user looking for an escape...

I'm curious why you don't run Linux on your G5, unless the old PC's processor is somehow faster.

This is not a Mac vs. Windows vs. Linux post; I'm really interested in your experiences and opinions. Perhaps your decision is based around TG2 itself and its performance on Macs, but you also mentioned audio work (Audacity?).
Title: Re: Mac/Win Performance Tests
Post by: ajoe on September 14, 2008, 07:44:49 AM
Quote from: plugsnpixels on September 13, 2008, 03:48:41 PM

This is not a Mac vs. Windows vs. Linux post...

Correct. This is not an OS issue but a software specific issue with Terragen. I've done tests with modo and Vue using this same set up and found the render times to be virtually identical on both Win and Mac.
Title: Re: Mac/Win Performance Tests
Post by: Firechild on March 15, 2012, 05:36:56 AM
I am really interested in more recent figures Mac vs Win performance.
I am building my scenes on a Mac Pro hexacore and I´m quite satisfied with the rendering speed. All 12 cores are working very hard. However the 3D preview is another matter... looking on tutorials on youtube made on windows the 3D preview screen updates WAY faster than on my Mac Pro. Also the 3D preview is not using more than 1 core, correct ? Or is it just on OS X Terragen2 uses only 1 core for 3D preview ?

I have also another 8 core Mac Pro that will be my rendering machine so my question is. Will I get faster rendering times with Terragen 2 installing windows 7 on a bootcamp partition instead of using Snow Leopard?

Edit: I saw in another thread that 3D preview is one core on both platforms.
However, still interesting in rendering times Mac vs Win ( on the same or similar machine of course ).
Title: Re: Mac/Win Performance Tests
Post by: jo on March 15, 2012, 06:31:22 AM
Hi,

With regards to rendering speeds I would say the Windows version is perhaps a smidgen faster than the Mac version but there's not much in it. The upcoming v2.4 release does make some further improvements in some cases. I don't think it's enough difference to install Windows. This is from testing with OS X (10.6) and Windows (64 bit Vista) on the same Mac Pro, using Bootcamp for Windows. If you have a copy of Windows already you could try and see if there's a difference with types of scenes you're rendering but I'm pretty confident the performance will be similar. When I was doing some testing a while back the Mac version was even a bit faster sometimes.

It's interesting you say you've noticed a speed difference in the preview rendering. I'm responsible for the Mac version but I also do a lot of the Windows stuff as well so I'm often booted into Windows on my Mac Pro. I've never noticed any particular difference but I also spend most of my time working with debug builds (when not doing performance tests) which are slower than release and not good for gauging performance. I'll have to check it out.

The 3D Preview uses only 1 core on both Mac and Windows.

Regards,

Jo
Title: Re: Mac/Win Performance Tests
Post by: reck on March 15, 2012, 09:03:42 AM
Quote from: jo on March 15, 2012, 06:31:22 AM


The 3D Preview uses only 1 core on both Mac and Windows.


Is there nothing to add after this bit? Like "... but in 2.5 the preview window will be multi-threaded and take advantage of all cores"  ;D

hehe I know that request is on the list but I guess we won't get that any time soon? Thinking about it wasn't multi-threading added to the preview windows around the same times that the main renderer was multi-threaded but it had to be taken out because of issues?
Title: Re: Mac/Win Performance Tests
Post by: Firechild on March 15, 2012, 12:49:29 PM
@ Jo
Ok, thanks. I don´t have a Windows version installed so I am glad that you say the performance should be similar on both platforms. You saved me 2-3 hours work of not going through a windows installation procedure... :D
Title: Re: Mac/Win Performance Tests
Post by: ajcgi on March 15, 2012, 01:05:59 PM
Quote from: reck on March 15, 2012, 09:03:42 AM
Quote from: jo on March 15, 2012, 06:31:22 AM


The 3D Preview uses only 1 core on both Mac and Windows.


Is there nothing to add after this bit? Like "... but in 2.5 the preview window will be multi-threaded and take advantage of all cores"  ;D

hehe I know that request is on the list but I guess we won't get that any time soon? Thinking about it wasn't multi-threading added to the preview windows around the same times that the main renderer was multi-threaded but it had to be taken out because of issues?

Man even just 2 threads would be nice. got an 8 core thing here that barely ticks over previewing. Just a waste. I end up quick rendering for the simplest of changes.
Title: Re: Mac/Win Performance Tests
Post by: elipsis1 on March 15, 2012, 04:44:02 PM
Quote from: jo on September 10, 2008, 08:55:44 PM
right now TG2 doesn't scale well beyond 4 cores and gets slower once you go over that.

On my Macbook Pro i7 machine, should I limit the cores to 4 then?

Title: Re: Mac/Win Performance Tests
Post by: jo on March 15, 2012, 06:08:17 PM
Hi elipsis1,

Quote from: elipsis1 on March 15, 2012, 04:44:02 PM
Quote from: jo on September 10, 2008, 08:55:44 PM
right now TG2 doesn't scale well beyond 4 cores and gets slower once you go over that.

On my Macbook Pro i7 machine, should I limit the cores to 4 then?

No, definitely not! That was really old information, from 2008, and things have moved on significantly since then. Multithreading performance is massively improved in the Mac version. OS X 10.6 helped a lot too, because 10.5 had some performance issues itself which were slowing TG2 (and other apps) down.

Right now TG2 will happily use all the cores you've got. The rate performance increases with more cores (otherwise known as scaling) isn't perfect but it's pretty good.  On my Mac Pro I leave the thread settings at their defaults and always render with 16 threads.

Regards,

Jo
Title: Re: Mac/Win Performance Tests
Post by: elipsis1 on March 15, 2012, 06:39:09 PM
Wow, I see now why you guys have that red warning about posting to an old thread (120 days or older)!

Thanks for clearing that up! LOL

I need to pay more attention to the times and dates of the original posts!

whew!