Strange CPU Slow-up

Started by jaf, September 20, 2018, 10:54:36 PM

Previous topic - Next topic

jaf

Ran into a rather strange CPU slow-up tonight on a new scene.  I used my usual setup for the final render of not overriding cores detection and 64 threads.  But when the render started after populating and the final pass, it looked like all sixteen threads were active as normal, but CPU usage was 25% instead of 100%.  I loaded a different scene with the same render settings and it went to 100%.

Rebooted and tried the problem scene again with the same results.  I started disabling things, like populations, and finally found it was the "puffy_low_cumulus.tgc" from the preset pack causing the problem.  I guess there is some negative interaction with TG4, but it really surprised me it could cause my core loading (from 100 to 24%).

Anyone run into this before?

[EDIT]  I noticed a mistake I made.  I still get the loading problem but it's with "large, fluffy cumulus.tgc" not "puffy_low_cumulus.tgc".  Both are in the same reset pack.  With puffy low, I get 90+ loading -- still not expected, but a lot better than large fluffy.

Sorry for the error.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

WAS

Quote from: jaf on September 20, 2018, 10:54:36 PM
Ran into a rather strange CPU slow-up tonight on a new scene.  I used my usual setup for the final render of not overriding cores detection and 64 threads.  But when the render started after populating and the final pass, it looked like all sixteen threads were active as normal, but CPU usage was 25% instead of 100%.  I loaded a different scene with the same render settings and it went to 100%.

Rebooted and tried the problem scene again with the same results.  I started disabling things, like populations, and finally found it was the "puffy_low_cumulus.tgc" from the preset pack causing the problem.  I guess there is some negative interaction with TG4, but it really surprised me it could cause my core loading (from 100 to 24%).

Anyone run into this before?

Do you think you could post the TGC here? You can paste the contents into a [*code] tag. Or is it by chance the settings within the shaders in that clip file? Perhaps some odd cloud settings? Those possible quirky settings will be including in the TGC paste. Might help someone spot an obvious issue.

jaf

#2
It's the tgc "puffy_low_cumulus" here: https://planetside.co.uk/free-downloads/terragen-presets-pack-volume1/
It seems to interact with the scene -- if I use the tgc in a real simple scene (like the default) I don't see the problem.

[EDIT] large, fluffy cumulus.tgc
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

jaf

I'm thinking the problem was how I added the objects -- I had made a tgc of a bunch of populations from a different scene to introduce to this current scene and just made the node hook-ups.  I've done that before and haven't seen a problem, and in this case the objects look fine. But with that particular cloud tgc, I get the problem with the cores at less than 40% when they should be 100%.

I'm going to play around with the scene and see if I can simplify it to something I can upload.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

jaf

Just to clarify a bit, I'm not meaning "slow rendering", though because of the cpu loading, it is slower.  But when I'm set up for 100% loading, it seems that should happen, especially at the beginning of the final render pass (I know towards the end, loading will drop off since there's nothing left to do for some cpu's (or threading.)
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

jaf

OK, figured it out.  That tgc was using a Cloud layer v2 instead of a Cloud layer v3.  Clouds now render at 100% load. Since it came from a v4 Presets Pack that way, I didn't check it close enough.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

WAS

Strange. V2 clouds are much faster than V3. Collision of cloud types?

jaf

Only using one cloud type with an unmodified atmosphere. Not sure what was the problem -- it's like all 16 threads would not process over 34% or even lower while rendering the clouds, but when it got to the terrain it eventually hit 100% (merging through rendering some clouds/terrain.)

So what would make all cpu's idle down to 34%.  All I can think of is some type of thread locking, but it's strange that all would do this.  I guess waiting for some cloud rendering calculation to finish, but throughout the entire (cloud) rendering. Didn't see an indication of one thread above the ~34%.

It seemed to by tied to the complexity of the scene -- terrain and objects.  I could disable all the populations and the cloud rendering would increase a lot (I'll have to try that again and get some more exact percentages.)

I was using a large World Creator terrain, but it renders fine with the v3 cloud layer.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

Oshyan

If you still have the TGD it would be great if you could share it with support. It sounds like it might be worth testing/investigating further on our end.

- Oshyan

jaf

I was able to isolate it to a small tgd (attached.)  At a 1920x1080 render, the cpu loading is around 50% when rendering the cloud.  At 3840x2160 it's 34% on my system.  I'm set to use all cores and 64 threads.  It's only with the v2 Cloud layer and not the bewer Cloud layer v3, so it's not really a big deal.  It was included in the free TG4 preset pack so it was unexpected.

(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

Oshyan

Strange. The GI pass does use fairly low CPU, but once the primary render pass starts it's about 99% on 32 threads on my machine. Were you specifically talking about the GI pass, or does primary rendering also only utilize a small portion of the CPU?

- Oshyan

jaf

#11
Man, I don't know what to think -- the attached scene in my previous post works okay now too, but it sure didn't earlier.  I went back to the original scene where I first noticed the problem and it still has the problem -- renders like this:
[edit] I do see I received a present from MS, a win10 update, but I don't think that has anything to do with it.  I'm going to repeat my testing, slowly removing parts of the problem scene to see if/when the unusual loading goes away, or not.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

Oshyan

Huh! And this is just loading the scene directly into Terragen and immediately starting the render process? You're not switching preview modes, or opening any settings, changing resolution, etc?

- Oshyan

jaf

#13
I thought so, but now I'm not sure.  Last night I got up and saw your response to the file I uploaded and loaded it up and it went to 100%.  I had  tested it several times and was getting the core 34% loading every time.  I went back to the original problem file that had all the objects (populations) and it went back to ~34% when rendering the cloud. 

This morning I checked again with the same results, so I created a new scene with only one internal object population (grass clump) and no terrain.  Positioned the horizon in the middle and rendered.  It showed the low core loading a bit below 50%.  I switched out the v2 cloud layer with the v3 one and the render load went to 100%.  Changed it back and it went to the low loading again. So I planned on uploading the new scene tonight.

As you can probably guess, I loaded the new scene up just to check it one more time, and it was at 100%.  I loaded the original scene and it was 34%.  I loaded the new scene and it was 34%????

So it seems that I have to run that original problem scene first to "prime" TG to have low core loading with the new (or the one I uploaded yesterday). 

Confused yet?  I sure am.  I'll upload the new scene here anyway, but I'm not sure you will see anything unless you had my original problem scene.  The one thing I do know is if I use the v3 cloud layer, it always goes to 100%.




(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

jaf

Okay, I made some progress on this and pretty much know why I was having so many problems isolating what was causing the low core loading.  I'm using two scenes now: the original scene I have called "SlowCloudsComplete" and a test scene called "SkyGrass".

The original scene is too large to upload here. It has 29 populations, a large 8192x8192 World Creator terrain, a stock atmosphere, the "Large, fluffy cumulus.tgc" file from the free TG4 presets package.  It is set to render at 3840x2160 (4K) with all cores and 64 threads. When populated, TG uses 7.1GB.  When rendering, it tops out at 28.7GB.  The render will successfully complete in 11+ hours.  If I change the cloud tgc out (it's a Cloud layer v2) with a Cloud layer v3 with data the same (as much as possible) with the v2, the scene renders in 3+ hours. 

The test scene is the same tgc cloud file and atmosphere and one large population -- the grass clumps.  Pretty simple setup (I uploaded it in one of my posts above.)

For some reason, I could see the same core loading problem in the test scene as I did in the original scene.  I tried it several times and finally uploaded the test scene to see if others saw the same problem.  The next morning I checked this forum and Oshyan said he didn't see the problem.  So I loaded the test scene and sure enough, it rendered normally at 100% until near the end where there isn't much work for all the cpu's.

This morning I tried something different.  I loaded the test scene and it rendered at 100%.  I loaded the original scene, without exiting TG, and the 34% loading in the cloud area of the scene was there.  I again loaded the test scene, again without exiting TG, and the 34% core loading problem was there.  I exited TG and loaded the test scene and it rendered at 100%.

So I'm pretty sure that's why I kept getting different results.  The problem scene seems to do something internally to TG which only reloading TG seems to clear (at least that's my theory for now.)

I'm going to try some rough memory usage tests to see if there's a difference when simply reloading the different scenes and reload TG and then the scenes.
(04Dec20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.51 (04Dec20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02