Slow Renders Again...

Started by WAS, August 11, 2019, 01:10:54 PM

Previous topic - Next topic

WAS

Could this be a symptom of Terragen being open too long? Memory issues? I did a render over night, MPD 0.6 AA6 and it finished in only 1 hour 15m, but when I started a preveiw render this morning with a new seed, it's crawling, not using much of my CPU at all, and no fans on again. I paused the render and started it and it would go up to about 80% and it went quicker for a moment, and than fell back down? What could do this?

The preview going now is MPD 0.5 AA2, and should be speeding along.

This has been happening a lot lately, and I don't remember this ever being a issue in past versions, in fact, I had issues with 100% load all the time (even when another program is competing) and fans overworked.

Oshyan

Is the seed the only thing you changed? Was it on a single node, or multiple nodes that you changed it?

- Oshyan

WAS

#2
Quote from: Oshyan on August 11, 2019, 01:15:17 PMIs the seed the only thing you changed? Was it on a single node, or multiple nodes that you changed it?

- Oshyan

The seed effects the base crater functions voronoi A shader, which is than multiple at least 6 times; scaled, translated, added and mixed around. Could be these seed just causing issues


To note though, on my inter-reflective issue, this was why things were slow, not much usage of the CPU. I'm wondering if it's something to do with cache maybe and all the preview renders/full renders piling up does something? After this preview is done I'm going to restart and try the same preview render and see what's different in timing.

It was same scenario here as inter-reflective issue. TG open for 24/hr+ with lots of previews/full renders.

Oshyan

Definitely let us know if a restart helps on the exact same seed. I can also see it being possible given how much the seed affects in your scene that you just hit on a somehow more demanding output of the network.

- Oshyan

WAS

#4
So, I'm not sure. The render times seem the same (I mean not exactly). Seems to be a one polygon difference between the two renders. But about the same.

They're just slow for what's going on, but what confuses me is it's not like TG is working 100% CPU to deal with the heavy node network, it's like instead it slows down and does less work.

CPU usage still fluctuated from 35% to a maximum of 83% (never saw it go higher, took screenshots of the highest numbers I saw). I used to be able to get consistent 80-100% usage, where it would  be dipping down because of background tasks or what not taking some priority away. But not this low.

To sum up, I'm just used to Terragen being heavy on my machine, fans roaring, using PC as a heater for the RV, and lately it's very easy on my machine. Fans never going to high RPMS, machine quiet.

Oshyan

There are definitely specific, generally unusual (as in "not common", but not "wrong") node setups or scenes that do seem to underutilize system resources for some reason. I've seen this on rare occasions. Some kind of inefficiency, maybe a threading/contention/lock issue or something. This may be one of those. I don't know if it helps Matt diagnose them, but it would be interesting to see if the scene does the same thing on different hardware.

- Oshyan

WAS

Quote from: Oshyan on August 11, 2019, 05:34:05 PMThere are definitely specific, generally unusual (as in "not common", but not "wrong") node setups or scenes that do seem to underutilize system resources for some reason. I've seen this on rare occasions. Some kind of inefficiency, maybe a threading/contention/lock issue or something. This may be one of those. I don't know if it helps Matt diagnose them, but it would be interesting to see if the scene does the same thing on different hardware.

- Oshyan

That could very well be the issue. See, I stripped out the distribution map for the craters, and the ejecta map, both used for shading, because it was even slower, but also not really overworking the machine. This version only outputs displacement, and I used that displacement to warp the colour shaders. Besides the function based displacement there is only the colour shaders and a base soft perlin for "moonish" terrain.

I can send it over.

Oshyan


Dune

I do think you have a tendency to use a good many nodes. I know it's often needed for certain effects, but I always strive to minimize, at least for areas that won't really benefit from it. To keep TG as fast as possible.
One example; in your railway file I rendered, there was a lot going on in the ground layer. I think I deleted about 50 nodes, exchanged for a few others and there was no real difference, at least not in the renders like they were. Up close on precise scrutiny there may, though. I'm talking speed versus quality here.

But of course you know all that.

WAS

Quote from: Dune on August 12, 2019, 02:21:52 AMI do think you have a tendency to use a good many nodes. I know it's often needed for certain effects, but I always strive to minimize, at least for areas that won't really benefit from it. To keep TG as fast as possible.
One example; in your railway file I rendered, there was a lot going on in the ground layer. I think I deleted about 50 nodes, exchanged for a few others and there was no real difference, at least not in the renders like they were. Up close on precise scrutiny there may, though. I'm talking speed versus quality here.

But of course you know all that.

I noticed the difference :P My scenes may be heavy there but I think it pays off. For example, no offense, but a lot of your surfaces are smooth and muddy, lacking microdisplacement and low scale color variations ontop of large. I always notice it, up close scenes or far. That's just me though. Think of that realism article and chaos.

It may look like a lot in the crater scene for surface but everything for the craters is needed to...be craters, and shading is just (not open now) really just couple or one shader(s) warped by the craters.

A similar scene that uses warping by terrain is very fast by comparison, as it is based pretty much only on reds.

This is likely a function issue.

Tangled-Universe

How many compute normal nodes do you use? That would also include the compute terrain node of course.
The compute normal/terrain node parses the network 3 times upwards to build its normal value.
If you have a slow/heavy node setup upwards of the compute normal/terrain then this is already slow, let alone if you build this displacement shader using multiple compute steps. Adding another compute normal/terrain further up/down the network will increase the normal evaluations another 3x, so 9x in total. Etc. etc.

This is exactly what Dune is talking about and is why he consistently tries to avoid using any calculations of normals, unless really needed.

WAS

#11
Quote from: Tangled-Universe on August 12, 2019, 10:34:34 AMHow many compute normal nodes do you use? That would also include the compute terrain node of course.
The compute normal/terrain node parses the network 3 times upwards to build its normal value.
If you have a slow/heavy node setup upwards of the compute normal/terrain then this is already slow, let alone if you build this displacement shader using multiple compute steps. Adding another compute normal/terrain further up/down the network will increase the normal evaluations another 3x, so 9x in total. Etc. etc.

This is exactly what Dune is talking about and is why he consistently tries to avoid using any calculations of normals, unless really needed.

None besides compute terrain. Which may not even be needed being orbital. I seem to have to use a Texture Coordinates anyway to use Y for northern and southern hemisphere desaturations I noticed in the second planet working on (which is based on the crater one without craters yet) to get exo with planetary detail.

WAS

Does anyone use Malwarebytes here? 

So last night I removed Malwarebytes cause the trial was over for awhile and it was pretty much useless over other free options.

Guess what happened when removing the expired Malwarebytes?

Yep, Terragen can grab all it can from the CPU now, and fans at max, and no more weird lockup on each bucket update. 

It seems Malwarebytes either sabotages PC performance when out of trial and in free mode, or something but PC is running like normal with it uninstalled. Rig is warm this morning, fans in high, render much further along. Weird. 

I couldn't find anything on slowdown but some weird "Proc Error" but I wasn't getting the error, just the slowness they spoke of.

Oshyan

:o Well that's good to know/hear! So glad you found the culprit. But shame on Malwarebytes! I have generally liked their product and used to use it regularly when doing tech support and virus removal stuff for friends and neighbors. I know antivirus apps can sometimes cause issues with slowing down frequent disk access due to actively scanning each I/O request, essentially, but I don't know why it would have such a big impact on performance for pure rendering. Odd indeed.

- Oshyan

WAS

Quote from: Oshyan on August 19, 2019, 03:17:48 PM:o Well that's good to know/hear! So glad you found the culprit. But shame on Malwarebytes! I have generally liked their product and used to use it regularly when doing tech support and virus removal stuff for friends and neighbors. I know antivirus apps can sometimes cause issues with slowing down frequent disk access due to actively scanning each I/O request, essentially, but I don't know why it would have such a big impact on performance for pure rendering. Odd indeed.

- Oshyan

Yeah it was strange. I did end up emailing them, and I let them know they can try Terragen Freeware themselves to see if there is some confliction or if it's just me. I won't put things past this CPU being simply weird though, the A-Series was a good idea, but imo sacrificing cores (from the FX line) for a GPU (IGP) wasn't the best compromise. I've seen weird stuff in the past I can't explain or ever happen on other CPUs. Some games I Just can't play cause the EXE is incompatible with the CPU. Go figure for a gaming CPU... lol