Optimising Path Tracing Renders

Started by mhaze, August 12, 2019, 06:23:17 am

Previous topic - Next topic

mhaze

Iv'e been working on a picture that will benefit from a PT render.
But
Standard render no clouds  - 6mins
Standard render with 3 layers of clouds - 40min
Path Traced 9 Hours 6 mins !!!!

Detail .8 AA 8 

Any tips for optimisation?

N-drju

I'm afraid I also don't get the idea of path tracing...

I tried it two days ago on a couple of trees and a cloud layer and the entire image was just one grainy piece of trash.

I've seen other people doing PT renders with fabulous shadow depth and fantastic surface details so it has to be doable. But I have no idea how...
"When dictatorship is a fact, revolution becomes a right."

cyphyr

August 12, 2019, 07:08:43 am #2 Last Edit: August 12, 2019, 07:44:20 am by cyphyr
To be fair the Path Tracing is under than 3 times longer, that's not too bad really! :)

First thing I would do is lower the Detail and AA settings.

Start low and work your way up. (0.5, 3 then 0.7, 5 maybe)

Grain in clouds comes mostly from lower quality settings in the cloud and atmo nodes not so much from the detail or AA settings.

If you're using any merge nodes see if the same results can be obtained through simple Function Add_Colour or Multiply_Colour nodes.

See what happens if you remove the Compute terrain node, depends on the scene how vital it is, and if you do plug any populations into the last node before the planet.
www.richardfraser.co.uk
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 3900X @3.79Ghz, 64Gb (TG4 benchmark 6:20)
i7 5930K @3.5Ghz, 32Gb (TG4 benchmark 13.44)

mhaze

40mins standard compared to 9hrs PT is 13.5 times longer!

cyphyr

Indeed it is
Pre-coffee I read "3 layers of clouds - 40min" ...  as ...  3hours 40min!!
lol
www.richardfraser.co.uk
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 3900X @3.79Ghz, 64Gb (TG4 benchmark 6:20)
i7 5930K @3.5Ghz, 32Gb (TG4 benchmark 13.44)

Tangled-Universe

As far as I know...if you render a clouds v3(!) only scene, so without any surfaces, there's no use in using the path tracer, since cloud v3 has its own GI and shading approach.
Path tracing is awesome if you have vegetation/shadows/low sun angle type of lighting etc.

Oshyan

First and foremost, I would ask: Does your scene need to be path traced? 

Path Tracing is most effective where there is a lot of geometry (e.g. imported object populations), or generally more potential for bounced light. It handles complex interactions of light bouncing better than the normal renderer. But for example in a mostly-terrain scene with a few scattered objects and a lot of direct sunlight, you're probably not going to see much benefit. If you move the sun lower in the sky to create lots of shadows, a sunset for example, then you might see more benefit. But the greatest gains will tend to be in thicker vegetation, or on more complex objects.

Path Tracing also doesn't really affect the atmosphere, its main effects are on the ground, the terrain and anything sitting on it. So don't enable it if you are hoping for some improvement in the sky. It can, however, create more realistic lighting on the ground from the sky, to some degree.

So, if you decide your scene would benefit from Path Tracing, then the render settings do often need to be different than regular renders. We are working on a guide for this, but in the mean time some quick tips (not necessarily in order of importance):

#1: Don't use high MPD

Path Tracing enables "Defer All" shading. Defer All is a rendering method which makes much more detailed terrain (similar to "Defer Atmo" which makes smoother clouds). This means MPD values can be lower for equivalent detail. You would be fine with 0.5. 

You can test how Defer All compares to regular rendering on its own, without Path Tracing. Just enable Defer All separately (without Path Tracing), and maybe render a crop of terrain and test the render time. You can use MPD 0.8 for the non-Defer All render and MPD 0.5 for the Defer All. See which is better and what the render times are.

#2: If you have Soft Shadows Enabled, Reduce Samples

Again, because of Defer All being enabled with Path Tracing, you get higher quality surface shading. This includes Soft Shadows. So you can use fewer samples. I recommend a value of 2 when Defer All or Path Tracing are enabled. Again you can test this with a crop.

#3: Use Robust Adaptive Sampling (especially in v4.4)

The new Robust Adaptive Sampling helps a lot to reduce render time with Path Tracing, while maintaining quality. You can use a value of AA8 or AA6 with Robust Adaptive and 1/64th samples as a starting point. Increasing the amount of adaptivity, i.e. the "First sampling level", will reduce render time but can add to noise. If you use AA8 you can consider 1/256, but at AA6 I would stick to 1/64.

Then adjust the Pixel Noise Threshold *only* to get the level of noise under control. Start with the default value and go down if you want less noise, or up if noise looks fine but render time is too long. You may find you can get away with a lower Pixel Noise Threshold. Do renders in crops to reduce time needed for checking the best values, and you should include an area of shadow in your crop for most accurate results.

Note that Robust Adaptive is improved in v4.4 vs. 4.3.

#4: Don't Bother with High GI Values

The GI Cache Detail and Sample Quality values do have an effect with path tracing, but they only control GI contributions from the atmosphere into the path tracer, which is used for the terrain illumination. I.e. the Path Tracer essentially replaces the cache-based GI system when rendering the terrain. So these GI settings can generally be fairly low (e.g. 2/2 or 3/3) because the atmosphere contribution is fairly diffuse in most cases, and the terrain lighting will be made accurate by the path tracing itself. 

Put simply: you don't need high Cache Detail or Sample Quality values to get good path tracing results on your terrain. 

---

That should get you a good ways toward faster path traced renders. We'll put out more info in the near future.

- Oshyan

N-drju

Thank you for this coherent explanation Oshyan. That should probably solve all the issues.
"When dictatorship is a fact, revolution becomes a right."

mhaze

Thanks Oshyan, brilliant explanation. The main reason I tried PT was for the terrain but I can see now ways in which I can improve it. BTW what is MPD?

Oshyan

MPD is Micro-Poly Detail or just "Detail", as you called it. It only affects elements of the scene that are rendered with the Micro-polygon Renderer, as opposed to being Deferred or Ray Traced. The "Micro-poly" part of the name is important as it does not control e.g. the detail of the sky when Defer Atmo or Defer All are enabled (and thus is not an overall "detail" control). It's a very understandable shorthand to use though ("Detail") because it's the most visible non-AA detail control, and up until Defer Atmo became common place, it did act largely as an overall detail control.

- Oshyan

WAS

Quote from: Oshyan on August 12, 2019, 12:41:35 pmFirst and foremost, I would ask: Does your scene need to be path traced?

Path Tracing is most effective where there is a lot of geometry (e.g. imported object populations), or generally more potential for bounced light. It handles complex interactions of light bouncing better than the normal renderer. But for example in a mostly-terrain scene with a few scattered objects and a lot of direct sunlight, you're probably not going to see much benefit. If you move the sun lower in the sky to create lots of shadows, a sunset for example, then you might see more benefit. But the greatest gains will tend to be in thicker vegetation, or on more complex objects.

Path Tracing also doesn't really affect the atmosphere, its main effects are on the ground, the terrain and anything sitting on it. So don't enable it if you are hoping for some improvement in the sky. It can, however, create more realistic lighting on the ground from the sky, to some degree.

So, if you decide your scene would benefit from Path Tracing, then the render settings do often need to be different than regular renders. We are working on a guide for this, but in the mean time some quick tips (not necessarily in order of importance):

#1: Don't use high MPD

Path Tracing enables "Defer All" shading. Defer All is a rendering method which makes much more detailed terrain (similar to "Defer Atmo" which makes smoother clouds). This means MPD values can be lower for equivalent detail. You would be fine with 0.5.

You can test how Defer All compares to regular rendering on its own, without Path Tracing. Just enable Defer All separately (without Path Tracing), and maybe render a crop of terrain and test the render time. You can use MPD 0.8 for the non-Defer All render and MPD 0.5 for the Defer All. See which is better and what the render times are.

#2: If you have Soft Shadows Enabled, Reduce Samples

Again, because of Defer All being enabled with Path Tracing, you get higher quality surface shading. This includes Soft Shadows. So you can use fewer samples. I recommend a value of 2 when Defer All or Path Tracing are enabled. Again you can test this with a crop.

#3: Use Robust Adaptive Sampling (especially in v4.4)

The new Robust Adaptive Sampling helps a lot to reduce render time with Path Tracing, while maintaining quality. You can use a value of AA8 or AA6 with Robust Adaptive and 1/64th samples as a starting point. Increasing the amount of adaptivity, i.e. the "First sampling level", will reduce render time but can add to noise. If you use AA8 you can consider 1/256, but at AA6 I would stick to 1/64.

Then adjust the Pixel Noise Threshold *only* to get the level of noise under control. Start with the default value and go down if you want less noise, or up if noise looks fine but render time is too long. You may find you can get away with a lower Pixel Noise Threshold. Do renders in crops to reduce time needed for checking the best values, and you should include an area of shadow in your crop for most accurate results.

Note that Robust Adaptive is improved in v4.4 vs. 4.3.

#4: Don't Bother with High GI Values

The GI Cache Detail and Sample Quality values do have an effect with path tracing, but they only control GI contributions from the atmosphere into the path tracer, which is used for the terrain illumination. I.e. the Path Tracer essentially replaces the cache-based GI system when rendering the terrain. So these GI settings can generally be fairly low (e.g. 2/2 or 3/3) because the atmosphere contribution is fairly diffuse in most cases, and the terrain lighting will be made accurate by the path tracing itself.

Put simply: you don't need high Cache Detail or Sample Quality values to get good path tracing results on your terrain.

---

That should get you a good ways toward faster path traced renders. We'll put out more info in the near future.

- Oshyan

This could use some cleaning up (make it more official), maybe more elaboration in some areas, but seriously keep this post on hand for docus in the future.

+Kudos. Quality content.
Check out my Terragen Discord: https://discord.gg/Vy5FRTE

Oshyan

Thanks! Like I said we're writing a guide, so this level of content will definitely be included. May indeed make sense to use this as a starting point for some sections.

- Oshyan

WAS

August 12, 2019, 06:31:44 pm #12 Last Edit: August 12, 2019, 06:33:34 pm by WASasquatch
Quote from: Oshyan on August 12, 2019, 04:21:39 pmThanks! Like I said we're writing a guide, so this level of content will definitely be included. May indeed make sense to use this as a starting point for some sections.

- Oshyan

Definitely, if anything something to refer to. It was very fluid, and understandable. That may be just dependent on my comprehension of TG, but it seemed laid out very nicely for a new user (especially when the mentioned settings and sections of TG are linked to pages on them them-self).
Check out my Terragen Discord: https://discord.gg/Vy5FRTE

cyphyr

Definitely one for the "Official Documentation".
Great work
www.richardfraser.co.uk
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 3900X @3.79Ghz, 64Gb (TG4 benchmark 6:20)
i7 5930K @3.5Ghz, 32Gb (TG4 benchmark 13.44)

Dune

Thanks Oshyan. Very good for (especially new) users to have it this concise and clear in one paragraph.