I'm having issue while rendering a complex scene with lot of vegetation.
I'm working on this project with a lot of speedtree models and populations.
I've got about 30 pops and 50 differents objets but all of them are using resonable texture resolution in only 1K (1024x1024).
My setup for this scene are pathtracer with 25 parameter value for landscapes and 5 ray depth, G.I in standard mode and antialias at 4. Detail in crop region.
When I'm making a low resolution render at 1200 per 600 pixels, then time scale is about 3 hours of computation but now, when I'm making my crops for the final picture (4000x2000) this seems to take eternity, very frustrating.
My scene is a representation of Shuksan mount shoot from picture lake in Washington State, so, I've got a peacefull lake with reflectivity surround by vegetation. There's also a big cloud behind the mount but it seems ok to render even in wide resolution.
Infact, the troubles are visible when come the time to calculate the vegetation.
My workstation is based on X299 chipset with a 16 cores 7820x processor @ 3.6 ghz, 80 Gb DDR4 memory and NVme hard drives.
Is there any tips to solve the problem?
Thanks in advance for your replyFirst shoot10.jpg
Welcome to the forums!
I can imagine you're not keen on distributing your work here freely, but it would help if I could have a look at your scene.
Your scene is quite lovely actually, it has a lot of potential!
I'll send you a PM to see how I can help you.
Cheers,
Martin
Hi there,
you have a lot going on in your scene, it's nice. Here are a few suggestion:
-detail 0.5
-ray detail multiplier (advance tab, subdiv settings 0.35)
-Anti Aliasing 8
-Max number of Path 16
-Max ray depth 2
In the atmosphere and cloud quality tab, check that "receive shadows from surfaces" is off
This is what I would try for a final render of this scene. For sure the ray depth doesn't need to be at 5, and you will probably want more AA to sample all the tiny details in the trees. With more AA you can reduce the number of paths as well. My guess is most of the time may be spent to produce a clean reflection of the cloud in the water.
You can render out the element "sample rate" in a render Layer an upload it here to help investigate where the renderer spend most of its time sampling!
Those are great suggestions, perhaps I should have started there as well.
I have a slightly different opinion about the ray detail multiplier, but all other suggestions are spot on.
The last paragraph is a bit 'hardcore user' advice and also spot on, may be I was a bit too careful with C-ramgfx! :P
Haha yeah sorry, I got a bit ahead of myself there.
About the Ray Detail Multiplier, it is mostly for the stones underwater in the foreground. They might be fine with a lower value though, I guess this calls for some experiments!
Yeah personally I prefer higher values, since base detail is 0.5, then 0.35 for ray detail multiplier would result in a poor'ish 0.165 for the stones underneath the water.
Personally I'd choose 0.7 or higher.
For those who scratch their heads now: the ray detail multiplier is a setting which multiplies the render detail by the set value, see calculation above for an example.
This setting affects rays from outside the camera frustum as well as rays refracted by the water.
By default the setting is 0.25 which means that with render detail 0.5, the render detail for the stones under water will be 0.25 x 0.5 = 0.125, which will result in somewhat facet-looking stones.
Shadows/reflections from elements outside the camera frustum will also be calculated at this ray detail level.
It is indeed a lovely-looking scene. With that many populations and a big v3 cloud, a somewhat long render time may be hard to avoid while also achieving a good level of quality. But there are often certain overlooked or misunderstood settings that can help bring down render time in complex scenes. If you'd rather not share your scene (TGD) publicly here, we are happy to look at it privately. You can email support AT planetside.co.uk. Just looking at the .TGD would probably allow us to give you some idea if your settings have a lot of room for optimization, but being able to test with the whole scene (for which you would do Export Gathered Project, compress, and send to us) would let us give better advice if that's possible.
- Oshyan
I just want to chime in that pops themselves shouldn't slow the render. I've never encountered objects rendering slower than TG assets. They usually render veeery fast. A whole 16k model (obj itself a couple gb) can render out in 22 seconds on my Ryzen 5 2600, much faster than even the default scene. If you are running out of memory though you are caching and calling large files from temp. That will be slow.
Quote from: Tangled-Universe on January 22, 2020, 11:43:14 AMWelcome to the forums!
I can imagine you're not keen on distributing your work here freely, but it would help if I could have a look at your scene.
Your scene is quite lovely actually, it has a lot of potential!
I'll send you a PM to see how I can help you.
Cheers,
Martin
Once again, thanks Martin.
Quote from: KyL on January 22, 2020, 11:55:31 AMHi there,
you have a lot going on in your scene, it's nice. Here are a few suggestion:
-detail 0.5
-ray detail multiplier (advance tab, subdiv settings 0.35)
-Anti Aliasing 8
-Max number of Path 16
-Max ray depth 2
In the atmosphere and cloud quality tab, check that "receive shadows from surfaces" is off
This is what I would try for a final render of this scene. For sure the ray depth doesn't need to be at 5, and you will probably want more AA to sample all the tiny details in the trees. With more AA you can reduce the number of paths as well. My guess is most of the time may be spent to produce a clean reflection of the cloud in the water.
You can render out the element "sample rate" in a render Layer an upload it here to help investigate where the renderer spend most of its time sampling!
Here are my settings : Detail : 0.8 / Ray detail multiplier : 1 / Anti alias : 4 / Max number of path : 25 / Ray depth : 5.
Now I need to try with your value if it can decrease that render time.
Quote from: Oshyan on January 22, 2020, 01:38:42 PMIt is indeed a lovely-looking scene. With that many populations and a big v3 cloud, a somewhat long render time may be hard to avoid while also achieving a good level of quality. But there are often certain overlooked or misunderstood settings that can help bring down render time in complex scenes. If you'd rather not share your scene (TGD) publicly here, we are happy to look at it privately. You can email support AT planetside.co.uk. Just looking at the .TGD would probably allow us to give you some idea if your settings have a lot of room for optimization, but being able to test with the whole scene (for which you would do Export Gathered Project, compress, and send to us) would let us give better advice if that's possible.
- Oshyan
As I've told to Martin, the entire scene with all the objects is about 20Gb. I've try to compress all the files but unfortunatly, the size is 15 Gb which is un-uploadable.
Quote from: WAS on January 22, 2020, 02:29:36 PMI just want to chime in that pops themselves shouldn't slow the render. I've never encountered objects rendering slower than TG assets. They usually render veeery fast. A whole 16k model (obj itself a couple gb) can render out in 22 seconds on my Ryzen 5 2600, much faster than even the default scene. If you are running out of memory though you are caching and calling large files from temp. That will be slow.
I don't think I'm running out of memory. This project is taking 53 to 72% of system memory while rendering, so I'm far from reaching the wall (or am'I also missing something about memory usage here?).
Thanks to everyone for your support, I'll try to solve the problem and I'll then let you know.
Quote from: C-ramgfx on January 22, 2020, 03:07:01 PMThis project is taking 53 to 72% of system memory while rendering, so I'm far from reaching the wall (or am'I also missing something about memory usage here?).
No, it does sound like you're fine there. I don't think the populations are slowing your render. But it is fair to note that as your scene renders, more complex calculations of your displacement etc will add to held RAM towards the end of the render. There is also another small spike I guess with image effects I guess (where a lot of people crash when near max ram).
Your MPD is rather high coupled with RDM at 1, but i can see it looks like you're trying to compensate physical quality for AA quality. This may not give you the best results or time. I would lower MPD and raise AA. Max ray depth may be overkill for that simple cloud setup (looks almost like one cloud layer? Maybe two, but sparse one shouldn't interfere too much with the large one)
Quote from: WAS on January 22, 2020, 03:32:22 PMYour MPD is rather high coupled with RDM at 1, but i can see it looks like you're trying to compensate physical quality for AA quality. This may not give you the best results or time. I would lower MPD and raise AA. Max ray depth may be overkill for that simple cloud setup (looks almost like one cloud layer?)
Agreed, RDM is quite high *especially* with MPD also at a fairly high value (since RDM is a multiplier of MPD). I would reduce MPD to 0.5 and return RDM to 0.25, then do a crop of the foreground water and see if the detail suffers too much. If so, increase RDM to 0.5. Lowering those two settings alone should help a lot in render time (although if, as you say, most of the time is spent on foliage, then perhaps it is still not the main issue).
Also
@WAS I think he's referring to Max Ray Depth for PT, not for Cloud GI. 5 is the default there and I wouldn't recommend reducing it much, if at all.
- Oshyan
Quote from: Oshyan on January 22, 2020, 03:46:28 PMAlso @WAS I think he's referring to Max Ray Depth for PT, not for Cloud GI. 5 is the default there and I wouldn't recommend reducing it much, if at all.
Oh well than that makes perfect sense. Nevermind. Lol
Quote from: KyL on January 22, 2020, 12:44:58 PMAbout the Ray Detail Multiplier, it is mostly for the stones underwater in the foreground. They might be fine with a lower value though, I guess this calls for some experiments!
I actually don't know if this applies to all the detail in infinite distance. For underwater detail it might be good to get it restrained to a certain distance. Like transparency can be masked by distance shader, RDM cannot. Using a distance mask for transparency might though decrease render time anyway, taking away the extra computation of higher RDM. Especially rough distant (transparent) waves can take a lot of time.
That's a good suggestion Ulco (Dune).
I beg to differ on the impact of objects on render time.
I have some high poly fir/pine trees from NWDA and Gumroad, coming in at around 70-80MB each.
Compared to XFrog's 7-8MB fir/pine trees they render much slower.
Personally I'd render the scene without any objects/populations at the suggested render settings in this thread and see how long that takes.
You should have a quick idea then how much your objects/populations are eating.
Maybe only a few models cause havoc, somehow.
Quote from: Dune on January 23, 2020, 02:38:25 AMQuote from: KyL on January 22, 2020, 12:44:58 PMAbout the Ray Detail Multiplier, it is mostly for the stones underwater in the foreground. They might be fine with a lower value though, I guess this calls for some experiments!
I actually don't know if this applies to all the detail in infinite distance. For underwater detail it might be good to get it restrained to a certain distance. Like transparency can be masked by distance shader, RDM cannot. Using a distance mask for transparency might though decrease render time anyway, taking away the extra computation of higher RDM. Especially rough distant (transparent) waves can take a lot of time.
I've seen this help, and also not help. Sometimes because I guess camera angle or w/e the renderer will noticeably render the surface below the water before the planet even though the whole distance to the horizon is masked except the show. This may be like when terrain is rendered behind mountains in certain situations.
Quote from: Tangled-Universe on January 23, 2020, 02:48:39 AMThat's a good suggestion Ulco (Dune).
I beg to differ on the impact of objects on render time.
I have some high poly fir/pine trees from NWDA and Gumroad, coming in at around 70-80MB each.
Compared to XFrog's 7-8MB fir/pine trees they render much slower.
Personally I'd render the scene without any objects/populations at the suggested render settings in this thread and see how long that takes.
You should have a quick idea then how much your objects/populations are eating.
Maybe only a few models cause havoc, somehow.
Could be the object mesh computation too. I have some objects from gumroad that needed to be repaired, and I'm sure past NWDA assets weren't the best pack compared to Xfrog.
All I know is I have a screenshot in my SSS thread of a 2 something GB object with heavy textures that popped out in 22 seconds without SSS, and I always noticed terrain takes ages compared to objects that pop in almost a whole bucket at a time. My RGB population example used tons of objects densely packed and average render took a couple minutes cause flat terrain.
Could simply be because of their size and being in pops you were just caching to disk.
Interesting to read you can have large objects render fast.
I think the biggest hit on render time for populations of vegetation is shooting rays at alphas.
Considering the objects I discussed there are many many more leaf instances and relatively large textures (4k) + high poly base mesh.
Those combined really hit on render time.
Perhaps your model did not have a ton of geometry to be tested against transparency.
Having a flat terrain is definitely not a good reference for render speed discussion.
That's why I suggested to disable populations and render, so that you can dissect the amount of render time spent on populations.
Takes some time, but very conclusive.
Similar to Oshyan's suggestion to explore RDM vs MPD on the water.
Usually optimizing scenes work better by tackling one aspect after the other.
What kind of repairs were you talking about?
Especially NWDA objects made by Walli (Jan Walter Schliep) are high quality well organized assets, so I'm a bit surprised you say they need fixing/repairs.
How do these repairs make them render so much faster? Or do you mean you perform mesh and texture size reduction? I guess not.
Slightly off topic:
No I actually never cache to disk.
I test render with single or a couple populations and then once I'm happy with all of those I do a test-render with all combined.
If I like that I render it final and post it.
After that I barely ever open the project again, so caching isn't a huge thing for me.
In case you meant caching to disk as in because of too low on RAM.
With my new machine I'm not suffering from that issue for sure, but in the past I may have been.
Perhaps a good moment to take a look at those models again and compare render times again.
Quote from: Tangled-Universe on January 23, 2020, 04:46:06 AMHaving a flat terrain is definitely not a good reference for render speed discussion.
That's why I suggested to disable populations and render, so that you can dissect the amount of render time spent on populations.
Oi. It's a fantastic reference if you want to trial objects, and pay any mind to your machines render times. It's a scene with just a sphere and atmosphere (usually disabled for testing anyway). There for you are essentially just rendering heavy objects since we know how fast our blank scene renders with nothing (8s for me at defuallt res). This was more about your specific claim to populations being slow, when I've never had slow objects ever besides things done on them shader wise like SSS. Even on my ancient machines. RAM is the key factor.
Quote from: Tangled-Universe on January 23, 2020, 04:46:06 AMWhat kind of repairs were you talking about?
Especially NWDA objects made by Walli (Jan Walter Schliep) are high quality well organized assets, so I'm a bit surprised you say they need fixing/repairs.
How do these repairs make them render so much faster? Or do you mean you perform mesh and texture size reduction? I guess not.
I was referring to Gumroad trees I had gotten with inverted normals and open seams, and went on to say basically NWDA assets (Walli or otherwise) are not commercial or based on feedback to optimize any issues (like a dead xfrog flower my friend had once and reported).
Quote from: Tangled-Universe on January 23, 2020, 02:48:39 AMThat's a good suggestion Ulco (Dune).
I beg to differ on the impact of objects on render time.
I have some high poly fir/pine trees from NWDA and Gumroad, coming in at around 70-80MB each.
Compared to XFrog's 7-8MB fir/pine trees they render much slower.
Personally I'd render the scene without any objects/populations at the suggested render settings in this thread and see how long that takes.
You should have a quick idea then how much your objects/populations are eating.
Maybe only a few models cause havoc, somehow.
Aye, that's what i've try. Render without object give me a good overview of how I need to modify/apply new parameters.
Yesterday, after some test and advice from KyL and you Martin, I've decide to go like this : Mpd : 0.5 / AA 6 / Max path sample : 16 / Max ray depth (P.T) : 2 / Rdm : 0.7.
The result ? I have render a new shoot at 2400 per 1200 in about 18 hours! which I think is good for now compare to the previous sessions.
I should have mention that there is some speedtrees in this scene with about 5 million polygons (one of them is about 600 mb).
I've also decide to order a new processor and I hope that my choice for a 32 cores i9 9960x will do the trick.
Greetings to all of you for your nice and precious support and celerity.
I'll publish my final render soon.
That definitely seems like a fair improvement considering dimensions. Depending on that speedtree pop though, it could be expanding in memory and caching to disk. That size in a pop is huge. I can imagine a small 1k area of just couple thousand adding up very quickly. It's hard to gauge popsize in RAM during render and things can jump. On my old old Xeon with only 4gb memory, i had to use a spare HDD for my "memory" and because it was always caching to disk, vegetated renders were painfully slow. They worked, and completed with a CTD because of no RAM, but sloow.
Windows too, automatically allocates 1.5 installed ram to virtual system memory, so if you have a lot of RAM, you have lots of virtual disk space as well, and windows will manage it accordingly to keep physical RAM when handling large files.
Quote from: Dune on January 23, 2020, 02:38:25 AMI actually don't know if this applies to all the detail in infinite distance. For underwater detail it might be good to get it restrained to a certain distance. Like transparency can be masked by distance shader, RDM cannot. Using a distance mask for transparency might though decrease render time anyway, taking away the extra computation of higher RDM. Especially rough distant (transparent) waves can take a lot of time.
Matt may correct me, but I'm pretty sure RDM is subject to screen-space (i.e. pixel density relative) Level of Detail (because it's a multiplier of MPD), so in essence the detail of the rays *relative to the scale of things* at a distance is lower. I.e. if you have 100 micropolys per centimeter in the foreground, you might have only 10 per *meter* in the far distance. The goal is for the overall distribution of micropolys to be roughly the same *per pixel* throughout the image. All of which is to say that modulating RDM by distance might not actually be such a good idea. But I could be wrong. :D
- Oshyan