Microsoft hates me/render times

[leeuwen12]

Dear Terragenners,

I haven't posted here yet, but I was actively stalking. My name is Christiaan and in the facebook galleries I've already posted a couple of minor things.

I was working on a scene with a lot of details and I wanted to create a spherical render to post on facebook. These renders should be 5000x2500 pixels for facebook to convert them into 360 images.

I started rendering and when it already rendered for 170 hours, (Yes, you've read that correctly, and this was only an image cropped to 1/10 of the final image.) Microsoft wanted to update my machine and force restarted the bastard. That being said here are a few of the specs I used:

Render settings
detail: 0.8
aa: 5
pixels: 5000x2500

The rest of the settings was pretty standard, but with defer atmo/cloud on.

I did notice that at the later hours during the rendering problem my CPU was only used for like 30% and not much ram was used either?


NOW THE REAL QUESTION

How can one optimize their render for the best quality/rendertime ratio? (maybe specific to 360 images?)

Does intersect underlying extremely effect render times? Does adjust affinity CPU cores in the task master cause errors later on?

Attached is an older preview render of my scene so you can maybe see what's going on.

my PC
OS: Windows 10 Pro
CPU: Intel Core i7-6700K 4.00GHz
RAM: 16GB ddr4
GPU: Gigabyte Nvidia GTX 670

[Dune]

Welcome to the forum, Christiaan/leeuwen12. Do stick around here instead of facebook 
170 hours is an awful lot and can be dimished, no doubt. First of all, the V3 clouds take longer than the V2, depending on the settings. There are quite a few threads regarding this, so it's good to envison what you want to show and what is really needed, and especially with test renders for the terrain, don't include all clouds and such. Same with the water. Transparent, rough water takes much longer than calm water, and for a render such as the one you showed, a water shader is faster (and good enough) without transparency. The combination of rough transparent water and heavy V3 clouds is quite killing.
Render settings; my standard is 0.6 and AA6, and you shouldn't really need going higher than detail 0.8 is my idea.
Oh, yes, and soft shadows take much longer as well, so in this case that would be overkill also. Soft shadows is important with vegetation and more close up views, or where mountain/object shadows fall quite closely, so to speak.
I hope this helps a bit.

[leeuwen12]

Welcome to the forum, Christiaan/leeuwen12. Do stick around here instead of facebook 
170 hours is an awful lot and can be dimished, no doubt. First of all, the V3 clouds take longer than the V2, depending on the settings. There are quite a few threads regarding this, so it's good to envison what you want to show and what is really needed, and especially with test renders for the terrain, don't include all clouds and such. Same with the water. Transparent, rough water takes much longer than calm water, and for a render such as the one you showed, a water shader is faster (and good enough) without transparency. The combination of rough transparent water and heavy V3 clouds is quite killing.
Render settings; my standard is 0.6 and AA6, and you shouldn't really need going higher than detail 0.8 is my idea.
Oh, yes, and soft shadows take much longer as well, so in this case that would be overkill also. Soft shadows is important with vegetation and more close up views, or where mountain/object shadows fall quite closely, so to speak.
I hope this helps a bit.

Thanks for the tips Dune!

I'm actually doing another test render right now and what happening is that my clouds render almost instantly (turned the settings down for this test render), but my terrain takes much, much longer to render.

And I cropped out my water shader so does terragen still calculate the water shader even if it's not visible right now? (I think it probably does, but still checking)

[Dune]

How do you mean, cropped out the water shader? Water will also be calculated where it's under ground, so not visible. There is a method to decrease that computation by using an opacity mask. Use the default shader for that, set color to black and insert a mask into the opacity tab, then lead the output of this default shader to the input of a (no color) surface shader, where you add your water shader as a child. Output of surface shader goes into lake or plane or sphere used as sea/lake, of course.
The mask can be any; painted (mask from photshop or painted shader), or derived from altitude information of the terrain (use a (blue) displacement to scalar and a color adjust with values (exceeding 0 or 1) of black and white to find the range where the water needs to be visible). Best for the latter is if the Y of the terrain fractal is stretched 10-1000 times, especially if your sealevel is higher than 0.
There are samples and files for this available if you search the threads.

[leeuwen12]

I meant I cropped the render view at the render settings.

I'm testing your tips right now.

[Oshyan]

Dune has some good tips, but I think any discussion of performance will lack real understanding without knowing what your system hardware is. For example if you are rendering at high resolution with some elements that use a lot of memory, and you only have 8GB of RAM, you may in fact be ending up using on-disk swap space for virtual memory during rendering. That's the kind of thing that might explain why your CPU use goes down (there are other explanations too).

So let us know what your system specs are, to start. And if you can, share the TGD. Your overall render settings don't sound too bad, but "the devil is in the details" and often a single unknowingly high setting (such as setting Quality to 2 or 3 includes, or using high atmo samples, etc.) can cause render times to go through the roof. Especially when you've got water in the scene that's reflecting *everything* else.

Also, I want to stress dune's point about the water roughness and transparency+reflection contributing to render time. I don't agree that a reflective shader would work in this situation because as far as I can see in your spherical render there is a *lot* of transparency going on. But I do want to note that the roughness of your water is making it look extremely noisy (which may be desirable, but to my eye looks harsh and unnatural) *and* it is going to dramatically increase render time due to massive interrefelection and interrefraction, i.e. it has to calculate a lot of stuff bouncing off of or refracting through other stuff, because of the fine details. Smooth out the water and I'd bet your render time will go down without touching any other setting...

- Oshyan

[Dune]

Oshyan is right, I hadn't looked at your second render, which indeed needs transparency. But often when glancing across a water area, especially againt the sun, transparency could be zero, just depends on the situation, but good to keep in mind.

[leeuwen12]

I used your water tutorials from NWDA, Dune, but I guess I still have to finetune the waves. I rendered this picture in 11 hours (so there's progress), with 0.6 detail and AA at 5 with the first sampling level on 1/64 and pixel noise threshold on 0.1. My hardware settings are at the bottom of my first post. I attached my file with the NWDA tgc stuff removed.

Sidequestion: Is it possible to render with bloom and starburst in spherical images? because now they always create seams?

[Dune]

Holy cow, you use a lot of nodes for such a 'simple' scene. IMO you can get the same result with much less. Some of them are quite 'consuming' as well, like the alpine.
And what I don't understand is that your water roughness, the waves, are zeroed, but you still see waves. You have set transparency to 1500m, which is way too much, you could almost see to the bottom of the ocean.
For this kind of render you wouldn't need any transparency at all. If you use waves and don't want the noisy reflections, it's best to zero reflectivity in the water shader and add a reflective shader under the water shader, set reflectivity smoothness to 0.002 to start with. If still grainy, up it fractionally (0.004--- 0.006----). But if you start see blocks in the water (after rendering a testcrop), you have to raise quality as well. Do that in small steps, as it will increase render time. But that smoothes out the grainy reflections of those snowy mountains.

I wouldn't know why bloom and starburst wouldn't work.

[bobbystahr]

Echo Dune...y know just because you can use tons of nodes doesn't necessarily mean you should. I think you should, and I do, start with one area (I start with the terrain) and build with a vision of where you want the scene to go. Then, here you will get many hints and tips as to how to improve it but with that many nodes few here would know where to start with advice. I know I have nothing I can add at present. And welcome to the forum.

[leeuwen12]

Yes, I actually want to create a simple animation where the camera goes from space, full planetary view and then zooms in on the heightfield I created. But during that animation I want to 'soar' across the earth so you will see a motion blurred version of the rest of the planet. So all those other nodes are to create a surface that is similar to that heightfield.

And for the water, I used your water shader Dune, because I wanted to have some waves in the water with a good surf, but I'm still tweaking those. And I want the water to be pretty transparant, but maybe 1500 is too much, but my whole scene has exaggerated measurements so maybe that's why?

Bloom and starburst caused seams in previous seems, when one side of the picture would be lighter than the other side, but by placing the sun in the middle of the scene I managed to avoid that issue for now. (the added picture has this error)

[Dune]

I see. If you don't mind some higher render times, there's nothing against it, but the less nodes you use the faster they render. But if motion blurred a lot won't be very visible during that soar, so I would really be as sparse as possible. Also with the surf, will you see it very clearly, or can it be a simpler setup? That's for you to decide.

Exagerating the terrain sizes is not something I would recommend, but if it's needed, it's needed.

You may have a point about the starburst and bloom. I haven't looked at the render in a device yet, but I can imagine that a seem will be occurring, as it's a post render thing.

[Oshyan]

Starburst and Bloom are applied as post effects, meaning they are applied to the rendered image and not during the rendering process. They do not currently have any specific handling of spherical rendering, to properly continue the effect across the edges of the image to the other side. That's something we should address in the future, certainly. I have filed a bug report for this.

- Oshyan

[leeuwen12]

I know what you mean Dune. I'll have to use the motion blur in a sophisticated way. During the soar I want to soar over the water. So the surf doesn't really need animation, but the water does I think, so the specular highlights change, which give the suggestion of motion.

Yeah, I'm actually revamping it know, because I'm working with populations now and the exaggeration is really backfiring right now. But I still want the water to be fairly transparent, so I've changed it to something 1000 for now.


Thanks for clarifying Oshyan! Is it the same for box rendering?

[Oshyan]

Yes, lens starburst, glows, etc. are keyed off of (triggered by) bright parts of the image. If the bright part is in a different face of a cube map, it won't be detected by the current render face and thus won't render a glow/starburst, etc.

It is not trivial to address this but we'll do so in the future when possible.

- Oshyan