[Question] Accelerate rendering of high density sky & city lights

Started by Regda, November 26, 2019, 06:39:59 AM

Previous topic - Next topic

Regda

Hi,

i want to create a sky from a city with a very high pollution at night, for that i used a lot of lights (light pollution) and created a very high density in atmosphere.
The test-rendering was what i have expected (see Screenshots) but then i raised the quality to eliminate the grain and the render time explodes...
I have to render 6 images but after 11h (4x 4GHz) the first image wasn't nearly 50% complete...  :'(

I need a very smooth sky so no grain or pixelations and the clouds should be smooth too (so i don't wanna a dense sky with sharp clouds...).
Camera related post processing doesn't work because i have to rotate the cam and render the next image, that's why i disabled GI.
Can somebody help me with accelerating it?

(using gimp and apply an Gaussian blur does not work too, because it will create seams between the images)

CityPollution_11x.tgd

darkcitypollution1.png
darkcitypollution2.png

Oshyan

I am not sure if you arrived at some of these settings based on testing or just assumptions about what might be needed, so I'm not certain whether my suggestions will work for you. But here is what I see that could be adjusted to improve render time and results:

Scene Recommendations

Your atmosphere is very, very dense and it is making your scene look very low contrast as well as blocking a lot of direct and indirect light. This basically somewhat defeats the apparent goal here which is to have all these light sources lighting up the clouds or atmosphere. In your scene as-configured your lights are mostly just making things brighter overall, but your clouds are already colored yellow and visible without them. You can achieve a very similar result if you disable all the lights besides the sun and then just increase brightness/contrast in post processing, or increase exposure for the render, or increase brightness of clouds (or some combination of those). And render time goes down from 18 minutes (after the below optimizations) to about 1 minute when you do this!

Not only that but simulating all those lights is a big part of what is adding noise to your image. When you disable them, the resulting renders are much lower in noise as well as being much faster. Your lights may therefore not really be necessary if the result you are getting is visually what you want to achieve. In other words you've already used a non-realistic method to achieve the result (using dense atmosphere instead of putting the sun below the horizon), so you might want to just focus on getting the visual result you want, rather than making it realistic. It will certainly render faster.

I have attached two images that show how relatively similar these results are, "CityPollution_11x_opt1_18m.jpg" and "CityPollution_11x_opt1-no-lights_1m.jpg". Both are using GI. I disabled the lights for the second one, of course. The only other changes were in post processing to increase exposure/brightness and adjust gamma, then add a bit of saturation to the no-lights version. Obviously the lighting is different, in the no-lights version the foreground is actually much brighter for some reason. But this could be addressed much more simply with other lighting or GI changes if desired. The point is that you can save render time and reduce noise if you avoid use of a lot of light sources - which is not contributing that much to the end result - and instead focus on getting the outcome you want in a non-realistic way (since your setup is already non-realistic), and use post processing to help get to your goal.

Alternatively  if you want a more realistic result with actual light from the light sources illuminating the cloud layers, you'll need to make your atmosphere less dense to see it. In fact, as I mentioned, it seems like you are using high atmosphere haze density partly to simulate night time here. Consider making the atmosphere less dense, turning off the sun, using GI, and maybe reducing brightness of some of your lights. You would probably also want to lower the cloud layers as, realistically speaking, you won't get that much light with layers as high up as these (10-11,000 meters).

But again if you're reasonably satisfied with the results you already have and you just want less noise, then consider getting rid of the light sources and/or see my recommendations in the next section.

Render Setting Recommendations

Update to the Latest Version
Your TGD indicates you're using Terragen 4.1. If you have the ability to update to the latest version (4.4.45), I highly recommend you do. There have been a lot of optimizations and new features added. I can't find a record of your email in our purchases database, and given that you disabled GI I'm guessing you might not have access to GI caching (which would be the preferred solution to GI differences between frames). So that suggests you may be using the free version. If that's the case you can simply download the latest free update here:
https://planetside.co.uk/free-downloads/terragen-4-free-download/

Using 4.4 will give you access to Robust Adaptive Sampling and Defer All rendering options, among other things, both of which will improve your results and reduce render time for equivalent quality. Even without using these two settings you may have faster render times as various optimizations have been done since.

Ray Detail Region Padding
You have this set to 360 with maximum quality, which is adding a lot to render time. Typically this setting will be necessary in a couple of situations. When using reflective or refractive surfaces (e.g. glass, water), it can be helpful to make reflection/refraction of off-screen or screen-edge terrain look more correct. That doesn't apply here. It is also sometimes needed to get correct rendering of shadows cast by off-screen geometry. Given how diffuse the lighting is here, there are no such shadows being cast. So I don't think any detail region padding is actually necessary. But perhaps you tested it without and were running into issue so you enabled it. In that case let us know what specific problems you saw and maybe we can find better, more efficient ways to solve them.

Make Use of Robust Adaptive Sampling
If you are able to update to Terragen 4.4, using Robust Adaptive Sampling with default sampling and adaptivity settings (1/64th samples in most cases) will generally give you faster results for an equivalent level of noise, meaning you can increase quality (increase AA and/or decrease Pixel Noise Threshold) to get better output in the same render time as old versions.

Soft Shadows
You have Soft Shadows enabled and with very high sample settings, but you also have Jitter Samples disabled. Given your very diffuse lighting it's unlikely that Soft Shadows will have much effect here, so they can probably be disabled. Even if you leave them enabled, however, when you disable Jitter Samples you need *less* soft shadow samples than with it enabled. Additionally if you use Defer All shading (which you have disabled), you can avoid use of higher samples, generally 2 soft shadow samples will work fine (with jitter enabled) when Defer All is enabled. Overall results will generally be better for a given render time. You can also reduce Micropoly Detail a bit when Defer All is enabled, so I used a value of 0.5.

Use Global Illumination (if you can)
Due to the thick haze in the atmosphere and heavy clouds, a majority of your lighting here is indirect or scattered (which I think is your goal). That being the case you're likely to have some noticeable effects from Global Illumination. I gather you disabled it to (I think) avoid seams between adjacent camera views/renders. If you have a licensed version, use GI caching, render out a cache for each camera view (you can use relatively low settings like 2/2 for GI), save each one numbered, e.g. "gicache_0001.gic", "gicache_0002.gic), etc., then for final rendering of each image, set it to read 6 GI caches and "Equal blend within range" and specify the file name as "gicache_%04d.gic" (which basically is  a wildcard for "gicache_" with 4 numbers after it). If you don't have a licensed version you may still be able to avoid GI seams by using GI Region Padding. Turning off GI entirely is a last resort. Remember that GI will affect cloud layers too, especially since you are using v2 clouds.

I have attached an image showing without GI, "CityPollution_11x_opt1-no-gi_16m.jpg", which you can compare with the "CityPollution_11x_opt1_18m.jpg" which does include GI. Notice that your yellow lights are having virtually no effect on the atmosphere, which should be colored more yellow as-in the with-GI image. If this is the effect you want then of course it is better to keep GI off though. Also if you do decide to keep GI disabled, keep in mind that your Enviro Light settings will have no effect. Strength in Atmosphere, currently set to 10, won't do anything. If you want something in-between, where there is some coloring of the atmosphere from the lights, then keep GI on and adjust Strength in Atmosphere down. A value of 1 is the default.

Atmosphere Samples
You are already using increased atmosphere samples at 32. A lot of your noise is probably coming from the simulation of light in the atmosphere. So you may need to increase this even further to get rid of noise, but I recommend using Robust Adaptive Sampling if you do increase atmo samples more. If your maximum AA is 6 (due to use of the free version), then increasing atmo samples may be one of your best ways to reduce noise since you can't increase AA further. Otherwise consider using a bit higher AA, along with Robust Adaptive, and keeping atmo samples at 32.

Disable Motion Blur
Last but not least, be sure to disable motion blur, at least if you are going to animate the camera to render out the different views. Otherwise you'll get unwanted blurring between frames.

Post Processing Options
You mention that you can't use Gaussian Blur in GIMP as a fix. I wouldn't recommend any simple blurring method, not just because it creates seams, but also because they generally do a uniform or semi-uniform ("naive") blurring of your scene, including removing many details you wish to keep. If you can you might try testing a good noise reduction filter instead, something that is designed to selectively remove noise rather than just applying a blur to the whole image. There are some free image editors that include adequate or even good noise reduction systems. I'm not sure if GIMP is one of them, but you can look at XnView, or Darktable, for example. Both are free.

I have attached a TGD with the above recommended optimized settings. Render time went down dramatically and results are equal to or better than your old settings. I don't have specific numbers since your old settings took more than 45 mins to get to 15-20% of render completion, and I stopped it before finishing as I knew I could make it faster. :D New render time on my machine is 18 minutes to completion though, so definitely much faster.

I hope the above helps. If you have specific questions about any recommendations, or run into further issues with optimization let us know.

- Oshyan

Regda

Thanks so much for your replay Oshyan, i will need some time to work through it (maybe next weekend :D).
Well yes i am using the free version, so i am a student but i don't know if it counts (related to the educational version) because i don't study something related to art or math.  ???

Oshyan

It doesn't matter what you're studying. As long as you're enrolled in formal education of some kind and will not use Terragen for any commercial gain (i.e. you do not make any money off of your use of Terragen), then you qualify for an educational license. :)
https://planetside.co.uk/educational-license-program/

- Oshyan

Regda

The Scene should be at night, and ... well yes i used the "Sun" as moon (so at least that was the idea...).

Some Light sources are behind hills and i played with "Receive shadows from surfaces" because i want to avoid that a light is visible through a hill.
So this would mean that the hill is casting a shadow too (maybe?), so i set "Ray Detail Region Padding" to 360 for avoiding that the frames will not fit after rendering is complete.

So the Light sources are also meant to create background light pollution and are not really meant to lighten up the clouds, it was only a sideeffect i was fine with. ^^'

Then i should definitly try to get the educational license  ;D

concept.png

Dune

Seeing this, what you could also try is make a low V2 cloud without any density shader attached, say 500m high at 250m, uncheck 'taper top and base'. That's a haze covering a dark city. Then make a vertically stretched PF with size settings adjusted to your 'light needs', and use that as either shadow function, or direct light function, or any of those in the cloud node, and see what you can get out of that in terms of streaks and patches of lighting up cloud. Theory, so you'd have to see if that works. You may also have to manipulate cloud color, or variables in the lighting tab.

Oshyan

The hill will cast a shadow regardless of "Receive shadows from surfaces", as long as as "Cast shadows" is enabled in each light source and it is set to cast shadows from surfaces. "Receive shadows from surfaces" is an option that only affects clouds and atmosphere (shadows of terrain into the atmosphere). If you enabled it in the atmosphere it would cast shadows into your haze, which might be a cool effect, though it would take longer to render, especially with this many lights. You're right, however, that if you have lights behind terrain and atmosphere is enabled, then you will need to turn on "Receive shadows from surfaces" to avoid seeing light sources glow through the terrain.

Personally I think you might be better off finding or creating some kind of "city lights" texture, applying that to the terrain, and making it luminous.

- Oshyan

Oshyan

Here's an example with a city lights image. Image is from here: https://visibleearth.nasa.gov/images/79616/baltimore-at-night
CityPollution_image-map-test1.jpg

TGD attached.

- Oshyan

WAS

I don't think this is something that will be correctly simulated in clouds alone, due to how TG handles illumination over point lighting. It's just not ready for that, and you may spend so much time tweaking to get it right just to end up with ridiculous render times for cloud lighting quality. But it can give you the overall glow in the haze, from there, you'll want to turn to Experimental Atmo Bloom to give you your actual blooms in the haze.