Planetside Software Forums

General => Terragen Discussion => Topic started by: Dune on February 05, 2014, 03:31:43 AM

Title: grain in cloud
Post by: Dune on February 05, 2014, 03:31:43 AM
I'm trying to get the last grain out of the sky, but it doesn't seem to work without greatly increasing render time.
Upping AA from 5 to 6 increased render time to 150%, but grain was as much.
Upping cloud quality from 0.35 to 0.6 added 40% to render time, but grain was as much.
Upping atmo quality from 16 to 24 didn't do anything either, but upped render time 50% as well.
It's only some darker areas near the sun that show grain, so it's not too bad to blur in post (or just leave it), but I want to know nonetheless. Any advice?
Title: Re: grain in cloud
Post by: kaedorg on February 05, 2014, 03:47:47 AM
I don't want to scare you about render time but here are my settings for clouds :

AA : 8
Cloud quality : 1
Atmo quality : 64

Of course, render time is huge. Last render for a 6000 x 1800 was 160 hours

As i am lucky to have 3 rendering PC and not in a hurry, that's fine for me and result has no grain.

I think that you can't afford such kind of render time for these grains. I would go to post.
As I didn't try to see which setting is the more efficient about grain, i can't say which one has to be up in priority.

David
Title: Re: grain in cloud
Post by: Dune on February 05, 2014, 03:55:06 AM
I'm terrified now  :(  That's awfully long.
That's the problem, I don't want these huge render times. Last calculation showed that with detail 0.5  AA5 and atmo samples 16, cloud Quality around 0.4 one whole render of 9600x6000px will take 16 hours. I (we  :D ) need to do 14 of those. So I try to keep it a bit low, but with quality that's just good enough. At a certain point it won't matter much how much you increase quality, and I want to find that point.
Title: Re: grain in cloud
Post by: Tangled-Universe on February 05, 2014, 08:08:08 AM
Well, how do I say....a voice in the wildnerness of me saying to don't use RTA?

Matt recently stated, and he's right of course, that using RTA gives you predictable results.
This is logical, since AA4 gives you max 16 samples per pixel to apply AA on and that results in a certain noise level.
So as long as you don't severely undersample you will always end up with a similar result when using AA.

But when do you undersample???

It's a big problem to assess how much cloud quality you need vs. AA samples. You just don't know.
So let's have a look at another part of the render process which also has AA as determining factor for final quality: objects/populations.
There we often use AA8 or sometimes even a bit more to get smooth result with low noise.
Go to your pixel sampler settings in your render node and increase AA.
See how the samples increase and the noise threshold lowers?
That noise threshold is the difference in luminence between adjacent pixels that AA setting is aiming for.
So the lower number = the smoother the results.

Now you want smooth and clean clouds and thus you need quite a few AA samples, but also not too few atmo/cloud samples to start with.
If we don't have enough of samples set in our atmo/cloud then we can even use AA16 and still have grain.
The lack of samples causes so much noise that AA just can't get the job done.

Now you may think: ah that's easy...I just stick to AA4 or AA6, turn on adaptive sampling and give it enough atmo/cloud samples to work with.
Well...no.
You can also give too many samples which means that for every adaptive AA stage it needs to chew on all those samples to then find out the threshold level for luminance is not met and then continues to the next level of AA.
It can sometimes even be slower than undersampling!

Matt changed the cloud samples settings to quality only recently to avoid the discussion: I use 256 samples and my cloud is noisy...
To then later read that the cloud depth is 1000 with density and sharpness of 5.
Now the relation between those 3 factors are tied in the quality setting of the clouds.
A HUGE improvement in that regard.

Since we don't have anything similar for RTA I can make a long story made short. Don't use it.
Unless you know exactly how AA interacts with the atmo/cloud settings.
You need to reinvent this for every scene.
Title: Re: grain in cloud
Post by: Dune on February 05, 2014, 09:55:32 AM
Thanks Martin. Ì take it that you mean not using defer atmo/cloud? I could try that, see what happens. My clouds are rather normal, max height is 500m, max sharpness is 5, max density 0.05. Nothing special.
Title: Re: grain in cloud
Post by: Tangled-Universe on February 05, 2014, 10:36:51 AM
Ghehe, yes :)
(did I say somewhere you should? I can't see where quickly)

Try it and see which one you find working best.
I just tried to explain why I often don't use it and generally advice against using it.
I do use it though, but quite rarely.
Title: Re: grain in cloud
Post by: RArcher on February 05, 2014, 11:19:25 AM
I assume you are exporting this out in render layers Ulco?  If so, you could easily go with the lower quality / faster render times and then simply do a little noise/grain reduction/smoothing on the cloud layers in whatever software you use to put the image back together.  Getting it perfect in the renderer is a great goal, but not when it takes so much longer.
Title: Re: grain in cloud
Post by: Oshyan on February 05, 2014, 07:27:52 PM
Ulco, can you post an example image of the issue you're seeing? Definitely test with Defer Atmosphere off, though if it's a noise issue I really don't know why Martin expects it to be *better* with Defer off. You already increased values fairly high and if it apparently had no effect, it's likely to *not* be a straight sampling issue IMO.

Danny, aside from some sampling changes in the *atmosphere* that we mentioned in the TG 3.1 presentation (not in clouds, as far as I'm aware), there should be no difference between TG2 and 3 with raytrace/defer atmosphere. More compellingly, the sampling changes would affect non-defer as well. We'd really appreciate if you could do verified, exact-same-scene, side-by-side tests to demonstrate this problem you're seeing.

Discussion on raytrace atmosphere render time differences split out to here: http://www.planetside.co.uk/forums/index.php/topic,17706 (http://www.planetside.co.uk/forums/index.php/topic,17706)

- Oshyan
Title: Re: grain in cloud
Post by: DannyG on February 06, 2014, 07:45:19 AM
Agreed they are high settings, however the times are relative to lower sample values.
Title: Re: grain in cloud
Post by: Dune on February 06, 2014, 11:16:04 AM
I'm doing a (large) crop of a 9975x6000px render now, with defer clouds off, see what it will look like (3 hrs busy now). I'll later do some smaller crops and test the difference.
Title: Re: grain in cloud
Post by: Dune on February 07, 2014, 02:36:25 AM
Difference between defer off and on. Huge! Same detail setttings though, so it might be better with higher cloud detail and defer off. Still have to check that. It does cost time, as the defer off took 6 minutes, defer on took 19 minutes for this crop.
Title: Re: grain in cloud
Post by: Tangled-Universe on February 07, 2014, 06:15:45 AM
So what were the settings? (both atmo and cloud)
Title: Re: grain in cloud
Post by: Dune on February 07, 2014, 08:36:29 AM
cloud quality 0.4 atmo 16, so fairly default. Fatter cloud height 500m, density 0.05. Atmo not changed from default. Perhaps if I up cloud quality to 1 and do no RTA/defer, it might work out quicker and without grain. Have to check still.
I decreased cloud coverage for the final renders by the way, finding it too heavily clouded.
Title: Re: grain in cloud
Post by: Tangled-Universe on February 07, 2014, 09:18:05 AM
Quote from: Dune on February 07, 2014, 08:36:29 AM
cloud quality 0.4 atmo 16, so fairly default. Fatter cloud height 500m, density 0.05. Atmo not changed from default. Perhaps if I up cloud quality to 1 and do no RTA/defer, it might work out quicker and without grain. Have to check still.
I decreased cloud coverage for the final renders by the way, finding it too heavily clouded.

Ok that doesn't help much ;)
What was the AA?
What was the cloud quality?

For example, any AA greater than 4 with 16 atmo samples takes very long.
Atmo samples are the most expensive with RTA.

The best way to test differences is on a more representative part of the scene, including objects and such.
You can optimize your atmosphere/cloud rendering with RTA and then very likely find out that it doesn't work for your project, since objects/populations look rather ugly with AA4.

Or I can repeat my general advice: leave RTA alone ;D
Title: Re: grain in cloud
Post by: Dune on February 07, 2014, 10:14:31 AM
You're cryptic  ??? What doesn't help much, the settings I gave you? AA was 5, detail 0.5. Cloud Quality 0.4 as said in last post.

And your general advice; leave RTA alone  ??? You mean no defer atmo? That's what giving me the grain!
Title: Re: grain in cloud
Post by: Tangled-Universe on February 07, 2014, 01:48:39 PM
Quote from: Dune on February 07, 2014, 10:14:31 AM
You're cryptic  ??? What doesn't help much, the settings I gave you? AA was 5, detail 0.5. Cloud Quality 0.4 as said in last post.

And your general advice; leave RTA alone  ??? You mean no defer atmo? That's what giving me the grain!

I didn't mean to be cryptic, sorry :)
I missed the 0.4 cloud quality, sorry for that as well, but didn't know that AA was 5.

Of course RTA/defer atmo looks better, with enough samples and enough AA it will pretty much always look better, but is also (much) slower.
(I need to stop saying RTA, because it's now called "defer atmo", but they are the same thing)

I suppose you're considering using RTA for rendering your final wall project?
If so, what settings do you want to use there for rendering your terrain and objects/populations?
It's why I was suggesting to not do crops on atmosphere only, despite that you want to fix an issue specifically related to the atmosphere.
The reason for this is pretty simple, since rendering with defer atmo using AA>4 is slower than "normal" rendering.
AA4 is in pretty much all situations the point where deferred atmo rendering becomes slower than normal rendering.
So unless you want to render your project in AA4 I would turn off defer atmo and bump up cloud quality till the desired amount of grain.

I hope I have explained myself a bit better now :)
Title: Re: grain in cloud
Post by: Oshyan on February 07, 2014, 03:32:48 PM
Ulco almost always uses (relatively) low AA Martin (I believe AA5 may be the *final* goal for the project), he's not doing AA 12-16 like you do. Therefore Defer is a legitimate option for him. Although his test above shows higher render time, it also shows lower grain. If the grain is equalized the render times may be equivalent or better for Defer. Meanwhile Defer gives *higher quality cloud edges* with main detail below 1 (which is how Ulco will render). Since he's already tested the rest of his scene at AA5, as long as he keeps AA the same while adjusting cloud and atmo samples to get the results he wants, then testing on this crop is fine (because it's changing AA that would affect render times on vegetation, etc.).

- Oshyan
Title: Re: grain in cloud
Post by: mhall on February 07, 2014, 03:57:00 PM
Ulco, you'll want to keep in mind that, once printed and installed in the museum, that grain may not be visible in the least. As the final output for this is print, I think you would best be served by evaluating test prints at various rendering settings at the distances you expect this to be viewed (I know I said in an earlier thread I'd stick my nose against this thing :) ), but you're not rendering for that scenario.
Title: Re: grain in cloud
Post by: Dune on February 08, 2014, 03:11:20 AM
Thanks for your words, guys. Martin; you're quite clear now, and Oshyan is hitting the nail on the head. Especially since these (14) files are so big (9975x6000px), using higher AA and detail will skyrocket the render times, and people will wait in line at the museum's entrance in vain, waiting for the wall to be put up  :'(
And Michael is right as well, most people won't notice a little grain (or a straight branch of a too simple tree), but I'm picky.

And Ryan; no I'm not doing layers (yet). haven't experimented with it yet, and I need to put 14 big files together again, and try to keep it as simple as possible. I'm afraid having even more files will block my machine (or mind).

By the way; I'm reconsidering the sky, reduced coverage now, but I'd like some really big clouds, and blue sky in other areas, so I'm going to set that up anew. Keeping the sky layer that's giving the shadows I want, but only as secondary (thus for shadows only).
Title: Re: grain in cloud
Post by: Tangled-Universe on February 08, 2014, 04:20:11 AM
Quote from: Oshyan on February 07, 2014, 03:32:48 PM
Ulco almost always uses (relatively) low AA Martin (I believe AA5 may be the *final* goal for the project), he's not doing AA 12-16 like you do sometimes ;) Therefore Defer is a legitimate option for him. Although his test above shows higher render time, it also shows lower grain. If the grain is equalized the render times may be equivalent or better for Defer. Meanwhile Defer gives *higher quality cloud edges* with main detail below 1 (which is how Ulco will render). Since he's already tested the rest of his scene at AA5, as long as he keeps AA the same while adjusting cloud and atmo samples to get the results he wants, then testing on this crop is fine (because it's changing AA that would affect render times on vegetation, etc.).

- Oshyan

Slight correction added :)
My favorite setting is still AA8 and AA10 to a lesser extent.

I know what you mean and the way you think about it Oshyan.
It's just that I have never seen it work in the end being both the best and the fastest.
Title: Re: grain in cloud
Post by: Dune on February 08, 2014, 06:35:30 AM
I've been doing some tests this morning (with a different file), and am not happy. Can't get the grain out of rather fat (but not high) clouds near the horizon and under the sun, with detail 0.6 AA 6, cloud quality 0.4 AND defer atmo! A crop took 15 minutes, while no-defer with cloud quality 1 took 4 minutes, and had only slightly more grain.

With clouds higher in the sky I got reasonable results; see pic. But there's a color difference between no-defer and defer!

So I'm not ready yet. I'll put up the atmo file for the museum here, see what you guys think of it. And also to find out what is happening if you render with no defer! Something strange, like repeating the prepass. This is version in 3.0.12.1.

I may reduce the buoyancy of the cloud fractal, as that greatly fattens the cloud, and thus induces grain.
Title: Re: grain in cloud
Post by: Oshyan on February 11, 2014, 06:33:58 PM
I'm not seeing particularly noisy results in the crop in your file, but I did have to turn off Motion Blur because it was really messing up the comparability of the output. I don't actually see much noise even in the crops you posted, certainly an acceptable amount for printing. Are you still having problems with this? You described what seemed like a fairly noticeable and serious issue that didn't respond well to rendering setting changes. Rendering with Defer Atmo in TG3 on the TGD you sent results in very low noise results (albeit with higher render time)...

- Oshyan
Title: Re: grain in cloud
Post by: AP on February 12, 2014, 12:25:05 AM
I made some adjustments. Try this unless I missed something here.
Title: Re: grain in cloud
Post by: Matt on February 12, 2014, 01:00:48 AM
Hi Ulco,

I don't think ray fuzziness is helping, and is probably adding noise around the edges of the cloud. I would leave this at 0.
Title: Re: grain in cloud
Post by: Dune on February 12, 2014, 02:48:25 AM
Thanks guys. It's ok now. I realized I'm very picky, a little grain is no big deal. It's only certain darker cloud areas near the horizon under the sun. I upped the atmo samples to 24, undid the ray fuzzyness. But I needed defer atmo, with only higher cloud samples I couldn't get the grain out. Though I didn't go over 1.
Thanks ChrisC, but would 'optimal' cloud quality be better than 'none (highest detal)'? And what would be the benefit of a 2D shadow mask? I must say, I've never done much work on clouds, and I should!
Title: Re: grain in cloud
Post by: Oshyan on February 12, 2014, 03:07:02 AM
I assume you're referring to the Acceleration Cache in clouds. Usually with still images using Optimal is fine. It's in animations where you can sometimes see issues. There are occasional situations where you'll gain some notable detail by turning the acceleration off, but it's not that common in my experience. I would categorically avoid Ray Fuzzyness at all times because it's a somewhat experimental and advanced feature. 2D shadow mask can reduce render times but at the expense of shadow detail, and it only works for Localized clouds anyway.

- Oshyan
Title: Re: grain in cloud
Post by: Dune on February 12, 2014, 03:10:58 AM
I am. Thanks, Oshyan.