Terrain and shadows changing during animation

Started by seanQuixote, March 30, 2010, 06:05:43 pm

Previous topic - Next topic

MKE

Quote from: dandelO on July 29, 2011, 07:21:39 am
<terragen_clip>
  <render_subdiv_settings>
  </render_subdiv_settings>
</terragen_clip>


That's it! In my case it's not "fully adaptive" which makes the difference but "Force all edges". This one should really always switched on when rendering animations!!! "Jitter shading points" had no effect at all in my test images and "Fully adaptive" resulted in a little bit more speckle than without it so I left it away.

I now started testing to reduce the detail values and it seems that I can get away with detail=0.8 instead of 1.0 . That's 5 minutes and 30 seconds per frame instead of 8 minutes. That's a huge difference together with an improved image quality. Great.

And look at the images. It seems as if the problem of the on-and off switched indirect light has gone too. Greatgreatgreat...

Kind regards,
a happy Martin

... who now has one week off and has to be patient to see the new animation when he comes back...

Matt

August 02, 2011, 12:17:39 am #31 Last Edit: August 02, 2011, 12:19:21 am by Matt
Hi dandelO,

Fully adaptive ON is the default setting. The hidden node allows you to turn it off (which can help with animations), but as you've shown it is not always a good idea to do so.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

Tangled-Universe

Hi Matt,

Could you elaborate a bit why these particular settings can help with animations?
Does "fully adaptive" has to do with reducing subdivisions where not many are needed?
What does "force all edges"?
I suppose the jittering of shading points can result in shading differences between frames and thus some flickering/popping.
So far in my experience switching this off is really useful.

Cheers,
Martin

Henry Blewer

I noticed the odd shadows on a test anim. The one I did which lasted 30 seconds I worked around this by deleting the enviro light and using a couple fill sun lights.
http://flickr.com/photos/njeneb/
Forget Tuesday; It's just Monday spelled with a T

dandelO

I guess I must have saved my own file with 'FA' unchecked at some point, or the one that I salvaged from a .tgd in here already had it unchecked when I saved it or something.

I have tried animating a set of frames as comparison between with and without 'fully adaptive' and I didn't ever notice any benefit to unchecking it, other than the faster render times. I wonder, under what circumstances would unchecking it be beneficial? I only ever noticed a nasty edge and ragged shadow without it on before. The scene I used to test was a relatively close up shot of stones with animated warping in the surface shaders(liquid stones, was quite nice, and strange), maybe it works best to uncheck FA when the scene is very large scale and small shadow details aren't as important, instead?

But, when I first noticed that shadows had become more ragged at the default render settings was when I rendered this ant that Marc made some time ago against a white background. I remember thinking, why are my shadows looking so horrible when I'm using a good detail and AA setting. I knew nothing of the render subdivision settings node at that time, just over a year ago it seems. I think the pixel filters had just been included around that time but there were no editable AA sampling settings... I think.

MKE

Quote from: MKE on July 29, 2011, 11:47:35 am
I now started testing to reduce the detail values and it seems that I can get away with detail=0.8 instead of 1.0 .


The testing went on and on at that evening and I ended with quite different values as before. Mostly because "GI sample quality" changes did not lead to a steady improvement at lower values. The shadows over snow were flipping from bright to dark when I changed the GI sample quality by one and only stabilised when using real high values. On the other hand "GI relative detail" didn't have much effect but slowing down the processing considerably.

I ended up with the following values:

Quality:
Detail = 0.8
Anti-Aliasing = 4
Ray trace objects
Ray trace atmosphere
GI relative detail = 1
GI sample quality = 10
GI blur radius = 200
Supersample prepass

Extra:
Pixel filter: Narrow cubic
Anti-aliasing bloom
Detail blending = 3
Displacement filter = 0
Microvertex jittering
(Detail jittering off)
(Lock subdiv to frame off)
Do ray traced shadows
Ray trace everything

Advanced:
Ray detail region: Detail in camera
Ray detail region padding = 3
GI prepass padding = 0

Render subdiv settings:
(Full adaptive off)
Roce all edges
(Jitter shading points off)
Ray detail multiplier: 0.25


The animation looks very fine with a lot of details, smooth edges and no pixel flipping in the background any more. The shadows keep stable from image to image. Just one thing didn't improve: I still get some on and off switching of the indirect light when the camera is high above the ground. See attached three images where the first is ok, the second has less indirect light and the third none at all any more. This must be a software bug because the values during the animation didn't change at all. Matt, Oshyan, do you know how to handle this?

Kind regards,
Martin

Tangled-Universe

I'm wondering why Oshyan enabled "raytrace everything" for you.
It explains directly why your terrain looks inferior to your previous tests, because the ray detail multiplier is set at 0.25.
That means that if you render at 0.8 detail with the raytraced renderer, then the effective render detail will be 0.8 x 0.25 = 0.2!
Consequently that might be the reason why you also need that insane value for detail blending @ 3, which also slows things down.

This is what I'd do for render settings:

Detail 0.7
AA4
Raytrace atmosphere/objects enabled
GI relative detail @ 1
GI sample quality @ 6
GI Blur radius @ 40
SS-prepass enabled

Pixel filter: Narrow cubic (shouldn't matter that much I suppose)
Anti-aliasing bloom (shouldn't matter that much I suppose, post-render effect I'd say)
Detail blending = 1
Displacement filter = default (can't remember I ever touched this for animation, so keep it at default, though I can't remember whether default = 0 or 1)
Microvertex jittering enabled
Detail jittering disabled
Lock subdiv to frame disabled
Do ray traced shadows
Ray trace everything disabled

Advanced:
Ray detail region: Detail in camera
Ray detail region padding = 3 (that's high, but considering your problems it might be necessary)
GI prepass padding = 0

Render subdiv settings:
Fully adaptive enabled
Force all edges enabled
Jitter shading points disabled
Ray detail multiplier: 0.25 (this setting won't have an effect anymore on your terrain, since you disabled raytraced terrain rendering in the extra tab)

With these settings I recently renderer a far more detailed and displaced terrain than your project, so these settings should at least get you in the right direction.

The latest alpha contains a new advanced feature which should allow TG to deal better with the shadows from behind camera and stuff.
Don't know when that will be available.

Another question:

Are you testing your animation settings at the resolution you're posting?

Cheers,
Martin

p.s. you can send me your project file and I can help you along with finding settings, if you'd like.

dandelO

August 08, 2011, 08:32:12 am #37 Last Edit: August 08, 2011, 08:33:57 am by dandelO
I'd like to add/contradict something here.

I've been working on a play-scene(TG generated skatepark) for fun.

I found the settings I needed to use are the exact same settings as Martin describes for the render subdivision settings(without RTE), all save for one. The detail multiplier DOES actually significantly alter the outcome of non-ray-traced terrain.

See these small crops of some sharp areas displaced from the planet surface;

RDM=0.25
[attachimg=#]

RDM=1
[attachimg=#]

The way I thought it would work would be just as you described, Martin but it actually isn't. The RDM does have an effect, even when all logic(and an almost confirmation from Oshyan in my 'rescale noise' thread) says to me that it shouldn't.
I'm kind of losing my mojo with TG at the moment, this scene was just to actually make me use it and get my finger out of my arse but all these questionable settings and inconsistencies, even just the lack of explanation on what long-in-place settings do(intersect underlying for example, anyone? I know how to work it my way but, that's it), are getting to be too much work to be fun.

Again, what would be the point in un-checking 'fully adaptive' in an animation?

Tangled-Universe

August 08, 2011, 08:56:25 am #38 Last Edit: August 08, 2011, 09:13:44 am by Tangled-Universe
Hi Martin... (wow man, there are actually 3 Martin's in here ;D)

That's definitely as well as adding as disturbing/contradicting what you say now.
Need to check that at home. We both have a history of finding odd things which eventually weren't bugs or odd things ;)
So far I have no idea why the RDM would improve the jaggedness of the terrain and shadows. Well, the terrain's shadows are still raytraced, but it clearly seems that the edges are smoother as well, so it also seems to affect geometry. Weird! It's indeed contradictive to what we've always been told.
OTOH, the displacements are so steep/unusual that this might be the reason why the raise in RDM helps, whatever the reason it is. It might be an exceptional situation.


I understand your pain and agony. I thought you are an alpha tester as well?
On the alpha forums there's a 2 part tutorial on Intersect Underlying. The first part covers the basics. The second part is announced/promised.
It took Matt over a year to do it, so it might take another year for him to make the second tutorial ;)

I'm still awaiting his response to my answer my questions related to "fully adaptive" and "force all edges".
Seems I'm not alone anymore.

Earth to Matt ;)

Cheers,
Martin

dandelO

It doesn't seem to just happen with really sharp edges, Martin, I've noticed the difference with the RDM on simple fake stones shader's displacement as well.
Just a clear explanation on the more obscure setting's functions would be adequate, of course, the subdivision settings aren't actually meant to be out there yet but there are many other areas where I find contradictory results and outcomes, for example, what exactly is the relaitionship to the IU functions in line with the smoothing tab(and compute terrain smoothing), so far, it's been officially said that those settings all relate, but not how they do. Similarly, the noise variation methods in probably the most common shader node(power fractal), even in the online documentation, is still missing that information, the node reference for the PFS seem to be the very same ones that Frank wrote around 3-4 years ago and as yet, hasn't been updated.

I think it just comes down to the old story that everyone already knows, there is no solid documentation and labelling of many features. Many of us can work things out by repeated trial and error methods but, not everyone has the time to just sit and render back to back tests checking one setting against another, I think people want to be able to read that this slider does this, that checkbox does that, is all.

MKE

Hi Martin,

thanks for your help. I startet processing frame 30 (the first frame where the indirect light was completely missing) using your values. It's faster than before: 2'28" instead of 4'02". The shadows are ok now so I repeated frame 30 again with my old values, and the shadows become completely black again. Damn... Interestingly the shadows are longer with my old set of values...


I tried again with frame 735 with near foreground and with 1137-1140 with shadows on snow. The details are still ok, just the area in the background changes quite a lot and new kinds of shadows are created. I'll let it run over nigt to see if these shadows change while approaching the terrain. If not, then this is fine. I just saw that your value of 6 for GI sample quality is not enough in my case. The shadows over snow change a bit from frame to frame. I'll stick here to 10. Hmmm, one more thing: After approaching the earth surface, the length of the shadows seem to be more or less the same. That means, the shadows with your values must change when approaching them. I'll see tomorrow.

Concerning resolution, don't be afraid: My renders have 1920x1080 pixels. I just always cut out as little as possible to show the problematic area.

Thanks for offering your help concerning the project file. I'll attach it but let's first look what happens to the animation that will run tonight. I'll write tomorrow about it.

Thanks,
Martin

Tangled-Universe

Quote from: MKE on August 08, 2011, 10:37:55 am
Hi Martin,

thanks for your help. I startet processing frame 30 (the first frame where the indirect light was completely missing) using your values. It's faster than before: 2'28" instead of 4'02". The shadows are ok now so I repeated frame 30 again with my old values, and the shadows become completely black again. Damn... Interestingly the shadows are longer with my old set of values...


I tried again with frame 735 with near foreground and with 1137-1140 with shadows on snow. The details are still ok, just the area in the background changes quite a lot and new kinds of shadows are created. I'll let it run over nigt to see if these shadows change while approaching the terrain. If not, then this is fine. I just saw that your value of 6 for GI sample quality is not enough in my case. The shadows over snow change a bit from frame to frame. I'll stick here to 10. Hmmm, one more thing: After approaching the earth surface, the length of the shadows seem to be more or less the same. That means, the shadows with your values must change when approaching them. I'll see tomorrow.

Concerning resolution, don't be afraid: My renders have 1920x1080 pixels. I just always cut out as little as possible to show the problematic area.

Thanks for offering your help concerning the project file. I'll attach it but let's first look what happens to the animation that will run tonight. I'll write tomorrow about it.

Thanks,
Martin


Interesting that the shadows become longer. Much longer or just a tad bit longer?

Concerning the changes you describe for frame 735 and 1137-1140.
How does the area in the background change? What do you mean?
Is the rendered outcome different from your previous tests? That would be logical, as you used quite different settings which likely give a different visual result.
Or is the background area changing in terms that the difference between frames is (too) big? Like geometry popping or shadows popping? That would be need to be solved then.

If it is the first issue then don't worry. This is how it looks and that's it. No one will ever know how it otherwise would look, except for you ;)
If you don't like it, then change the design, as this is likely the way you designed it.

You still might try GI 1/8 instead of 1/10. It still could shave off a bit of rendertime.

I'm not sure what you mean with " the shadows with your values must change when approaching them"?

Sorry I had to ask about the resolution ;D In the 4-5 years I'm active here I've seen the weirdest and simplest things done wrong :)
I just had to ask/make sure.

Cheers,
Martin

MKE

Quote from: Tangled-Universe on August 08, 2011, 12:03:05 pm
Interesting that the shadows become longer. Much longer or just a tad bit longer?


Quite a bit. Compare the two images of frame 1, once processed with your parameters and once with mine. I like to use irfanview to flip from one image to the other using the arrow keys, then you can see slightest differences in two images.


Quote
Concerning the changes you describe for frame 735 and 1137-1140.
How does the area in the background change? What do you mean?

Is the rendered outcome different from your previous tests? That would be logical, as you used quite different settings which likely give a different visual result.
Or is the background area changing in terms that the difference between frames is (too) big? Like geometry popping or shadows popping? That would be need to be solved then.


The background is a bit different depending on the way how it is processed. This is why I let the computer run last night to process some 80 frames. Now I'm sure that the differences in the background are no problem because it is consistent from frame to frame, no pixel flipping in the background (as I already had in the past...)

But there is a serious problem: I get again shadows in areas where they should not be and when I approach, they disappear, just look at the subsets of frame 1120-1124.

I think, that's the reason why Oshyan switched on "Ray-trace everything" in the tgd-file some weeks ago... Bad luck, your parameters only need half the processing time.

ReCheers,
Martin

Tangled-Universe

Quote

I think, that's the reason why Oshyan switched on "Ray-trace everything" in the tgd-file some weeks ago... Bad luck, your parameters only need half the processing time.

ReCheers,
Martin


Hmmm...yes that could be the reason, not sure though.

Bad luck? I don't know!
I think this is a good point to start from. Basically the only problem left is the shadow inconsistencies. The length of the shadows, I wouldn't be bother too much about it, but that's me. It works and it's the way it looks. Pragmatic, I know.

So did you render these last 4 frames with my settings or a mix of mine with yours? What's the GI sample quality?

Perhaps you can post the current rendersettings again? I think it's worth to start tweaking from these settings, especially because the rendertimes are significantly shorter.

MKE

Quote from: Tangled-Universe on August 09, 2011, 03:58:53 am

Bad luck? I don't know!
I think this is a good point to start from. Basically the only problem left is the shadow inconsistencies.


Well, I think you're right. I saw that the glass is half-empty, you see that it's halv-full. That's the better way to look at things... ;-)

Quote
The length of the shadows, I wouldn't be bother too much about it, but that's me. It works and it's the way it looks. Pragmatic, I know.


I'm with you. I don't want to have unreal features in the elevation  model like fractal details but the shadows are just a way to make the DEM look nice. So it doesn't matter if they are a bit longer or not. But they should stay on their place on the different frames. It looks really strange if the shadows crawl away when you come nearer.

Quote
So did you render these last 4 frames with my settings or a mix of mine with yours? What's the GI sample quality?

It's your settings but with a GI sample quality of 10. I did some tests concerning the GI settings some days ago using a smaller render image size. I got the following render times:
GI sample quality  8: 2'39"
GI sample quality  9: 2'40"
GI sample quality 10: 2'43"
GI sample quality 20: 3'07"
This is why I prefer to keep the sample quality at 10. Maybe the whole animation would be good with a value of 8 too, but maybe there is a tricky shadow area again where it makes a diference. And I cannot test all frames before, so I prefer to have the render time increased by 2%...

Quote
Perhaps you can post the current rendersettings again? I think it's worth to start tweaking from these settings, especially because the rendertimes are significantly shorter.

I didn't change the tgd-file in the meantime, it's still the same.