Terrain and shadows changing during animation

Started by seanQuixote, March 30, 2010, 06:05:43 PM

Previous topic - Next topic

Tangled-Universe

Ok, thanks.

So I'm thinking out loud now: maybe the shadows change in length because the closer by the shadows you come the more screenspace the shadows take.
Consequently more samples would be taken and the shadows accuracy will change over frames.

*If* this is correct we'd need a setting which increases the samples. Before we start raising GI relative detail from 1 to 2 it might be good to test this by actually disabling GI. Does it still occur? If not, then it has to do with GI. If it still is, then not.

I can't stop think about Dandel0's test with the RDM increased to 1 without raytracing the terrain. It did affect shadows and edge accuracy, although we both completely do not understand why.
So after checking the GI, you might give the 4 frames a shot with increased RDM.


Dude, I've never seen someone having so much troubles with his animation :(
But we'll figure it out, eventually :)

MKE

Hi Martin,

without GI the non-existent shadow is still there. So afterwards I run with GI and a RDM of 1.0. And it looks good, but it's rather slow, about 14 minutes.

Quote
Dude, I've never seen someone having so much troubles with his animation
Hmm, I wonder why it should be more troublesome in my animation. It's a normal DEM without any special details so the problems I have should occur everywhere else...

The animation has already improved a lot thanks to the help of different people of the forum but still:


IT WOULD BE MUCH EASIER IF WE HAD A KIND OF HANDBOOK!!!


MKE

I did some tests concerning the ray detail multiplier now. The shadows of my last post already disappear at rdm=0.4 . Unfortunately there are others which are more stubborn. So I need rdm=0.9 to get rid of them all (at least I hope so...). Again, I'll let an animation run over night to see if I'm right.

Tangled-Universe

Quote from: MKE on August 09, 2011, 06:07:50 AM
Hi Martin,

without GI the non-existent shadow is still there. So afterwards I run with GI and a RDM of 1.0. And it looks good, but it's rather slow, about 14 minutes.

Quote
Dude, I've never seen someone having so much troubles with his animation
Hmm, I wonder why it should be more troublesome in my animation. It's a normal DEM without any special details so the problems I have should occur everywhere else...

The animation has already improved a lot thanks to the help of different people of the forum but still:


IT WOULD BE MUCH EASIER IF WE HAD A KIND OF HANDBOOK!!!


The test you did isn't very conclusive.
It is "w/o GI + rdm 0.25" vs "w/ GI + rdm 1", so the beneficial effects cannot be certain because of the GI or RDM alone.
Looking at your previous test it seems to be RDM.

Things start to make sense now, take a look at this quote from the description of RDM at the alpha test forum:
...allows you to control the detail of surfaces underwater, shadow-casting detail etc. See the Alpha Test forum for more information.

So the shadow-casting is being handled by the raytracer anyway it seems. Regardless whether you use ray trace everything (RTE).
With RTE enabled probably only the surface shading will also be rendered with the raytracer.
Increasing RDM makes sense for the shadows now. Why the edges are more smooth in Dandel0's example, don't know yet.
More information didn't show up on the test forum, so this is all.

MKE

Quote from: Tangled-Universe on August 09, 2011, 10:25:32 AM

The test you did isn't very conclusive.
It is "w/o GI + rdm 0.25" vs "w/ GI + rdm 1", so the beneficial effects cannot be certain because of the GI or RDM alone.
Looking at your previous test it seems to be RDM.

Befor this test I had GI and rdm 0.25.
Then the first test was just to switch off GI which had no effect on the shadows.
The second test was then with GI again and a rdm of 1.0 which eliminated the shadows.

So I always changed only one parameter and it's definitely the rdm which helps to avoid the wrong shadows.

Kind regards,
Martin

Tangled-Universe

The naming convention you used for the files clearly said something else, so that's why I made the remark :) peace ;)

How are things going now for the other frames with RDM increased?

Oshyan

When I messed with these files some months back I tried RTE as a sort of baseline comparison (because with a ray detail multiplier of 1 the raytracer is in some ways more "accurate", as I understand it). It turned out to solve some of his problems so I posted the file with that enabled. It clearly didn't turn out to be a comprehensive solution. What has been discovered since about RDM affecting regular shadows makes perfect sense to me, but I didn't think of it at the time. I'm not sure how much more there is to add but I'll have Matt look at the thread again to see if there are any more ideas on his end. As he said before though there are certain limitations which can't be easily resolved at this point and do require core changes.

As to why this animation is seemingly more problematic than others, my guess would that it's actually because it's such a simple scene. In an animation of a more complex scene there is so much for your eye to pay attention to and so much breaking up the shadows, lighting, etc. that it makes relatively small (or even not so small) errors harder to notice. So if your scene had trees and more complex terrain texturing, it would likely make these issues largely unnoticeable. But I understand this look is probably what you need to achieve.

- Oshyan

MKE

#52
Hi Martin and Oshyan,

Oshyan, thanks for telling Matt about this thread. I wonder if "ray trace everything" means the same as "ray detail multiplier = 1.0". When I look at the render time, it could be the case.

Quote from: Tangled-Universe on August 09, 2011, 02:13:40 PM
The naming convention you used for the files clearly said something else, so that's why I made the remark :) peace ;)
??? Yes, peace. But it doesn't say anything else. I just didn't upload the original problematic image with GI on and RDM=0.25, that's all. But that doesn't matter any more, it's clear now that it's not the GI which causes these problems and that with a higher RDM one can solve this problem.

Quote
How are things going now for the other frames with RDM increased?
Just before I wantet to leave office, I noticed that there is still a specular reflection in the shadow which looks unnatural. Therefore I increased the GI sample quality from 10 to 12 and now it's much better and not disturbing any more in the animation. There seems to be no problem any more with retreating and disappearing shadows, that's fine.
On the other hand, the render time gain of your solution melts with every parameter value I increase. It's now 17 minutes per frame compared to 19 minutes as it was before.


This morning I started the processing of the three frames which had the specular reflection inside the shadow again with my former set of values (RTE on), to see, if there is any difference.

Kind regards,
Martin

Tangled-Universe

Quote from: MKE on August 10, 2011, 05:04:04 AM
Hi Martin and Oshyan,

Oshyan, thanks for telling Matt about this thread. I wonder if "ray trace everything" means the same as "ray detail multiplier = 1.0". When I look at the render time, it could be the case.

No as far as I know that does not mean the same.
Remember when your previous settings were detail 0.8 and RDM @ 0.25? That's 0.8 x 0.25 = 0.2 effective detail.
RTE basically tells TG2 to not use the rasterized renderer but the ray traced renderer. Consequently the above equation will be used.
So to have your raytraced terrain detail in line with the detail setting in the renderer you need to set RDM to 1, because then the effective detail rendered will be the same, according to that equation.
I think you can test this yourself with a crop render. Set detail to 1 and RDM to 0.1 and compare that with detail 1 RDM 1. The first should look horrible, the last a lot better.

Quote
??? Yes, peace. But it doesn't say anything else. I just didn't upload the original problematic image with GI on and RDM=0.25, that's all. But that doesn't matter any more, it's clear now that it's not the GI which causes these problems and that with a higher RDM one can solve this problem.

Ah right, now I see, missed that info. Oh well, never mind :)

Quote
Just before I wantet to leave office, I noticed that there is still a specular reflection in the shadow which looks unnatural. Therefore I increased the GI sample quality from 10 to 12 and now it's much better and not disturbing any more in the animation. There seems to be no problem any more with retreating and disappearing shadows, that's fine.
On the other hand, the render time gain of your solution melts with every parameter value I increase. It's now 17 minutes per frame compared to 19 minutes as it was before.

That's a pity :( So basically you only increased RDM to 1.0 and set GISQ to 10-12? That's quite an increase in rendertime unfortunately.

I think Oshyan is right when he said that normally the complexity of a terrain covers things up and that the extreme simplicity in this situation painfully exposes the slightest potential problem which you normally really wouldn't encounter.
I don't know that doesn't help you, sorry.

dandelO

#54
Just for the point of confirmation on my earlier posts in this thread(sorry for the interuption on your animation thread, MKE, but I think it's important to keep this issue here where it was first brought up. :);

I said I'd noted this happening on fake stones as well.
Here's two tests, a simple and entirely default fake stones shader, no surface shaders or other modifications added. Both tests at final render detail=1 for consistency. Both with the default micropolygon renderer.

Firstly, an RDM setting of default=0.25 with fully adaptive enabled;
[attachimg=#]

Lastly, the very same with RDM=1;
[attachimg=#]

The difference is plain to see.

I'm out.

Tangled-Universe

Quote from: dandelO on August 10, 2011, 09:38:39 AM
Just for the point of confirmation on my earlier posts in this thread;

I said I'd noted this happening on fake stones as well.
Here's two tests, a simple and entirely default fake stones shader, no surface shaders or other modifications added. Both tests at final render detail=1 for consistency. Both with the default micropolygon renderer.

Firstly, an RDM setting of default=0.25 with fully adaptive enabled;
[attachimg=#]

Lastly, the very same with RDM=1;
[attachimg=#]

The difference is plain to see.

Scroll back and see why Martin :)
Well, not entirely why, but it definitely explains why the shadows are improved.
The edges is still a mystery so far.

Perhaps you have a lucid idea?

dandelO

Right. I knew rendered shadows were RDM dependant. Not that 'shadow casting detail' was.
This still really confuses me, Martin, as the shadow casting geometry is being rendered with the MP rasterizer and surely every element in a scene that has height and a light behind it will cast a shadow, what if I remove the light or make it directly above, so it won't cast a shadow? ...

So, pretty much every element in a scene is degraded by the default 0.25 RDM setting, all but colour shaders, really. Do I have that right?

dandelO

Or, even uncheck 'do ray traced shadows'? Will this then make the displaced surfaces render correctly?

Do you see what I mean, or am I confusing things further?

Tangled-Universe

Quote from: dandelO on August 10, 2011, 09:50:01 AM
Right. I knew rendered shadows were RDM dependant. Not that 'shadow casting detail' was.
This still really confuses me, Martin, as the shadow casting geometry is being rendered with the MP rasterizer and surely every element in a scene that has height and a light behind it will cast a shadow, what if I remove the light or make it directly above, so it won't cast a shadow? ...

So, pretty much every element in a scene is degraded by the default 0.25 RDM setting, all but colour shaders, really. Do I have that right?

Rendered shadows and shadow casting detail might be two different things, I thought it is the same.
I don't think the result will be the same with a sun's elevation at 90 degrees. It's not that TG2 thinks "hey sun's right up so no need for shadows"...hence overhangs by the way.

Yes, I'd think geometry is rendered with the MP rasterizer, but it seems some things aren't completely separated.
It seems indeed every element except the colour shaders.
Although I believe the shading of your stones with RDM 1 is just a tad bit smoother, but that might also be the improved geometry.

My guess is, you're right.

Quote from: dandelO on August 10, 2011, 09:52:25 AM
Or, even uncheck 'do ray traced shadows'? Will this then make the displaced surfaces render correctly?

Do you see what I mean, or am I confusing things further?

Yes and no? I don't know! ;D


Oshyan

Only guessing here, but maybe it *is* entirely down to the accuracy of the shadows and inaccurate shadows with lower RDM settings make it *look* ragged, but the micropoly rendered displacement *is* smooth. Hard to test this though because without shadows there's little definition to see whether the edge is smooth. But that's what I would try, disable shadows, leave RDM at .25, and see if you can tell.

- Oshyan