River/Lake model revisited (solved)

Started by bigben, September 01, 2008, 05:39:58 PM

Previous topic - Next topic

rcallicotte

These are very good.  Congratulations on discovering and clarifying how this works. 
So this is Disney World.  Can we live here?

Oshyan

Nice work Ben, it's good to have a workflow developing on this. I remember some earlier work you did on the same subject - do you find it better or worse in the more recent TP versions?

- Oshyan

bigben

#17
Quote from: Oshyan on September 06, 2008, 01:28:42 AM
Nice work Ben, it's good to have a workflow developing on this. I remember some earlier work you did on the same subject - do you find it better or worse in the more recent TP versions?

- Oshyan

The earlier model you suggested to me of using a default shader with masked opacity doesn't seem to work now. The visibility of the lake object doesn't seem to be affected at all. (very first example in this post). I haven't looked too closely into it yet, but that model may be more efficient from a performance point of view... explained later.

The behaviour of the masked terrain model (missing chunks on the top surface) was the same in the earlier versions of TG. The latest model described here is a bit simpler in that the water surface consists of modified data from just one source (the TER file).  With the addition of transparency, this makes creating rivers and lakes on real data like this so much more worthwhile. There is plenty of scope for roughing things up a little with some additional displacements of the river bed so it should be possible to create some very realistic bodies of water over a large area with relatively simple masks.

The main drawback to both the masked terrain and double terrain displacement models is that the render is effectively doubled, with one terrain/water shader and then the normal terrain.  In both models the vertical faces where the water surface drops below the terrain seem to take a very long time to render, possibly because of the complexity of the reflections. This is where the masked opacity model may be more useful, as you simply wouldn't be rendering those areas of water outside of the mask. I'll have to do some benchmarking on this, but it may be possible that displacing the areas outside of the visible water surface by even larger amounts so that they are out of the frame may improve render times.

The extra render time is particularly exacerbated with more horizontal camera angles, where you have to wait for the water terrain to render all the way to the horizon, and then again for the normal terrain. In the case of my full Grand Canyon terrain set this can be a long wait indeed... but I'm just glad that it's working and looking really good.

The last render is still going but it is looking very promising. Just waiting for the bottom half to see how sexy the water transparency looks in the foreground.

bigben

#18
Here's the rest of that render.  The fake stones are nothing special.. just providing some extra detail to test the water transparency and the use of the river mask for controlling fake stone distribution.  Final render time was around 17 hours, about 5 of which was waiting for the last remaining bit of the lower right bucket to be rendered before the overlying surface could be rendered. I'm not quite sure why this particular bit took so long, but it makes further benchmark testing of different displacement settings potentially worthwhile.

While you can't see it here, the water surface extending below the bank of the river is much better than the masked terrain model which tended to have a more rounded edge,  giving the river a slight bulge.  This is much flatter, and with the surface glitches solved is going to make for some very interesting work.  I used the Grand Canyon  as an example partly because it has been rendered so often. In most cases people simply flood the floor of the canyon, but the river in my scene was traced from Landsat imagery providing more accurate changes in width, course, islands/branching etc...

dandelO

#19
This last one I think has nailed it! How far lateraly into/under the banks does the river extend? One of the main problems when rendering water is waiting while the sub-terrain portion of a default waterplane is rendered, unnecessary time. I know you will still need to wait on the verticaly displaced portion of the river's 'walls, which you say take an age to render, but could you mask the water shader distribution of these vertical parts with an altitude/slope constraint, so as to only make the water shader render just slightly lower than the height of the riverbed and, not on vertical surfaces at all? that should still be significantly quicker, I'd think. That way, the reflections/transparency do not need to be calculated beneath the riverbed, only the 'terrain' displacement, which you could maybe also completely mask beneath the 'real' terrain awell?

INSTANT EDIT:

QuoteThis is where the masked opacity model may be more useful, as you simply wouldn't be rendering those areas of water outside of the mask.

Lol, sorry, I missed that.  ::) :D

bigben

#20
I have a render running at the moment so it's a little hard to check, but from memory the model mentioned in the very first post is supposed to do just that.  [edit] no warning of the edit while I was posting ;) [/edit] The adjusted terrain is applied to a lake "object" which also has a default shader using the river mask as an opacity mask, so that only the masked portions of the lake object are visible. I will be looking at this more closely later on, as it would be the most efficient model in terms of minimising the amount of subterrain rendering.

Another workaround may be to lay another surface above the water shader so that it covers the vertical faces and regions outside of the exposed water so that there are no complex depth/reflection calculations required for these areas... see, this is why I blog my experiments here... ideas come from the feedback  ;)

bigben

#21
Quote from: bigben on September 06, 2008, 09:43:17 PM
Another workaround may be to lay another surface above the water shader so that it covers the vertical faces and regions outside of the exposed water so that there are no complex depth/reflection calculations required for these areas... see, this is why I blog my experiments here... ideas come from the feedback  ;)

Cancelled the render, as this was more interesting  ;)  A few carefully selected numbers for the colour adjust shaders and I've added a grey surface above the water shader that only covers the smallest areas of the upper water surface (but is still well below the terrain). The precision of the colour adjust shader is for creating multiple masks from images is fantastic.  I'm using differences invalues in the range of 0.01 and it's working exactly as I expected.

Thanks for making me think some more dandelO (and thanks for the mixed rocks that that are about to go on the riverbed  ;D)

The screengrab shows the adjusted node network and a preview of just the water surface. A quick test showed an improvement in speed. The calculation of the vertical faces is still relatively slow but greatly improved.

Also added an additional displacement of the riverbed to rough it up a bit to start making things look a little more natural.

dandelO

I noticed when I made this: http://forums.planetside.co.uk/index.php?topic=2992.0
That the large vertical displacements do take a bit of time to render. This is simply terrain displacements with lots of crater shaders and minimal texturing(which doesn't react correctly to the displacement, displacements come BEFORE colour so, why don't the textures overlay the pre calculated displacements at their correct scales/resolutions? Instead, the textures are stretched in the same way as the terrain. verticaly displaced walls do take longer than they should, in my opinion.

Why not try a TG plane object for the river? It's not connected to the planet surface like the terrain method and, could be more easily masked from the riverbed, down, than the default lake disk.
A TG plane can be masked via black/white channels so that it will only render portions of the plane designated to 'white'. The whole ground portion of the river could be removed from rendering completely by using the invert of the river mask, and vertical constraints on the water shader would stop the need for rendering reflections on the already slow vertical displacements.
I dunno... Great tests though.

bigben

#23
I had contemplated using a plane as well, and have constructed a model based on that as well.  Getting the displacement and water surfacing applied to both is easy, but I can't seem to use the river mask as an opacity function with either object type. Can someone confirm whether this should work or not?... and if it should work, some suggestions on how to get it working?

Applying the colour adjust node from the image mask to the plane's opacity function results in a black object instead of invisible.


[update]
NVM, almost there.
1: Add plane, set size to image mask
2: Connect second terrain (minus fractal noise) to input of plane's default shader.
2b: Connect river mask to opacity function
3: Insert surface shader between default shader and plane's shader input
3b: Connect water shader to surface layer child input
3c: Connect river mask to surface layer blending shader
4: (Kinda weird ???) Edit vertical adjustment of second terrain to position water level. In my test this is -16m even though the water surface is only 20cm or so below the unadjusted terrain level.

This results in just the top surface of the water. The preview render is looking OK. Running a test render... very promising  :D
Back to low detail (Detail 0.35, AA 3, GI 1,1) but as you can see by the subsurface render in the second bucket there are no vertical faces, and the time is reasonable.
[attachimg=#2]

bigben

Quote from: bigben on September 07, 2008, 02:44:51 AM
4: (Kinda weird ???) Edit vertical adjustment of second terrain to position water level. In my test this is -16m even though the water surface is only 20cm or so below the unadjusted terrain level.

Ummm duh!!!  A plane, of course is flat, which is why you need the lake object on larger terrains.  I'll redo this with the lake object tomorrow.  Off to bed now while a larger render of the last attempt completes.  This is about as efficient as you can get, looking at about 4 hours for 1000x500, Detail 0.5, AA 6, GI 1,4

bigben

Here's the render of the plane model described above. Final render time 3h 23m.

Not a bad results given that the waves need a bit of work and there are no other shaders for additional shore effects (wet sand etc...)  Had a quick play to apply this to other object types, but so far the lake, sphere and planet objects all result in vertical faces. Perhaps an imported sphere?

Matt

#26
Hi Ben,

The Water Shader ignores any shaders that come before it, so opacity masking of any object that uses the Water Shader isn't currently possible. (And even if you can get a shader further down the chain to mask away parts of the object it will be too late to prevent the Water Shader from being evaluated, unfortunately.) I will do something about this or maybe add a separate mask shader input to the Lake and other objects.

The good news is that you can use other shaders to control where the Water Shader is applied, e.g. by making it a child layer of a Surface Layer, and use your mask as the Surface Layer's blend shader. That should save you a lot of render time. When you applied a Default Layer to cover up the water shader I don't see how it could have saved render time, because the shading system isn't quite clever enough to prevent the Water Shader from being calculated when the later shader completely covers it up. Some shaders work that way, but many don't.

Maybe you could combine the Water Shader masking with the water object masking (if you can find a way to apply an opacity mask in a later shader without blotting out the reflections - I can't think of a solution at the moment) to mask away the black parts of the water object.

(Edit: Use the inverse of the mask as the blend shader for another Surface Layer that drives a shader with zero opacity, to mask away parts of the water object. You'd still want to combine this with masking of the Water Shader as explained above to get the shortest render time.)

Matt
Just because milk is white doesn't mean that clouds are made of milk.

Matt

Quote from: bigben on September 07, 2008, 05:33:53 PM
Had a quick play to apply this to other object types, but so far the lake, sphere and planet objects all result in vertical faces. Perhaps an imported sphere?

The vertical surfaces are caused by your displacement shader, so the choice of object won't change this. Or do you specifically mean that you don't like the way the surface is tessellated into long triangles? All the objects will suffer from a similar problem with very steep faces, although they might give slightly different results.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

bigben

#28
Thanks for the feedback Matt.... and once again, getting me to think through things a little differently. Firstly, here is a screengrab of the river surface from the previous render using a plane. This is what I'm trying to achieve using the lake object. A distorted plane for the surface, with 0 opacity where there is no water.
[attachthumb=#1]

I've had a quick play again, rearranging nodes and have solved it  ;D It all depends on how you connect the water shader as you metioed.  Put it in the wrong place and you get the steep sides, extending back down to the planet (with the possibility of gaps in the upper surface).  But now I have it working properly..
[attachthumb=#2]

[Edit] Attached a clip of the basic river model. Insert your own TER/river mask. Posted the clip file in file sharing http://forums.planetside.co.uk/index.php?topic=4459.msg49935#msg49935. The river bed consists of two displacements, a small uniform displacement to provide a minimum depth and a variable displacement to break up the smoothness resulting from the  antialiased image map. The rest is relatively simple, and pretty efficient in terms of rendering speed. 

The speed of rendering water (especially given that it now has transparency) is noticeably faster than earlier versions of TG2TP.  Now for some image based depth maps of lakes based on Landsat imagery... and other possibilities...  :)

bigben

Just giving this a bump to say that I checked the clip I uploaded and found half of the nodes were missing. Fixed the file and posted in File sharing.
http://forums.planetside.co.uk/index.php?topic=4459.msg49935#msg49935