Newbie - help on generating specific worlds from heightfields

Started by Todd, January 12, 2012, 01:38:48 AM

Previous topic - Next topic

Todd

Sorry guys, I was away from my PC yesterday, so only picking things up this morning (Australia):
@ cypher - the -30 displacement was set as this was the value in Dunes example (I did wonder about it, but thought maybe -30 did some undocumented magic/trick to a shader  :-[... I assume it sunk blacks below 0m height to ensure my sea was at sea level.  I did try alternative displacements (positive, to see if I could lift the terrain). Also see below

@Oshyan - I have 2.3.22.1(Free) running on a Win7 32 bit.  My laptop is Win7 64 bit (U7300 processor at 1.3GHz with 4GB RAM)...do you think I'll have better luck running this on the laptop even though the processor speed is much slower?  I'm reluctant to look at using another application, as I'm having trouble enough learning this....but I do appreciate the advice (and have seen similar advice elsewhere) so will give it another shot (I had a go a week or so ago, and it wasnt obvious how to load the segments so I meover back to TG2 to see if I could do it here).  Thanks for your advice - and its impartiality (and thanks for giving me the reason to justify to my wife why I MUST have a new PC  ;))

@ Matt - thanks for the comments.  I started with a straight heightfield load, but it seemed to crash my system more often than not.

@ Dune - Yes I did hit generate...thats one trick I learned early (always regenerate after every change).  I found the problem....basically at that scale unless I set the amplitude to 1000 the height just doesnt get off the deck, so as soon as I pug the shader into the heightfield generator it flattened it totally out.

@stgermain - I'm interested in your second paragraph as it sounds like what I want to do...are you able to provide a bit more detail about how you did it (Sorry blend shaders are a mystery to me so any pointers will help!!)

So, here's where I'm at - I now have a working, non crashing heightfield loaded!!!! See the next post (just to keep the size of each post small(er).  I'll post details which I'm sure most of you all know, but will state it just in case a newbie like me reads this post....


Todd

Per my previous post I now seem to be able to reliably load my data and add oceans.

This morning started with me following Cyphyr's advice of a load directly from the shader into the compute terrain node.  I started with a small file covering an island (1000x1000pixels) to test it and it worked perfectly in that it gave me a clear image of the island BUT it only gave me some small bumps where it existed. 

I adjusted the "Displacement Amplitude" up and the terrain popped out nicely.  I then rescaled the map from my 1000 x 1000 to 2000 x 2000 and found that I needed to increase the amplitude again to keep the heights. By the time I increased the scale to 10k x 10k the amplitude needed to be >1000 to show any meaningful height...and 2000 was better.  This indicated what I'd been doing wrong previously when I only set amplitude up by a few hunderd on a 10kx10k map.

At this point I tried loading my original TIF (10k square) and set amplitude to 2000 - it showed up OK as long as I had all other seetings as per dunes screenshots.

Subsequently I found some limitations to Cyphers approach for what I wanted as it (obviously in retrospect) does not give me access to the heightfield operators such as erode, resize and adjust vertical which I'd learned and knew how to operate to give me the look of the terrain I wanted.

I then followed the approach oultined by Dune and pointed my Shader into the Heightfield Generate node, shader input and after hitting "generate" ( ;))  I had the terrain loaded as a 10k heightfield - and no crashes at all thus far!!!  Anyone new reading this, make sure you ensure the dimensions in the shader are exactly the same as the dimensions in the generate...otherwise you may not see the terrain you're looking for  (it being offset from the bounding box)...and dont forget to hit "Generate Now" after each change.  Also, un-check the "Flatten Surface first" option on the Displacement tab of the "Heightfield shader 01" as shown

Next I decided to increase the size to 2900km sq...and hit my first crash.  To be fair, I'd probably run 10-20 variations, loads, resizes etc, so I restarted the application and loaded the last set of settings and it loaded OK.

Next I wanted to load the oceans.  Before doing so I looked at the height of the black areas of my map - which should always be below sea level, however (I assume due to the spherical nature of the planet) I noted that from the centre point, the height of the lowest point varied the further I went from that centre-point (for newbies, the problem is worse if you load the image from the "Lower Left" as you now have to deal with the curvature of the earth over 2900km rather than half that distance).  To counter the problem I changed the radius of the planet (Objects>Planet 01) from the default 6.378e+006 to 6.378e+009 to effectively make the surface 'flat' over a 2900km distance.  Not sure what impact this will have on the atmosphere/sunset etc later,...but I'll sort that out later.  Would love to hear if there's some smarter (easy) solution...I did read on the forum, someone projecting the image spherically from the centre of the planet, but the description of how that was actually done was pretty sparse and I couldnt follow it.

Now that I had my sea level relatively flat, I played safe by loading in 4 x lakes of 500k-700k radius just in case this put less load on the memory/app.  That seemed to work OK. .  I've now saved my map, with all relevant settings as shown in the attached (those NOT new to this will find it mundane, but anyone new and trying to do the same thing may be interested to see my settings).

Now to try some erodes (may go to another app based on other recommendations), adjust the heights and do some smoothing before I start colouring it in....oh yeah, and try to define the rivers (currently a 1 pixel black trench drawn in my TIF).

stgermain

Quote from: Todd on January 14, 2012, 05:23:23 AM

@stgermain - I'm interested in your second paragraph as it sounds like what I want to do...are you able to provide a bit more detail about how you did it (Sorry blend shaders are a mystery to me so any pointers will help!!)



I attached a picture of a simple node setup here. The basic idea here is that i have a camera high up over the terrain (200 km in this case) looking down on the planet. This camera is connected to an Image map shader that has a black and white map loaded and projection set to camera. This is a blending shader with white parts of the loaded image blending 100% and black part 0%. This is connected to a blending shader input of a power fractal that has small amplitude but large positive displacement. Now the fractal rises from the ground wherever the map is white. The other fractal is just to give some variation to the ground.

Advantage of this method is that you have the image mapped on the planet without any problems from the curvature. You can use as low res maps as you dare because all the detail comes from fractals. You can use other maps projected through the same camera blending other aspects of your terrain.

I made a sea from another planet in the same coordinates as the original, but with 200 m larger radius. Water shader is connected to its surface shader input. (If you try this, don't forget to remove the atmosphere shader from it or you will have very strange haze...)

Todd

Thanks for that!
I could see what you're suggesting and it looks such a simple and elegant solution.  I would have said foolproof if I hadnt spent the last 4 hours trying to get it to work...and I really cant see what or how I went wrong. I've fiddled with just about every combination of variables I could think of (hence the last 4 hours).  The attached chronicles my frustrations up to the time I threw the keyboard through the screen...

dandelO

Hi, Todd. I have been intrigued by this post but I'm kind of struggling to understand.
Is it absolutely necessary to have this .tiff run through a heightfield shader? Do you desperately need the data to be loaded as a geotiff heightfield? I've steered away from posting here because I'm really uneducated about using geotiff data in TG, never needed to use it before and so I'm not the best qualified to post here, as I know many of these guys have much greater knowledge on such things.

What I'm stuck on is, you seem to be facing multiple problems and discrepancies using heightfield generate nodes, I've read all your uploaded notes and I think you might be making things much more complicated than they need to be by even using any heightfield operators at all.

Using just the small preview sample uploaded on p.1, I can load, map and locate accurately the areas of interest into TG without distortion. Could you get away with simply loading an image map shader of your greyscale terrain, plugging it into 'compute terrain' and then using different mask maps of each area you want to mask, as masks 1 through 6(top to bottom in your colour chart, for easiness sake) Like the following .tgd...

Replace your own 10k image map into the terrain map shader and your own 10k masks in each of the 6 mask shaders(white=yes/black=no cover).
Then apply each set of shaders you require to each region, using the correct mask as blend shader to those particular areas. As long as your image maps are all made the same size in the image map 'size'(29,000m/plan Y/centre) parameters then, you should be easily able to single out individual areas to apply the appropriate shaders to by assigning each mask as their blender.

If I'm way off the mark and this must be done through the geotiff heightfield method then, please just ignore this post, as I said, I really am not qualified to advise on those methods. It just seems like you could create this setup with much less hassle than you're facing so far.

*EDIT*

There's also a sea object in the .tgd, enable it and then open the node 'water level - offset' and use the 'colour offset' parameter in the colour tab to control how high you want your sea level in metres.

dandelO

In each of the 'areas' shaders, colour is just applied as an example guide, according to your mask image. To use each surface layer to apply appropriate shaders to each area, just uncheck 'apply colour' and plug in the new shaders you need to each surface layer's 'child layer' input.

* Actually, this is pretty much exactly what stgermain just wrote before me, use your image map simply as a blend shader to the following shaders. Sorry, I wasn't paying attention, typical me. :/ *

Todd

Hi Dandel0

Thanks for taking the time to read/model and reply...are you able to send over the tgd file for the above images, I'd love to see the inner workings of how you got there.  I absolutely follow what you've done, but I have had no end of problems trying to get a similar result - probably just missing one setting. 

What happens if that map is stretched over 2900km?

It seems (to me at least - though I'm sure Ive missed a magic setting somewhere...) that over a 2900km scale TG2 just doesnt wrap the image around the surface.

I tried to load separate heightfields using individual masks as you've shown there, but kept on hitting a wall - the masks didn't appear to work as each time I loaded a new heightfield it seemed to ignore the mask and overwrite whatever terrain was already there, so effectively all I got was the last terrain loaded. I read extensively on the forums - there being no great documenation that I could find - but I never seemed to get it to work.  Regardless, as this was also loaded as a 2900km scale I still had the curvature problems i.e. black at the centre was 0km above the ground, whilst black (and most heights) at the edges were at -16km, white at the origin would be +12km, whilst the same white at a point 1450km away would be -3km.  I then tried loading individual maps of a small scale with the thought that I could stitch them together, but I always had a massive drop off at the edge of each map (eg at the colour boundary), so I could never put them together without a canyon being present.

The only solution I could figure out was to increase the radius of the planet so that a 2900km segment was effectively flat, this works but leaves me with a flat horizon (which I can live with as I'm a bit fed up with curving earths anyway!)....

Todd

Oh, and I am using Black+White masks, not colour....the colour was just for explanatory purposes.

dandelO

All the files I used are in the archive I attached to my first post above the preview image(just download, unrar/zip to find the project folder containing it all). It should load every image automatically when you open the .tgd.
The images are all loaded at 29,000m, you'll want to change those to 29,00000m, sorry, my mistake in translation but the end result should be the same, just raise the camera to compensate and fit everything into view.

As for canyons appearing at each blend shader border, you might try to blur the edges of each mask by a few pixels in an image editor before loading your masks. That should take away the sharp cut-offs. :)

Dune

Maybe you can adjust your image map so that it can be projected spherically to the earth.... Ration 2:1, but how to adjust a square map, I don't know. No merely stretching I presume. There must be a way, or there wouldn't be such nice planet maps.

stgermain

Quote from: Dune on January 17, 2012, 02:33:26 AM
Maybe you can adjust your image map so that it can be projected spherically to the earth.... Ration 2:1, but how to adjust a square map, I don't know. No merely stretching I presume. There must be a way, or there wouldn't be such nice planet maps.

I used Hugin to make a spherically projected map. I needed to calculate the horizontal field of view of my map (it was nearly 90 degrees) and Hugin did the rest, but i still got some distortion. The problem was that my map was not originally made in any real projection so directions and distances got warped a bit.

I think you could put a projection camera to the center of your planet, adjust the fov to what your map represents and have the map mirrored in x-axis. This way terragen would do the projection by itself. (directions would be totally off near the poles of your hypothetical planet, but if you only want something in the scale of few thousand kilometers somewhere below 60 degrees it should work fine.) I got to try this...

In this setup it would be very easy to adjust the rotation of your camera to center your scene location in the map to the origo and you could have nice upright populations wherever you want. Then you need a transform shader to rotate your terrain with the same numbers and the terrain would stay the same too.

Quote from: Todd on January 16, 2012, 10:57:59 PM
Hi Dandel0

Thanks for taking the time to read/model and reply...are you able to send over the tgd file for the above images, I'd love to see the inner workings of how you got there.  I absolutely follow what you've done, but I have had no end of problems trying to get a similar result - probably just missing one setting. 

What happens if that map is stretched over 2900km?

It seems (to me at least - though I'm sure Ive missed a magic setting somewhere...) that over a 2900km scale TG2 just doesnt wrap the image around the surface.

I tried to load separate heightfields using individual masks as you've shown there, but kept on hitting a wall - the masks didn't appear to work as each time I loaded a new heightfield it seemed to ignore the mask and overwrite whatever terrain was already there, so effectively all I got was the last terrain loaded. I read extensively on the forums - there being no great documenation that I could find - but I never seemed to get it to work.  Regardless, as this was also loaded as a 2900km scale I still had the curvature problems i.e. black at the centre was 0km above the ground, whilst black (and most heights) at the edges were at -16km, white at the origin would be +12km, whilst the same white at a point 1450km away would be -3km.  I then tried loading individual maps of a small scale with the thought that I could stitch them together, but I always had a massive drop off at the edge of each map (eg at the colour boundary), so I could never put them together without a canyon being present.

The only solution I could figure out was to increase the radius of the planet so that a 2900km segment was effectively flat, this works but leaves me with a flat horizon (which I can live with as I'm a bit fed up with curving earths anyway!)....


Problem with heightfields is that they don't wrap up very well, but a properly projected image will. The strange height readings you get when far off the origo are because your coordinates are straight but your planet is not. Therefore it is easiest to always work near the center of the coordinates.

With a setup like DandelO's or mine you would not need heightfields at all. You would just use blended fractals to build your terrain. (Don't forget to check the box "Blend by shader" in your fractal settings though). If you want to play with erosion for example, you could export few kilometers around your camera as a heightfield (I've never tried this...).

dandelO

Hi again, Todd. I had to sleep last night but I reopened this today and things scale up fine to 2,900km with no distortion or planning artefacts around the planet.

I would definitely avoid the heightfield operators, if they aren't entirely necessary, saving generation time and other scaling hassles. You'd then obviously need to find another way to erode areas if you need to do that.

As for spherical mapping, your entire area is not a complete globe map so you would be effectively stretching that square of data to cover the entire planet with spherical mapping, I don't think that's the practical or sensible way to go, far easier to just use planned Y projection and have all masks stretched to exactly the same dimensions. And, as stgermain says, keep everything as close to the coordinate origin as possible, that's where the 'position centre' option is handy. Good luck! :)

dandelO

If you place this edited .tgd in the same folder I uploaded yesterday and then open it up, you'll get the following result(probably requires a bit of displacement/water level tuning to suit your needs but it is all planned correctly to the planet);

[attachimg=#]

.tgd; [attachimg=#]

Todd

WOW - Thanks for the work on this...I hope I can repay the favour sometime in the future.

I havent had the time to read this today as I've been helping my dad clear out his house - 30 years of accumulated junk....just switched the PC on for the first time today and its 11:43 PM.

Will try this tomorrow afternoon (after a few more trips to the tip) - can't wait, it looks sensational .

Thanks again.

Todd


PS Also feeling a little but dumb for not spotting the first TGD!

dandelO

No prob's. I just thought it was being made much more difficult than was necessary with all that green node palaver. Richard and stgermain were on the right track by saying to just plug your image straight into your compute node, it saves loads of hassle over scaling heightfield operators and getting all those distorted, stretched surfaces on your planet.

Of course, you could still always plug them into a HF generate of the same size as your image maps and then, by right-clicking the node, save your terrain as .ter. But that shouldn't really be necessary, unless you desperately require the use of other heightfield operators(erosion etc) to use once you replace your generate with a 'heightfield load' on the saved .ter, instead.