TG Earth model progress.

[bigben]

What should I really be expecting out of the box here?

An excellent question... with a few answers, partly due to the way TG allows the combination of data (images) and procedural surfaces.
1. There will be a few different resolutions of data available. For the most part, this will simply mean pointing an image or DEM loader at a different file, but the TG file itself will remain the same.  This is good for people with not much RAM, and also good for speeding things up for previewing work.
2. The maximum resolution of the terrain will be a bit higher than a 36K bump map (0.01°/px) along with 0.025° and 0.05°/px. At the absolute top end will be a set of 4 DEMs in 90x90° chunks (can't remember the exact resolution) but these require 6Gb of RAM just to load the DEMs.  Modified Blue Marble texture will be available in the same resolutions as well as 0.005°/px and the original resolution (85K) ... with antialiasing, the Blue Marble image will be the first thing to go so that's the limit of how close you can get.
3. IF (and I'm kinda hopeful here) the idea of masking colours in the Blue Marble images and replacing them with coloured TG surfaces works, then this should get you at least twice as close as just the image alone... closer depending on how creative you get with the TG surfaces.
4. ... and this is where TG really shines IMO... You don't need the entire globe at insanely high res to get to the surface. Having geotiff support makes it really easy to produce multiple LoD data and load it without having to do anything special. This model will be the foundation on which you could add additional data where it's needed, whether it's for stills or an animation.

In GlobalMapper for example, I can export only the data I see on screen at the equivalent screen resolution.  I run a screen res of around 1400px wide so for the USGS 10m DEM I get a 14km terrain at 10m/px.  Zoom out and I get 28km DEM at 20m/px.... and so on.  So it's quite easy to prepare data for a scene with extensive line of sight using relatively small data files. To avoid the need for masking terrains I always start with my global DEM in Global Mapper and then load in additional data on top of that. Then in TG it really is as simple as just loading all of the DEMS from low to high res.  Repeat that with images and yes, in theory, it should be possible to do a kick arse ground to orbit animation. Horizontal flight obviously increases the need for more data, but the vertical transition would nevertheless be an impressive proof of concept.

[Dune]

You might want to take care with the use of fake stones for masking ice flows, as the borders are absolutely sharp. Unless it's a very thin sheet with no real displacement above the ocean, it's better (IMHO) to use a quite hard billow (warped by quite angular PF's if you will). But you will most likely know this, as an old dog 

[bigben]

I'm more than happy to take suggestions.  The vast majority of my tinkering has been of a technical nature and I can assure you there's a lot I'm not very familiar with   I was leaning towards PFs as well, although for different reasons. Animating the extent of the ice over a year with fake stones would probably result in a lot of popping as the density map changed.

[lat 64]

Those sort of things were in the back of my mind while doing this but I've just used random numbers for now and haven't researched ice formations yet... Thanks for the kick start. I'm just building the basic model for now but keeping in mind the need to control settings and create masks for additional features such as you have mentioned. The massive ice shelves in the Antarctic are included in the DEM and this ice map adds in the seasonal sea ice. By combining the monthly coverage maps into a single greyscale image I think it would be reasonable to use this to also generate masks for different ice formations based on the age of the ice.

PS. The polar bear was a freebie off T3D. You can have the bear if you really want   Must check the scale of the bear. Ice is 2m above, 7m below in this

I should have known, you are way ahead of me. I'll sit on my hands and watch this thing being born. My old mac has only 6 gigs of ram so Im delighted to learn you may have a "DEM lite" version to play with too.

I'm off to hunt polar bears now,

Russ

[bigben]

Added in a "flat" ocean model.  Having real bathymetry and using a sphere for the ocean surface was working pretty well but at really high altitudes (1/2 - 2/3 of the planet visible) sections of the sphere were starting to disappear where the water was relatively shallow

DEM is still the same (including bathymetry) but the ocean bed is masked to be flat. The water colour is used to simulate depth (mask consisting of Displacement to scalar -> Colour adjust). This render is a simple 2 colour shader, but I'll dig up version I used for this: http://www.planetside.co.uk/forums/index.php/topic,2502.msg24609.html It should look better than that old project as the bathymetry data has improved since then.

Also found a glitch in the Black sea bathymetry in my model... back to the drawing board for that one, but I think it's an easy fix.

[bigben]

A quick terrain test using the 4 small geotiffs of 10, 20, 40 and 80m (total 16mb) resolution from another post with a 0.05° resolution global DEM. Last image swapped in the 0.01° resolution global DEM (2gb)

In the 4th image you can see the difference between the 80m DEM and the global DEM. The 80m DEM could be doubled in area, adding another 12mb or there is also a global DEM 0.005° resolution for those that have a lot of RAM. I don't think there'd be any problem with going from orbit to ground on the terrain front if you can get the data.

The rendering away from the top of the planet has certainly improved a lot. This is with the north pole at the apex.

[Oshyan]

Nice illustration Ben. The 0.01° data makes a noticeable difference and seems worth the memory use to me.

- Oshyan

[bigben]

It makes a big difference to the image as it should and I'd probably double the area of the smaller DEMs as well (only increasing to 64mb). The 2gb global DEM slows down the loading of the project into TG quite a bit on my home PC (open file, get cup of coffee) but once it's loaded performance is not too bad. I'll run some comparison renders at work on the full DEM set. The 0.005 degree DEM also makes a noticeable difference but it takes my home PC pretty close to max RAM usage.

[bigben]

Exporting a new DEM. Most of the SRTM data had no data over sea surfaces except for the Black and Caspian Seas. These have large blocks of identical altitude extending into the water, probably due to the presence of land at lower levels in the SRTM tiles (Black Sea altitude 0m, minimum altitude -89m; Caspian Sea -29m, minimum altitude -176m) 

I'm going to ignore the Caspian sea for a little while so that I can release a mini beta to get some input from others here...

[edit]I think it will be easiest to release an additional geotiff to patch the Caspain Sea rather than figure out how to flatten it in TG (AND rotate it with the rest of the globe)  .... oh crap... also figured out how to restore the SRTM data to low lying areas for the previous fix so that will be another 16hr export of the high res data. 

[bigben]

OK, here's a little Easter egg for those following this thread. Attached is a VERY basic TGD with a mask image. It should uncompress to a TGD and a "data" folder with an image mask. 

To this, you will also need a DEM to load into the empty Geog heightfield load. Here are the 2 low res versions I've made: https://cloudstor.aarnet.edu.au/sender/?vid=73861ca8-e40e-4f68-da38-00006512e7fb (600mb, link expires 14/4/2015)

... and a texture map of your choosing for the Geog image map shader. I'm using the Blue marble images from here: http://visibleearth.nasa.gov/view_cat.php?categoryID=1484  You want the images WITHOUT topography and bathymetry. ... but any full globe image will do.  The final model will use a modified Blue marble image.

The other bits that I've shown in this thread are still at the proof of concept stage. Now that the DEM is done I can get back to polishing these into shape.

If you're doing renders from orbit, try increasing the Height multiplier in the displacement tab of Global DEM to 1.7 for some extra relief

[bobbystahr]

Holy Smoke Ben those are huge, So are they applied to the planet or an added sphere?

[bigben]

he he, these are the small ones    Full release will include 0.01° resolution @ 2.1gb and 0.005° which is 4 tiles of that size.... and that is only 20% of the original resolution used to create it.
If you're setting up from scratch, Terrain > Heightfield Load DEM.  Must turn off stitchable borders or you will get a groove from pole to pole at long 180°.

[bobbystahr]

he he, these are the small ones    Full release will include 0.01° resolution @ 2.1gb and 0.005° which is 4 tiles of that size.
If you're setting up from scratch, Terrain > Heightfield Load DEM.  Must turn off stitchable borders or you will get a groove from pole to pole at long 180°.

muchos gracias

[Dune]

Thanks so much Ben. They are huge indeed, but I must try this, so your bits and bytes are traveling across the globe as of now.

[bigben]

They're big, but nowhere near as big as the source data they're derived from.
[attach=1]

Looking at those numbers now, I'm amazed at how GlobalMapper actually let me load it all at once and export the data with only 8Gb of RAM.