Combining low and high resolution DEMS with NODATA values

[ssalvador]

Hello,

I have loaded a set of low resolution adjacent real world DEMS into my Terragen model.  I also have a single high res DEM file consisting of several mining pits that I would like to incorporate into the model.  There are approximately 5 pits within the high res DEM, each separated by NODATA cells in between.  When I load the pits DEM into the model I was hoping the NODATA values would be ignored and only the real values within the boundaries of the pits would replace the existing low res terrain.  Instead, the NODATA values (-32767) are treated as real elevations and what I get is essentially a rectangular hole in my existing terrain.  What I was hoping for is for the "Replace NODATA values" to BE the low resolution values in between the pits.

I am currently evaluating TG3 to see if it could potentially replace my existing time-consuming methods used for visualizing future open pit mines.   If anyone can provide some techniques on how to accomplish what I'm trying to do I would most grateful.  I know I can simply blend the high res and low res terrains outside of Terragen but that means additional effort and time which I am trying to avoid.  If it's the only way to accomplish this goal then that's OK. 

Thanks

[bigben]

AFAIK this is not practical within TG in this situation as the boundaries between data and no data would have different elevations across the DEM.  You would need to merge the DEM data prior to loading it into TG.  I use GlobalMapper for this although it could also be done in QGIS. 

For TG, I'd export the merged data at the resolution of the low res data and then export smaller rectangular sections of the high res data at the higher resolution.  This keeps the file size down to a more manageable level.  In TG, then load the low res data first and then load the other sections.

If you're looking at doing a lot of this I'd seriously recommend looking at GlobalMapper or some other GIS tools as it makes the preparation of data very easy.  The demo version is fully functional although you can't save/export.

I can merge the data for you if you're having difficulty.

[bigben]

There may be a way... I'll have a play and get back.

[bigben]

This may work.

Replace no data values with -1 (too big a difference and you get spikes at the edge of the masked data)
Use the DEM shader node as a child layer to a surface, and create a mask using a Displacement shader to scalar and a Colour Adjust node.

The attached TGD has this set up. Just point the green TIF.Loader node at your file and set the lat/long at apex of the planet node to the middle of your DEM (to bring your DEM up to the top of the planet where the camera is)

[ssalvador]

Hi Bigben,

I downloaded your sample TGD and loaded my file as you suggested.  When I tried to load my low resolution terrain I couldn't get it to blend properly with my masked topography.  Your sample file doesn't include a low resolution terrain component so I had to wing it.  I don't have enough experience with Terragen to figure out to stack the masked terrain onto a low resolution background terrain.

Any further advice would be much appreciated.

Thanks

[bigben]

I thought I should have connected it in to demo as it's not entirely straightforward.  Turns out it didn't work anyway.  Try this one which has a low and high res DEM loader.  It works... kind of, but it's not really the best way of doing this.  Much better feathering the data in a GIS application prior to loading.

Sample screengrab shows 2 merged tifs with a large resolution difference (about 30x). Sharp drop off at points were elevation differs a lot (arrow) and streaky blending (curly bracket)  This is a limitation of masking the high res DEM rather than feathering along the edge of available data. I'll send you a PM with an upload link if you would like to send me a ZIP of the data to try and merge the data for you

[ssalvador]

Hi bigben,

I tried your latest TGD file with my data and it still doesn't produce the effect I'm after.  I have attached a few test DEMS representing my low and high resolution terrains.  The low res terrain is a plateau set at 1000m and the high res terrain is an open pit in the middle of the plateau.

The files are:

1.  test_terrains_global_mapper_3D.jpg - 2 terrains viewed in Globalmapper 3D mode.
2.  test_terrains_no_mask_3d.jpg - rendered image in TG3 with the low and high res DEMS loaded with no masking of NODATA cells.
3.  test_terrains_bigben_mask_3d.jpg - rendered image in TG3 with my low and high res DEMS loaded in test_terrains_bigben_mask.tgd
4.  test_lores_terrain.dem - low res plateau (1000m)
5.  test_pit.dem - high res pit (200 m to 1000 m )
6.  test_terrains_bigben_mask.tgd (originally mask_nodata.tgd)

Thanks very much for all your help

[bigben]

OK, so in GlobalMapper, all you'd need to do is export all of the available data as a geotiff... all merging done 
[attach=1]

Depending on where the pit is located, you could also load in real world data from online sources within GlobalMapper...    If your pit is really where the coordinates say it is, this is your pit on a global DEM I produced from a range of sources...
[attach=2]

It's too far north for SRTM data (90m/px) so this would be GMTED2010 data.  Attached your merged file here.

Masking 2 terrains in TG in this instance is not a good idea as your nodata regions are higher than your area of interest leading to the potential for spikes around the edge of the mask.

[bigben]

Add in a screen resolution export from the World Imagery layer in GlobalMapper: (TGD and image attached)

You could also dress it up with trees, or fill the pit with a lake etc.... depending on what you were wanting to demonstrate

[ssalvador]

Thank you very much bigben.  I appreciate all your efforts in helping me.  I probably should have stressed that the sample terrains I uploaded are completely artificial and don't represent any real world data.  I just wanted to provide something to illustrate the type of data thatI work with for my mining visualizations.  Ultimately, I would need to incorporate cut and filled road corridors that typically extend over long stretches of terrain.  I am convinced now that it's best to incorporate my engineered surfaces either as part of a larger rectangular region (DEM or geotifff without NODATA cells) and in the case of road corridors, smaller high resolution rectangles.

Thanks again.  I'm sure I will be posting more questions as I continue my evaluation of Terragen for use in my work. 

[bigben]

My typical workflow is to load all of the data into GlobalMapper. Export low res DEM to pad out the areas of interest and then export high res DEMs of the specific areas of interest. If there are obvious elevation differences between the high and low res data I feather the high res data using the bounds of the available data.

By having all of the data loaded at once, any nodata areas in your high res DEMs are filled with low res data. Cropping the high res tiles close to the high res data avoids unnecessary exporting of low res data at high res.  In TG, then, you just load in all of the DEMs from lowest to highest resolution without any need for masking or tweaking settings.

With 8Gb of RAM I've rendered 3Gb of DEM data without any problems (after a bit of a wait loading the data when opening the file)  On a 32Gb system I've rendered 12Gb of DEM data with ease (haven't tried more than that yet)

And check out Aerometrex's Youtube channel for TG animations.    https://www.youtube.com/user/Aerometrex