Hello
Today I read RArchers tut on how to use satellite data in TG2 I was surprised that both the tut and TG2 are easy to use in this regard. Before reading the tut I just glanced through it and it looked really complex. But not so. The problems I ran into is with the websites for getting the data in the first place.
http://nationalmap.gov/viewer.html
http://earthexplorer.usgs.gov/
I find this use of TG2 very interesting, and want to put the methods in my "tool box". But I need some help with the sites.
With the USGS, the problem is the web sites have changed dramatically from the time RArcher made the tuts. And perhaps the data formats too?
I was able to get the site to generate a download to use, but when I un-ziped the data there was nothing in it TG could open.
Could some one who is familiar with the USGS site and TG2 please talk me through the changes in the download process from the tut?
http://www.archer-designs.com/tutorials/usgs-elevaton-data-with-terragen-2/
earthexplorer is I think a much easier site to navigate, and to select the area of interest. But I could not figure out what files to take. I tried "Elevation" and "DEM" type. And I got a .tiff in the download. But the image was 100% black. Also I don't think that the file was of the area I tried to select from the map. TG2 did open the files though.
Can someone check out the site and talk me through the correct process for getting the right files, of the desired area, for use in TG2?
I know there has been other disscusion on this general subject. If I have asked questions that have already been answered, could you please post a link.
Thank you much.
Hmm, the process seems relatively simple but it has changed since Ryan's tutorial. Here are the steps I just took to download a section of the Grand Canyon.
0/ Create an account (free)
1/ Clicked on your Link http://earthexplorer.usgs.gov/ (http://earthexplorer.usgs.gov/)
2/ Typed in Grand Canyon in the Address Coordinates box and pressed "show"
3/ This reveals a link just bellow that when clicked places a Google pin on the map.
4/ Zoom into the area indicated. (+- buttons just like google maps)
5/ Press Clear coordinates and use the Cross hair cursor to draw out a rough box about the Grand Canyon area I want to examine. This will show up as a red box.
6/ At the bottom of the screen on the left press the button "Data Sets"
7/ From the drop sown menu select Digital Elevation and then "Aster Global Dem"
8/ At the bottom of the page click on "Results"
9/ You will now be presented with a visual list showing the products that fall within your search criteria. Mouse over the row of 6 icons top see their meanings.
10/ Click on the download link and agree to the t&c.
11/ Download the zipped file.
Here's the interesting bit.
If you open the downloaded geotiff (it's got a normal tif extension) in Photoshop it will likely be plane black or white. You'll also notice it's a 16bit image. To see any detail play with the Levels and Exposure settings. I would advise doing as little as possible in Photoshop as when/if you save the file you can loose some of the meta data. If however you have Global Mapper you can see the file properly and combine it with other elevation maps and crop it. There's a free demo on the Global Mapper site.
When you open the file ion TG use the "Heightfield load" node and after it place a "re-size terrain" to your desired size (in metres)
Hope this helps
Richard
I'll see what I can do to get the tutorial updated to reflect the new website in the next few days. This is the site I would use to download stuff now: http://viewer.nationalmap.gov/viewer/
For the elevation data in the US you want to either get the NED 1/3 data or if available the NED 1/9 data.
@Richard,
I did do it right the first time,thank you for confirming the steps. I just could not be sure.
But I would probably never guessed to play with the resulting file in photoshop. That bit you wrote was very helpful! 8)
I got the file into TG and can now take a look at how all this works!
@RArcher,
Thanks for making the tut! It was much easier to follow than I thought it was going to be.
Can you tell me why the USGS site you recommend is better than the nasa site discussed in this thread? Is the data or resulting file different? I did notice that the link you posted here has less restrictions on the data than the earthexplorer site.
From Earth Explorer you can get Aster Global DEM, GMTED2010, and SRTM Elevation Data. These are all global DEM data sets. The data you can get from http://viewer.nationalmap.gov/viewer/ is far more detailed for the United States with the resolution getting down to 1/9 arc seconds as opposed to the SRTM data which is at either 1 arc second (approx 30m/pixel) for the US or 3 arc seconds (approx 90m/pixel) worldwide.
Hello.
After reading RArchers explanation of why the one site is better than the other I decided to try the recommended service again.
The problem I again found is that the data I get from the download looks like it is not usable in TG.
Here is a screen grab of the first set of selections. You can see what I clicked.
[attach=1]
Here is the next page
[attach=2]
The resulting download was 481.1MB
The file contained the following file types;
.htm, .txt, .dbf, .shp, .prj, .shx, .doc, .dat, .nit, and some others.
Clearly at 481MB I got some data. but not the right stuff as far as I know. I'm obviously making the wrong selections, but what are the right ones?
Sorry to be so simple, but this stuff is as new to me as a new born baby.
Thanks guys.
If the zip file you downloaded did not contain a .tif file then make sure you download the selection that is not prepackaged for ArcGrid or Float. I just downloaded a small tile and it worked as expected.
Quote from: RArcher on August 10, 2012, 06:06:13 PM
If the zip file you downloaded did not contain a .tif file then make sure you download the selection that is not prepackaged for ArcGrid or Float...
Thank you.
Cyphyr
QuoteWhen you open the file ion TG use the "Heightfield load" node and after it place a "re-size terrain" to your desired size (in metres)
I thought I understood this when I first read it but...
You are saying that the .tiff from the download does not come into TG2 at the right size, yes?
After creating the "Resize Node" I could not be sure where to plug it in. I Tried every input option. Some would not except the node. And where I could get it plugged in, it made no effect regardless of the values I entered.
Can you tell me what I'm doing wrong?
Try this (see attached) sorry not in front of main pc
Terragen will assume 1 pixel equals 1 metre unless otherwise specified.
Richard
If you are using real world data then there is no reason to resize anything. That sort of goes against the whole reason of using real world data in the first place. As long as you are using properly georeferenced data then it will be imported at the correct size and height range.
I know it should work but often I have found, especially after combining/exporting from Global Mapper that the terrains come in way too small. For example I just downloaded a random AsterDEM (ASTGTM2_N44W101_dem.tif) which is an 80km square. On loading into TG (after exporting from global mapper) it loads as an 3600m x 3600m square.
I had to take it through Global mapper because the Terragen says on loading "Please convert to SGI for non standard Bit depths". Maybe I'm loosing some meta data in the conversion although GM still loads both old and new files at the same size.
Agreed if your data is PROPERLY geo- referenced then this "should" not be an issue.
Cheers
Richard
RArcher, I cannot access the website. Is the URL correct?
As of right now, all of the links on this page are working for me.
I have to say that I have had to re-size everything too. I am going to look and see if global mapper is available on mac. In the end I like being able to abstract the maps. But I would really like to see them come into TG correctly first.
By the way guys, thanks for helping to understand all of this. Its really very nice of both of you! :)
*EDIT*
Just checked. global mapper is not on mac, and is not currently free. Though there is a trial version.
Hi Richard,
Quote from: cyphyr on August 11, 2012, 01:26:35 PM
I know it should work but often I have found, especially after combining/exporting from Global Mapper that the terrains come in way too small. For example I just downloaded a random AsterDEM (ASTGTM2_N44W101_dem.tif) which is an 80km square. On loading into TG (after exporting from global mapper) it loads as an 3600m x 3600m square.
Looking at your screenshot the terrain isn't georeferenced. TG2 reads the georeferencing for TIFFs from a .tfw file with the same name as the .tif and in the same folder.
Regards,
Jo
Hi Jo
Yes, of course you're right. I just re-ran my test with the same file. It worked fine however the tif had to be run through Global Mapper to generate the tfw file as this wasn't included in the download from the USGS site.
As I mentioned earlier:
Quote"I had to take it through Global mapper because the Terragen says on loading "Please convert to SGI for non standard Bit depths". Maybe I'm loosing some meta data in the conversion although GM still loads both old and new files at the same size."
The geo-referencing data must be within the geotiff file somewhere, in the header I assume, but it's odd that my exported geotiff is nearly twice the size of the original but lacks this geo-referencing data which has to be supplied via a separate file. (It's twice the size as the file has been converted from 16bit to 32 bit in GM).
It would certainly be helpful if TG could read "non standard Bit depths" without having to convert the file via another program especially since GM is relatively expensive and the SGI plugin for Photoshop is not readily available (well I haven't been able to find it or when I have, get it to work).
:)
Richard
Hmmmmm :-\
When I downloaded from http://viewer.nationalmap.gov/viewer/ my resulting .tiff opened in Photoshop at 32bit. The file I got from the other source was opened at 16bit.
But I must convert to 8bit.
1) What if any value was there to the file being 32bit in the first place (from a TG2 perspective)?
2) Does 32bit help to preserve detail against loss when adjusting levels and exposure in PS, again from a TG2 perspective?
3) Does it matter if I convert "mode" before or after making adjustments? Normally I would say yes! But with this stuff, I have no idea.
4) Should I treat the data in photoshop just like a photo?.. I have to say, It does not feel like I should.
5) Please someone clarify. I should always have a .tfw file in my data download, or I must always/sometimes make one?
On a related subject. There was a video posted in image sharing a few months (or so) ago. Where the video was made entirely from satellite data. I remember that it included a view of a construction site that would have been a long nightmare to do manually/procedurally. Does anyone recall this video? It was posted by someone who does not post often, it may have been his first post?
In the light of this thread, I really want to see that video again. But I cant find it. Hope someone reading this can remember it?!!
That's crazy. Both IE and Firefox state that the viewer.nationalmap.gov server does not exist. Did you guys install something in order to see this site?
Hmm I cant get the viewer page to load properly either. It just says loading just below a quick tip box.
Badger, why do you want to convert to 8 bit? You'll end up with only 256 levels of grey and get very obvious "stepping" in your terrain. Although you will end up with an easily editable image you'll loose the fine resolution. As I mentioned above be very careful using Photoshop to edit GeoTiffs, they are NOT to be treated as a photo/image, the height data is only "represented" by greyscale values.
Richard
Quote"Please convert to SGI for non standard Bit depths"
Did I misunderstand this?
Quote from: TheBadger on August 12, 2012, 10:00:36 AM
Hmmmmm :-\
When I downloaded from http://viewer.nationalmap.gov/viewer/ my resulting .tiff opened in Photoshop at 32bit. The file I got from the other source was opened at 16bit.
But I must convert to 8bit.
I wouldn't do anything with it in Photoshop if you're using it as a terrain file. It's just elevation data wrapped up in an image format.
On a related note, I still prefer to export as TER mainly because I change the projection from geographic to UTM
QuoteI wouldn't do anything with it in Photoshop if you're using it as a terrain file. It's just elevation data wrapped up in an image format.
HI Ben, thanks.
But you confused me on this again. If I cant use photoshop, or shouldn't. How do I manage the file? Or does this ".tfw" file control the data, so that I don't need to do anything to the file unless I want to distort it?
After being informed of this file extension .tfw I looked and sure enough it came down with the .tiff. So I just need the two files and the rest I can trash? Or do I need to keep all of the files that come in the downloads in a library?
The tfw file is just a text file. Open it in a text editor and you'll see that it defines the boundaries of the data. A proper geotiff file should still import OK into TG without a tfw, it's just that TG won't know how big/where to make the terrain.... and extrapolating from that, if you know the correct values to put in, you could create the tfw yourself.
It has been a while since this thread was active but it started out right on topic with my question. I am using the relatively new http://viewer.nationalmap.gov/viewer/ as my source of DEM. The problem is that the GeoTIFF file does not end up containing a .tif file. I have tried the only two sub-formats I have seen provided: ArcGrid or float and neither results in .tif file in the zip. Any help would be appreciated.
There's a note on the USGS site that reads as follows:
QuoteNote: As part of the National Geospatial Program's geospatial data delivery plan for delivering data from a public internet cloud provider by the end of 2013, The National Map is preparing prepackaged, staged products for faster and more efficient downloads. Beginning December 18, 2012, NED 1- and 1/3-arc-second data will be available through prepackaged, downloadable, 1- degree tiles in ArcGrid and GridFloat formats only. Users can still define custom areas of interest. However, all tiles intersecting the area of interest will be included in the delivery.
In addition, beginning with the April 2013 release of the NED, it is anticipated that all elevation products (1-, 1/3- and 1/9-arc-second DEMs) will be available as prepackaged, downloadable files in .img format, a robust format easily read by nearly all GIS software. Due to 1/9 arc-second data's very large size, it will only be available as prepackaged data in '.img' format through online download starting with the April NED release. And while USGS continues to offer ArcGrid and GridFloat formats through our bulk order process, all other format options for on-line download of 1/9 arc-second DEMS will no longer be available after the move to staged products is completed.
We are considering offering a second prepackaged format for 1/9 arc-second data if budgets allow. We will be posting more information about the change to prepackaged products soon. If you have questions or concerns about these changes, please contact us at tnm_help@usgs.gov.
So unfortunately it looks like much of the data is no longer available in GeoTIFF. Terragen will support more DEM formats in the future. In the meantime you'd have to convert, but there are some free tools available that can do this (e.g. Landserf).
- Oshyan
Rats!
Well Oshyan, please don't drop support for tiff import. I like to create my own in Photoshop for carving rivers into DEM tiffs. Sortof like Terragen classic. I have to work in a very narrow band of shades when I "carve" the land by using a brush tool.
1 pixel = 1 meter This I did not know, very useful info. Thanks.
I'm having some dialog with a friend at the University of Alaska about lidar data and he says he can probably help me with getting high resolution DEMs as tifs that I can play around with. Things are changing at his shop too. I'll try to make sense of it and share any good news.
Russ
We won't be dropping support for any formats, just adding more. :)
- Oshyan
The USGS has updated the site now to include a link to scripts that will facilitate conversion to the legacy formats including GeoTiff.
>>>
In addition, beginning with the April 2013 release of the NED, it is anticipated that all elevation products (1-, 1/3- and 1/9-arc-second DEMs) will be available as prepackaged, downloadable files in '.img' format, a robust format easily read by nearly all GIS software. Due to 1/9 arc-second data's very large size, it will only be available as prepackaged data in '.img' format through online download starting with the April NED release. And while USGS continues to offer ArcGrid and GridFloat formats through our bulk order process, all other format options for on-line download of 1/9 arc-second DEMS will no longer be available after the move to staged products is completed. To assist users in this transition to our new product format offerings, we are also providing downloadable file conversion scripts to assist users that need the data in other formats.
>>>
The link is http://imsdemo.cr.usgs.gov/Website/Ned_Conversion_Scripts.zip and uses the GDAL tools from http://www.gdal.org/ which are open source.