These TIFs appear blank, and have no profile when I import into photoshop. Both settings to apply to create/apply a profile are blank black images.
However, in affinity photo it does open, but is just all messed up and not even a heightmap.
What's going on here?
https://astrogeology.usgs.gov/search/map/Mars/Mars2020/JEZ_hirise_soc_006_DTM_MOLAtopography_DeltaGeoid_1m_Eqc_latTs0_lon0_blend40
The software mentioned on this page that was used to create it, appears to be removed from the website. Even searching it directly, every link goes to a 404: https://nsidc.org/search-results.html?q=velmap
Maybe check out this link for better tifs: https://planetarymaps.usgs.gov/mosaic/mars2020_trn/HiRISE/ (https://planetarymaps.usgs.gov/mosaic/mars2020_trn/HiRISE/)
That's just the raw files, such as uncomposited ones. That's where I actually originally downloaded the composite by file size.
I didn't have a closer look. Not my kind of game anyway.
Hmm seems it wouldn't matter for now for me. My buddy was able to send me a whopper of a converted tif at 4gb. But I cant manipulate them because alas, I don't have enough RAM. I have scratch set on my 1tb with plenty of room, but ram memory immediately gets filled regardless and system reduces to a crawl. Just pasting a duplicate layer is taking over 10 minutes and about to end task.
And to properly smooth for 32bit float I still need to upscale 100% and guassian blur and reduce back down in 32bit mode to get the bilinear smooth stepping of the terrain. :( Just handling single maps from these assets, or models in blender are painfully slow.
PS Terragen itself handles these large files amazingly, I must add. Its crazy how TG just spits out these huge asset scenes the way it does. Especially when I calculate a scenes assets at nearly double my available 30gb (after OS).
Not sure why a DEM would be all black. Normally high quality DEM's appear mostly white in Photoshop because they are 32-bit.
You would normally do an adjust Exposure, and scrub through the various exposure levels with the slider to get a preview of the different areas.
If you have a specific DEM you want me to look at I can load pretty much anything on my 128GB Threadripper box. Just email or PM me.
I have been doing a lot of work with DEM's lately and have had good luck with the 1/3 Arc Second stuff from USGS NED as Tiff.
The GeoTiffs have been improved in many cases over the old GridFloat DEM's.
Derek
The tif I was trying to get into PS is this one (beware it's 1.8gb): https://planetarymaps.usgs.gov/mosaic/mars2020_trn/HiRISE/JEZ_hirise_soc_006_DTM_MOLAtopography_DeltaGeoid_1m_Eqc_latTs0_lon0_blend40.tif
It's a HiRise DTM at 1m so should have good detail. But loading it directly in TG has issues, and aligning textures in TG would be a pain with rotation/scale, which is where I was trying to use Photoshop to quickly snap in the correct orientation and scale.
Quote from: SuddenPlanet on February 26, 2021, 08:49:21 PMscrub through the various exposure levels with the slider
I did that and briefly saw a glimpse of what it needed to look like, but slided past it and never found that particular setting again. So it's obviously a very defined small area that would show something. Automatic doesn't do a thing either.
Quote from: WAS on February 26, 2021, 11:06:30 PMThe tif I was trying to get into PS is this one (beware it's 1.8gb): https://planetarymaps.usgs.gov/mosaic/mars2020_trn/HiRISE/JEZ_hirise_soc_006_DTM_MOLAtopography_DeltaGeoid_1m_Eqc_latTs0_lon0_blend40.tif
It's a HiRise DTM at 1m so should have good detail. But loading it directly in TG has issues, and aligning textures in TG would be a pain with rotation/scale, which is where I was trying to use Photoshop to quickly snap in the correct orientation and scale.
Wow that web site has a super fast server/connection. I have never seen a near 2GB file download so fast!! I'll take a look and let ya know what I see.
Derek
I checked into that tif file and also went to the directory it is in and downloaded a bunch of other files. It seems the files that contain "DTM" in the file name are all black, and just contain a bunch of negative values.
There is a good amount of data in the file, since when I 7-zip compress it there is still 500MB of data left. For reference, I compressed a 32k 32-bit all black grayscale .psb image from 4GB down to 618KB.
It could be some tif format that needs to be read by some specific program.
Derek
If anyone wants a closer look and get this to work better in Terragen this is the scene file i made.
Very basic actually.
The real problem could be the Tif file itself maybe or our workflow. Or Terragen's geotiff loader etc. needs an update i don't know.
You can see the problematic borders.
I used the same Geotif file Jordan posted in the first post.
It loads into Terragen fine, but it's not very nice. Maybe at ground level you could do a few camera angles with it that might look ok.
I'll try doing a Heightfield generate from shader to extract the DEM into a more normal heightmap.
I tried some other things without success. So then used the most basic way by masking it.
You can see the geotif file in Affinity photo. I made a crude mask (shrinked and blurred the borders) based on that original and used that as the mask.
This worked. The problematic borders became smooth.
By the way if you use stepped in the heighfield node (Heightfield shader 01-displacement tab-interpolation method) the borders look good.
But the ground becomes yeah stepped. Oversmooth looks fine too. But i didn't compared it with Catmull-Rom. Might be enough too.
The image below is from the masked version. You don't need the "Heightfield clip vertical" node with this method.
Quote from: Kadri on February 27, 2021, 10:22:39 PMI tried some other things without success. So then used the most basic way by masking it.
You can see the geotif file in Affinity photo. I made a crude mask (shrinked and blurred the borders) based on that original and used that as the mask.
This worked. The problematic borders became smooth.
By the way if you use stepped in the heighfield node (Heightfield shader 01-displacement tab-interpolation method) the borders look good.
But the ground becomes yeah stepped. Oversmooth looks fine too. But i didn't compared it with Catmull-Rom. Might be enough too.
The image below is from the masked version. You don't need the "Heightfield clip vertical" node with this method.
I wonder if resaving that as a EXR since it's a heightfield would work. See the edge boundary issue is fixed here.
Quote from: WAS on February 28, 2021, 12:46:09 AMQuote from: Kadri on February 27, 2021, 10:22:39 PMI tried some other things without success. So then used the most basic way by masking it.
You can see the geotif file in Affinity photo. I made a crude mask (shrinked and blurred the borders) based on that original and used that as the mask.
This worked. The problematic borders became smooth.
By the way if you use stepped in the heighfield node (Heightfield shader 01-displacement tab-interpolation method) the borders look good.
But the ground becomes yeah stepped. Oversmooth looks fine too. But i didn't compared it with Catmull-Rom. Might be enough too.
The image below is from the masked version. You don't need the "Heightfield clip vertical" node with this method.
I wonder if resaving that as a EXR since it's a heightfield would work. See the edge boundary issue is fixed here.
Yes.
Just wanted to be sure and exported a high quality EXR file and got closer to see if there is any stepping or so.
The landmark is 2 m high. The size of the EXR file is 1 Gb.
The exported EXR file from the Heighfield generate node works so it's not a problem,
but i am curious if it is a 16, 24 or 32 bit file, as i saw different properties depending on which program i tested.
There was no option when exporting as EXR (for 16 or 32 bit). I searched the forum and wiki etc. but could not find this information.
I suspect 32 because it looks so clean. But seeing 24 looked strange:
Affinity : RGBA/32 (HDR)-srgb iec61966-2.1 (linear)
Krita : Unsupported colour space. Could not open.
Gimp : Grayscale 32-bit linear floating point.
XnView Classic : 24 bit RGB
That is a little strange. Perhaps it's the maximum capabilities of the viewers? Maybe reducing them down.
Yeah strange.
I tried with Hitfilm 3 pro it crashed :)
Vegas pro 14 could load it but in the properties i could only see this "22400x22488x128, EXR"...hmm...
Anyway no big deal.
Blackmagic Design Fusion also says it is a 32bit float exr.
Perhaps the exr settings on the Renderer tab are used by the exporter.
(Render - Sequence/Output - EXR Options...)
CHeers, Klaus
Probably.
And thanks for reminding me about Fusion Klaus. I was confused after the free version 9.
Looks like it is now together with Davinci resolve 17 one install and still free.
Klaus i tested with those render settings but the file looked the same.
I think the heighfield generate node saves always in 32 bit.
I just made a last test and made a standard render and saved it as a 16 bit EXR.
Affinity showed it still as a 32 bit file.
Xnview still shows it as a 24 bit file.
But Gimp showed it as 16 bit the right way.
Hard to trust such editors as this is what we are rely on them.
As these 16, 24, 32 bit things are already confusing enough especially for newcomers.
I think this has to do with the PixelFormat EXIF tag being saved and related viewers capabilities. If it can't read one format, It most likely must fall back to another format else fail to open it. For example, these are all the possible PixelFormat methods:
0x5 = Black & White
0x8 = 8-bit Gray
0x9 = 16-bit BGR555
0xa = 16-bit BGR565
0xb = 16-bit Gray
0xc = 24-bit BGR
0xd = 24-bit RGB
0xe = 32-bit BGR
0xf = 32-bit BGRA
0x10 = 32-bit PBGRA
0x11 = 32-bit Gray Float
0x12 = 48-bit RGB Fixed Point
0x13 = 32-bit BGR101010
0x15 = 48-bit RGB
0x16 = 64-bit RGBA
0x17 = 64-bit PRGBA
0x18 = 96-bit RGB Fixed Point
0x19 = 128-bit RGBA Float
0x1a = 128-bit PRGBA Float
0x1b = 128-bit RGB Float
0x1c = 32-bit CMYK
0x1d = 64-bit RGBA Fixed Point
0x1e = 128-bit RGBA Fixed Point
0x1f = 64-bit CMYK
0x20 = 24-bit 3 Channels
0x21 = 32-bit 4 Channels
0x22 = 40-bit 5 Channels
0x23 = 48-bit 6 Channels
0x24 = 56-bit 7 Channels
0x25 = 64-bit 8 Channels
0x26 = 48-bit 3 Channels
0x27 = 64-bit 4 Channels
0x28 = 80-bit 5 Channels
0x29 = 96-bit 6 Channels
0x2a = 112-bit 7 Channels
0x2b = 128-bit 8 Channels
0x2c = 40-bit CMYK Alpha
0x2d = 80-bit CMYK Alpha
0x2e = 32-bit 3 Channels Alpha
0x2f = 40-bit 4 Channels Alpha
0x30 = 48-bit 5 Channels Alpha
0x31 = 56-bit 6 Channels Alpha
0x32 = 64-bit 7 Channels Alpha
0x33 = 72-bit 8 Channels Alpha
0x34 = 64-bit 3 Channels Alpha
0x35 = 80-bit 4 Channels Alpha
0x36 = 96-bit 5 Channels Alpha
0x37 = 112-bit 6 Channels Alpha
0x38 = 128-bit 7 Channels Alpha
0x39 = 144-bit 8 Channels Alpha
0x3a = 64-bit RGBA Half
0x3b = 48-bit RGB Half
0x3d = 32-bit RGBE
0x3e = 16-bit Gray Half
0x3f = 32-bit Gray Fixed PointI haven't checked what PixelFormat Terragen uses, but I imagine it's something pretty high fidelity to it's source. And I do know there are all sorts of TIFF readers with all sorts of functionality, some more then others, and I think very few have "all capabilities" because it should only follow under scope of common use. Doing a search for C based readers, and I found tons, ranging in all sorts of character lengths showing a huge variance of code complexity/libraries used.
Yes i can kinda relate. There are tons of different files and extensions and what not probably. Still...
That is their stuff so to speak.
I pulled that tif into Global Mapper and was able to export it into something more readable in Photoshop. It turned out to be 16-bit though.
As you can see the elevation range is all negative and very deep, but only about 1026m of total range. That's probably why it was black in Photoshop and hard to find the sliver of data by scrubbing through exposure.
If anyone wants it let me know by Private Message.
Derek
I wonder if that's the exporter exporting 16-bit, or the tif being 16-bit? I actually haven't heard of any 32bit tifs from DEM/HiRISE but maybe I just haven't found them. Even looking online right now I can't find any but find tutorials on how to upscale/smooth and composite multiple TIFs to make a 32bit tif of 8/16-bit stuff.
I'd definitely be interested in it. I knew there was negative displacement that I wasn't seeing so I actually made a inverted disp map clamped to the flows and lake to push it down a bit in my scene.
I'll take a copy if you don't mind, may save me some time. I'm not sure how big it is maybe my gmail will take it?
I found a link on NASA`s website that points to the original colour images used for the map.
"This map is composed of two layers: a grayscale Jezero Crater map, and a true-color base map. The greyscale base map was created with images from the HiRISE camera on NASA's Mars Reconnaissance Orbiter, while the color base map is from the European Space Agency Mars Express High Resolution Stereo camera. Some color processing has been applied to highlight surface features. The original image can be found here. A high-resolution Digital Elevation Model was created from the images to provide critical information for rover drivers, who need to know how steep the hills are as they plan a path forward through this rocky terrain."
https://maps.planet.fu-berlin.de/#map=3/0/0
You can zoom in quite high. Nice.
CHeers, Klaus
Something else...
I downloaded to larger images:
"JEZ_hirise_soc_006_orthoMosaic_25cm_Eqc_latTs0_lon0_first.tif" which is about 7GB and
"JEZ_hirise_soc_007_orthoMosaic_25cm_Ortho_blend120.tif" which is about 3.5GB.
When I try to load those into any of the image loader nodes in TG I get an error:
Cannot locate file: blablabla
Any idea why that is, anyone?
CHeers, Klaus
PS: after a some more testing I think the files are corrupt. So, never mind.
Klaus i only tried the "JEZ_hirise_soc_006_orthoMosaic_25cm_Eqc_latTs0_lon0_first.tif"
file which is about 7 GB in Terragen too and got the same result as you.
But the file looks ok when you open it in QGIS. Terragen doesn't like the file or maybe does have a file size limit etc. i don't know.
QGIS looks like a very capable package and is free to use open source. There are many options to export the file.
But as i don't work with heigfields much, the options are a bit overwhelming:
https://qgis.org/en/site/
This example is from the nearly 10 gb tif file ( JEZ_hirise_soc_006_orthoMosaic_25cm_Eqc_latTs0_lon0_first_dd.tif ).
There is actually quite a detail in there as it looks. I tried to export just that area.
My imported cropped area render is bad. Someone who knows what he is doing can do much better then this.
I like OSGeo4W (https://trac.osgeo.org/osgeo4w/) which includes QGIS, and some other stuff. Made it a bit easier getting everything installed.
Quote from: undefinedOSGeo4W is a binary distribution of a broad set of open source geospatial software for Windows environments (Windows 10 down to XP). OSGeo4W includes QGIS, GDAL/OGR, GRASS as well as many other packages (over 150).
https://trac.osgeo.org/osgeo4w/
Thanks for the link.
Quote from: KadriMy imported cropped area render is bad. Someone who knows what he is doing can do much better then this.
I think its cause that is a shaded relief from true color, so it is including shadows. I think they are created using heightmaps, but then they mix with a texture so you can see the details in the relief. It may be possible to level the shadows out to a degree and blur a bit, but you'd loose close up details. Hmm.
Quote from: WAS on March 01, 2021, 03:26:52 PMQuote from: KadriMy imported cropped area render is bad. Someone who knows what he is doing can do much better then this.
I think its cause that is a shaded relief from true color, so it is including shadows. I think they are created using heightmaps, but then they mix with a texture so you can see the details in the relief. It may be possible to level the shadows out to a degree and blur a bit, but you'd loose close up details. Hmm.
Yes i looked a little closer and real heigfields are different of course.
That tif isn't suited for a direct render as you said.
Weirdly when I tried exporting the 1.8 GB elevation file from QGIS, it just wants to export a tiny image. I give up for now. Lol
I think i used the "Map canvas extend" then locked the output width and height and then used something over 2000 for resolution.
But this software needs more attention for sure to get what you want :)
I doubt that i will have a use more then these tests as i am not into heigthfields.
The total time i tried these days is probably more then i tried in the last 20 years :)
Thanks
@Kadri and @Sudden Planet.
Then I`ll try QGIS and OSGeo4W before I delete the files.
CHeers, Klaus