working in a 32-bit DEM

Started by Dune, August 23, 2019, 01:49:48 am

Previous topic - Next topic

Oshyan

Quote from: digitalguru on August 23, 2019, 01:08:00 pmOnly because you're seeing values over 1, just add an Exposure Adjustment layer and drag the exposure down and you can non destructively see your terrain. You can then work in PS with the file, though it has to be said PS is still a bit crap at editing 32bit, and your options are limited.
I just want to highlight this quote because as far as I am aware it is *the* fix that's necessary for minimal disruption of workflow. No need to switch to EXR, etc., the data is probably already there in the file it just needs to be adjusted for *display*. So using an adjustment layer allows you to *temporarily* adjust the exposure so you can see what you're doing. Give it a try and let us know if it works on these files.


- Oshyan

Dune

Thanks guys. Especially indeed that advice from digitalguru sounds like the way to go.

WAS

Really though, and why the tool exists... if you're editing DEM data, usually USGS, you should just convert to a normal float TIFF.

This topic was about working in a 32-bit DEM. Obviously just converting to a conventional floating point would be the easily to handle, across applications.

Super simple.
Check out my Terragen Discord: https://discord.gg/Vy5FRTE

Oshyan

In most cases they are basically float TIFFs. The issue lies in how the how the values are represented (i.e. what "scale" or unit and how that translates to numerical values), and then how your image editor interprets that for display. Height data represented as actual meter values will be interpreted very oddly by an image editor that is expecting to see high dynamic range float image data from an HDR capture or something. For example it's the difference between a value of say 567 (for meters) vs. a value of 0.23092035898923 (for a single channel color value, 0-1). The application expects a 0-1 value for a floating point image but sees 567, well, it's going to show that as pure white.

Converting to a different format won't help either, unless that format also applies some kind of data/image transform, i.e. remaps the data, or if the viewing/editing application for some reason interprets and displays that format differently. And only certain limited kinds of transforms will maintain the correct characteristics of the height values, so you ideally want to do that non-destructively, which is what the Photoshop Adjustment Layers do. 

Using an Adjustment Layer will probably allow you to see the data more clearly, if not entirely "correctly". The limitations one may then run into are due to Photoshop's limited 32 bit-capable toolset, which would be an issue no matter the format, as long as the data were 32bit/channel. Depending on the level of editing needed, the available tools may be sufficient (you *can* at least paint). If not, GIMP has a version that apparently can work more comprehensively with 32 bit data, so that's something to consider. Or more dedicated heightfield editors, of which there are few that have painting capabilities that are reasonably sophisticated. Leveller was one such option, I believe.

- Oshyan

digitalguru

August 24, 2019, 07:51:24 am #19 Last Edit: August 24, 2019, 08:18:33 am by digitalguru
I did a bit of an interweb search and found this program: Affinity Photo
https://affinity.serif.com/en-gb/photo/

Downloaded a 10 day trial, and on playing with it for half an hour am very impressed :)

Loaded up a 32-bit OpenExr and seems to edit with a full range of tools, in particular Dodge and Burn work (which don't in Photoshop) which is what you want if you need to make height adjustments.

It even has a handy 32-bit preview tab which can adjust the exposure so you can see your terrain data easily.

The only caveat (on a quick play with it) is that it doesn't like 32 bit tiffs (has only 16 bit options) so you'd have to convert to OpenExr with the Photoshop method  mentioned earlier or an export out from Terragen. Saves out to full 32-bit OpenExr and loads in TG just fine.

Reasonably priced too, one time purchase for about the cost of two months Adobe subscription.

Edit:
Did a quick screencap:
https://www.dropbox.com/s/kt1g4jsw6wdjuqr/affinity_01.mp4?dl=0

Oshyan

Ah nice! I was aware of Affinity Photo just as a photographer, and that it had some notable improvements over Photoshop. But I hadn't thought of it for this kind of utility. You're right that the toolset seems much more applicable to terrain editing. The limitation with 32 bit TIFFs is odd though. I wonder if that could be reported as a bug or opened as a feature request with them...

- Oshyan

digitalguru

August 24, 2019, 01:21:42 pm #21 Last Edit: August 24, 2019, 01:52:45 pm by digitalguru
Yes, it's a nice find :) Still have to play around with it, but it feels very usable and seems to share a lot of similarities with Photoshop (hope Adobe don't think of taking any action ), but is a lot less fussy  For most of the work I'd usually do in Photoshop, it could be a good successor. In fact I only have used Photoshop and Premiere CS6 and I've switched over to Davinci Resolve for editing/grading, so maybe I can be Adobe free soon!

Quote from: Oshyan on August 24, 2019, 12:33:24 pmThe limitation with 32 bit Tiffs is odd though. I wonder if that could be reported as a bug or opened as a feature request with them...
Agreed, it's not a bug though, the export options for tif only go up to 16bit.  I remember years ago when we rendered out to tifs! But OpenExr has more or less taken over especially in regards to 32bit output.

p.s. I forgot about Picturenaut for conversions, it's been around for a while and hasn't been updated for years, but it's a very lightweight and free prog and will easily do any 32bit tif/hdr/exr conversions:
http://www.hdrlabs.com/picturenaut/

Dune

Thanks a lot, digitalguru. Affinity is something to keep in mind, though this time I will probably get away with some basic handling in Photoshop with your adjustment layer idea.

Dune

Another question regarding DEM. I have cut out a crop of a global DEM, but that is of course an equirectangular image, spanning the whole globe. So the crop is stretched somehow, which I noticed when overlaying stitched satelite images from the web. The crop is near Sinai, so pretty much near the equator, which makes it easier to de-stretch it, but do any of you have any experience with the amount? Looks like 20% width decrease will do, but I'm sure there is a mathematical calculation under the equirectangularity.

Matt

You could use the georeferencing feature to map it to lat-long coordinates. Georeferencing will automatically undo any distortion you see, as long as you input coordinates that are reasonably accurate for the corners of the crop.
Just because milk is white doesn't mean that clouds are made of milk.

Dune

Thanks Matt, sounds good, and I'll certainly keep that in mind, now need to find the coordinates. In the meantime I think another solution would be to project a 'normal' satelite image and squeeze the dem and stretched satelite image over that until it sort of fits. Latter is probably easier as I located the Sinai on the North Pole for easier working.

Matt

Working at the top of the planet shouldn't be a reason to avoid georefencing. Georeferenced DEMs and satellite images are repositioned to the origin in a typical Terragen georeferenced workflow. You just need to enter a center point into the Planet's "lat long at apex" parameter.

https://planetside.co.uk/wiki/index.php?title=Geog_Heightfield_Load
Just because milk is white doesn't mean that clouds are made of milk.

Dune