Hey all,
I have 2 issues today.
1) Having an issue with an sgi file I'm using to create much of the displacement in my scene. It's a 16-bit sgi file, 4096x4096.
The scene loads, and comes up with an error that says 'ReadImage: Unable to load image: *image location*' This goes away if I go into my image map shader, select the path, alter the name of the file then change it back again. Then the sgi loads and weee! Realistic terrain ahoy.
Small problem is that this isn't a solution once it goes onto our render farm as the nodes pick the file up, open it up, happily ignore the sgi file and render the frames minus displacement.
Is there a work around for this issue? I need it to be 16-bit I was getting stepping on 8. It needs to be this large in pixels as it's a 40km wide shot and lower resolutions weren't introducing the detail needed. Any other formats are requesting to be converted to sgi and a cursory look around the forum suggests this is a normal thing.
2) When exporting my camera data, and indeed any objects out of softimage into terragen, they come in at the correct location, move the correct way and so on, but the terrain is lower than my terrain in softimage. I've checked this using stills of my soft scene and tg2 scene with test objects in place. The objects line up... the landscape doesn't. I've tried many times today exporting the terrain out of TG2 and seeing if it magically lowers itself in Softimage, but to no avail.
I'm thinking this might be to do with the terrain generate node I'm using to get the terrain out of TG2. I'm saving it out as a .ter file, then importing that into Softimage using the import elevation dataset plugin.
All my displacement occurs before the heightfield generate node, but somehow it comes into Softimage slightly taller than I'd like.
If there's no solution to this, I may have to fake it by scaling the object down, but I'd prefer it to be accurate. I don't want a hint of floating between softimage and the tg2 content.
I've attached the tgd, even though it refers to multiple images here that are way too large to upload.
Anybody got any clues how i can resolve my loading issue or heightfield issue?
Hi. I notice that the .sgi image map shader seems to be the only one without the 'textures/' location prefix in the filename.
Is it still in the same location as the .tgd, like I assume it was when the project was saved(otherwise the entire filepath would be visible from the .tgd location onwards)?
I just tested with an image map shader here and, as with an object, image maps only seem to require the filename as long as they are in the same directory as the .tgd. Perhaps you've moved the image to a new location and it's not in the same directory as the project file anymore? Just a thought.
*I didn't use an .sgi image to test the IMS', maybe there's a problem with that type of file? The rest of yours appear to be .tif, have you tried converting the .sgi to another format? Otherwise, sorry, that's all I can think of at the moment.
I moved the sgi out of the textures folder as it was the only problem child. Was just testing if the farm machines preferred it that way or something.
SGI is the only 16-bit format that TG2 will let me use for displacement. Otherwise it brings up the error mentioned in my post.
Turns out the .ter is exported as though the landscape is flat, whereas I had my TG2 heightfield set as though curving over the planet. Damn. Have fixed to flatten surface first.
Temporarily using a 8-bit SGI for now. Must've got something wrong before as no longer getting stepping in my displacement now.
So now stuff lines up a heck of a lot better AND loads on the render farm. Just got a detail issue in my mesh. Will resolve it Monday.
Happy beer everybody!
The Image Map Shader and the Heightfield Load can also read EXR files (I think both 16 and 32-bit), and 16-bit TIFF files.
Quote from: Matt on March 26, 2011, 01:58:44 AM
The Image Map Shader and the Heightfield Load can also read EXR files (I think both 16 and 32-bit), and 16-bit TIFF files.
I'm having issues trying to load 32-bit Terrains in EXR and TIFF format. Both formats load in and display the heighmap in the Shader Preview, but the displacement doesn't happen in the 3D Window (I just get a flat Square). If I crunch the image down to 16-bit, save as Tiff in Photoshop and load it then it does work. I can't save 16-bit EXRs from PS so can't test that.
Mainly I would like to use the 32-bit EXR exports from Gaea as heightfields in Terragen without reducing them down to 16-bit.
Any ideas?
Derek
OK I figured out a work around. For some reason for 32-bit EXR files to work you have to increase the displacement Height Multiplier to a whopping 1000. :)
The Displacement of EXR (and any incoming file without georef/height information) is assumed to be 0-1, in meters.
- Oshyan
Quote from: Oshyan on April 13, 2018, 08:52:09 PM
The Displacement of EXR (and any incoming file without georef/height information) is assumed to be 0-1, in meters.
- Oshyan
Actually it simply reads the pixel value in the EXR and that is the height in meters. The range you get will depend on what the exporting software writes into the image. Terragen's choices might be different from those of Gaea. I would encourage Dax to consider using the same mapping that Terragen does (pixel value = height in meters), because it eliminates the need to manually enter a scaling factor on a case by case basis.
Matt
Oh right, my bad. And yes, there is an argument to be made for the scaling to work the way Terragen assumes it to, since EXR otherwise has no scaling info in it.
- Oshyan
Curious what the manual information of this is. What is the contrast scale of a pixel to meter ratio, in general for Terragen in the event of other formatted image maps.
Not sure I understand your meaning, but in EXR you can literally encode numeric values, like 1000 (which Terragen would interpret as 1000 meters, I believe). So when TG exports an EXR, it will record values into it that correspond to the height of the exported terrain in meters. When it imports, it interprets incoming data in the same way.
- Oshyan
Oh I see. That's handy. What I mean is you can supply other image formats, where TG interprets colour contrast as displacement, but the multiplier must usually be upped. Wondering what the actual scale is for a pixel with no information based on it's black to white contrast.
For example using the same style color picker in PS I can clamp a images contrast per pixel.
Quotewhere TG interprets colour contrast as displacement, but the multiplier must usually be upped.
I think I see what you mean (but correct me if I'm wrong :))
It sounds like you're looking at a map that may be normalised - that is, all the values are between 0 and 1 and it's only the contrast between those values that give you a sense of the displacement from the original terrain. You then need to add a multiplier to "expand" those values so the displaced terrain matches the original. This is very arbitrary though, and prone to errors.
This is why OpenExr is the way to go when using images for displacement and height-fields - the pixel value corresponds directly to height of displacement - a pixel value of 150.675 will displace to 150.675 metres in Terragen. Though a common mistake is to not match the area covered by the displacement image - that is. if your displacement map is generated from a terrain 1000 metres square, then it must be mapped to the same size area in Terragen (or whatever software) for the ratio of height to width to be correct.
Another reason why OpenExr is a good idea to use is that, in the main, it is saved as linear data - most software (can't speak for Softimage, though I would hope it would) would read and write an OpenExr and not apply any gamma correction to it - if it did, your height data will change - it must be interpreted linearly.
Quote from: digitalguru on April 17, 2018, 07:41:23 AM
Quotewhere TG interprets colour contrast as displacement, but the multiplier must usually be upped.
I think I see what you mean (but correct me if I'm wrong :))
It sounds like you're looking at a map that may be normalised - that is, all the values are between 0 and 1 and it's only the contrast between those values that give you a sense of the displacement from the original terrain. You then need to add a multiplier to "expand" those values so the displaced terrain matches the original. This is very arbitrary though, and prone to errors.
This is why OpenExr is the way to go when using images for displacement and height-fields - the pixel value corresponds directly to height of displacement - a pixel value of 150.675 will displace to 150.675 metres in Terragen. Though a common mistake is to not match the area covered by the displacement image - that is. if your displacement map is generated from a terrain 1000 metres square, then it must be mapped to the same size area in Terragen (or whatever software) for the ratio of height to width to be correct.
Another reason why OpenExr is a good idea to use is that, in the main, it is saved as linear data - most software (can't speak for Softimage, though I would hope it would) would read and write an OpenExr and not apply any gamma correction to it - if it did, your height data will change - it must be interpreted linearly.
Photoshop can't read EXR data appropriately without a plugin, that's for sure, though I did find this that can both read, and write OpenEXR format all in PS.
https://www.exr-io.com/
Exr-IO appears to have a 2GB file size limit. :(
Didn't catch that. :( I wonder if it's a 32-bit limitation? A lot of plugins target 32bit Photoshp... for whatever reason.
No, I don't think it has anything to do with EXR being 32-Bit. Different EXR readers/writers have different limitations. For example, Teragen can output EXR's that are HUGE! But other programs can't always deal with them. Like Photoshop for example; it can load huge 4GB EXR files, but it can't save them with the standard (built-in) OpenEXR reader/writer. There is Exr-IO, and also ProEXR, and others I imagine. Programmers here on the forum might have more insight into why some formats have file size limitations. Tiff also has a 4GB file size limit. Recently I have been just using Photoshop PSB for everything I work on that is huge (It's 32-bit RGB and you can save files that are hundreds of GB).
Quote from: WASasquatch on April 21, 2018, 02:08:14 AM
Didn't catch that. :( I wonder if it's a 32-bit limitation? A lot of plugins target 32bit Photoshp... for whatever reason.
Actually I got a reply from the company that developed Exr-IO, and they said the 2GB limitation is a Photoshop limitation.
Quote from: D.A. Bentley on April 26, 2018, 09:03:56 PM
Quote from: WASasquatch on April 21, 2018, 02:08:14 AM
Didn't catch that. :( I wonder if it's a 32-bit limitation? A lot of plugins target 32bit Photoshp... for whatever reason.
Actually I got a reply from the company that developed Exr-IO, and they said the 2GB limitation is a Photoshop limitation.
maybe try it in IrfanView...it's free and there may be a plugin.
Just checked and the is an EXR plugin, would that work. It's a 64 bit program, or 32; but if you run 64 use that one.
https://www.irfanview.com/64bit.htm
Quote from: Matt on April 14, 2018, 08:59:05 PM
Quote from: Oshyan on April 13, 2018, 08:52:09 PM
The Displacement of EXR (and any incoming file without georef/height information) is assumed to be 0-1, in meters.
- Oshyan
Actually it simply reads the pixel value in the EXR and that is the height in meters. The range you get will depend on what the exporting software writes into the image. Terragen's choices might be different from those of Gaea. I would encourage Dax to consider using the same mapping that Terragen does (pixel value = height in meters), because it eliminates the need to manually enter a scaling factor on a case by case basis.
Matt
32bit EXR files handle color depth and contrast differently than 16bit, probably where the changes are from. I'm not entirely sure why you'd need a 32bit EXR for Terragen, honestly. The effects where you'd assume artifacts may show up are pretty much useless to TG.
Yes, 16-bit integer is different from 16-bit float. 16-bit integer (as in TER and 16-bit TIFF) is usually enough for most terrain needs, as long as the data is normalized to fill the space from black to white. But 16-bit float (as in 16-bit EXR) has variable precision which can cause visible stepping at high altitudes. I worked on a project last year where this was noticeable, so I needed to add support for 32-bit EXR to solve this. The alternative is to stay with 16-bit integer TIFF, and this will save some disk space, but the downside is that you have to keep track of where black and white are.
Matt
Quote16-bit float (as in 16-bit EXR) has variable precision which can cause visible stepping at high altitudes
Yes, I had the same thing before 32 bit exrs were introduced - see:
https://planetside.co.uk/forums/index.php/topic,21829.msg240240.html#msg240240 Reply #6
shows exactly that :)
Quotebut the downside is that you have to keep track of where black and white are.
That's why I prefer 32 bit exr, the values are a real world correlation to Terragen.
I see a lot of post with people wrestling with large file sizes - for displacement I presume, and running into limits imposed by Photoshop and suchlike.
I have a solution that might work for some people - more info soon.
I have been doing some more testing with 32-bit EXR heightfields, as well as 32-bit Tiff heightfields and has discovered a few things. I think Oshyan mentioned it before, but older versions of Photoshop like CS5 don't properly save 32-bit exr files. The exr files get clamped at 16-bit levels for some reason, so when loading on in as a heightmap in terragen the max height for the heightfield will show as 65.5 km, which I think relates to the max channel value of 65536 (https://www.cambridgeincolour.com/tutorials/bit-depth.htm) in 16-bit images. I did try saving the same 32-bit exr as a 32-bit grayscale tiff out of Photoshop CS5 and that worked perfectly in Terragen as a heightfield.
erx-io (https://www.exr-io.com/) as mentioned is a plugin that supports saving 32-bit exr's with a bunch of options and that does properly save the 32-bit exr for use as a heightfield in Terragen.
Another interesting thing I discovered is the "Height range" displayed at the bottom edge of the Heightfield load shader is very useful for knowing how much elevation data is in your heightmap. With an 8-bit heightmap the max value you will see there is 256m (0-255 channel values) and for 16-bit heightmaps you will see up to 65.5 km (0-65536) and for 32-bit heightmaps you can see values beyond 1.03e+003 km (0-1,024,000 +).
Not that I would recommend doing this for any reason but just because I was curious I made a heightmap with such a huge range I had to use a displacement multiplier of "0.00000000000000000000000000000000001" to get my heightfield to the appropriate height within the terragen camera view. The range for that 32-bit heightmap was "min = 0 mm, max = 3.4e+032 Mm".
Well, glad I got that figured out. :)
-Derek
So erx-io is the way to go it looks like. Did you try proexr? That's what I was looking at for awhile.
Quote from: WAS on May 01, 2020, 03:01:57 PMSo erx-io is the way to go it looks like. Did you try proexr? That's what I was looking at for awhile.
No I haven't tried ProEXR (https://www.fnordware.com/ProEXR/) yet, and I thought it was a paid plugin but now looking at the web site it looks like it is free.
I'll download it and check it out as well. Thanks!
ProEXR works well. With some renderers you can use it to store all of the render passes in a single layered file, very handy. (Perhaps other EXR plugins do this as well, I've only used this one and OpenEXR.) Don't recall ever running into any limitations with very large files. I licensed it years ago and saw just recently that it is now free.