Loading large TIFF/EXR Files

Started by WAS, March 14, 2021, 03:44:46 PM

Previous topic - Next topic

WAS

When a size is above 16k I get a "Cannot locate file" error. Is this because of my 32GB of RAM and it failing to load into memory? It seems odd when the files are only 1-2gb in a fresh TG instance.

This is with the Image Map Reader. Obviously the file is there, I'm double clicking on it from the file openers, and can open it in Affinity/PS. Even if the EXR/TIFF is coming from TG it doesn't seem to matter. Over a certain resolution TG just says the file can't be found on the HDD.

sboerner

Never had this problem with large files (tiff or exr). Maybe try replacing the filename with a relative path? (I type relative paths in Notepad and paste them in.)

WAS

So i tried that. Same deal. It can load the 8k versions fine, but 12 and 16 won't load. Same filenames besides the 12 and 16 part too. Most of these saved by TG itself.

luvsmuzik

You have probably discussed this ad infinitum, but why not use heightfield load for the tiff and then adjust height, and  resize?  I took the color image into PS and reduced to grayscale and then increased to 16K

WAS

#4
Heightmaps dont load either. Nothing over 12k will load in TG on my system. Regardless of location or filename. The origjnal tif from hirise loads but not ones exported from TG or even heightmaps i made for devils tower at or above 12k. 8ks work just fine.

Kadri

#5

That seems strange. Kinda like you can not load big files in 32 bit programs. But that shouldn't be your problem of course.

sboerner

Jordan, I couldn't remember the last time I used a 12k or larger image file, so I just created a random image in Photoshop (cloud filter), 12288x12288, and saved it out as a 32-bit exr and 8-bit jpeg and tiff. No problems loading any of them.

I'm also working with 32GB ram.

Matt

Can you show a screenshot of the error? I am searching the code for this error but haven't found it yet.
Just because milk is white doesn't mean that clouds are made of milk.

WAS

Yeah here's an example. It's just normal file not found error like loading a project without assets.

WAS

Some extra info since Kadri had asked, I have tried files from all 3 storage mediums, the SSD, HDD, and M2, so I don't think it's a storage issue + Photoshop / Affinity can load the files fine.

SuddenPlanet

WAS, I tried loading those Mars EXR's I made for you and they loaded both as Heightfields, and using the Image map shader.  The Tiff files however made Terragen 4.4.67 go POOF!

I have 64GB of RAM in my machine too.  I have had issues with Tiff files larger than 30k though in the past.

Matt

This appears to be an error with files larger than the 32-bit limit, i.e. just a little over 4 Gb. We were using a 32-bit system function (_stat) to get info about the file. I am working on a fix now, so this shouldn't be a problem in future.

If your EXR was rendered from Terragen then it'll have 3 channels (RGB). Perhaps you can reduce its size by converting to single channel.
Just because milk is white doesn't mean that clouds are made of milk.

WAS

I'll try to reduce the TG ones.

However most these TIFFs/EXRs are over the 2gb mark, and under 3gb. Two are over 4gb, and I believe when exporting them from PS there was even a warning about exceeding conventional file sizes.

Matt

Quote from: WAS on March 17, 2021, 10:13:19 PMI'll try to reduce the TG ones.

However most these TIFFs/EXRs are over the 2gb mark, and under 3gb. Two are over 4gb, and I believe when exporting them from PS there was even a warning about exceeding conventional file sizes.

Maybe the limit is 2 Gb rather than 4 if there's a signed int involved, but I didn't gather that from the _stat documentation. Anyway, I'll fix it.
Just because milk is white doesn't mean that clouds are made of milk.

sboerner

Just wanted to confirm that I also ran into this issue when trying to place obj files > 2GB. TG acted like it was importing the file, then reported a "cannot locate file" error.