99% of final pass - Stuck

Started by SuddenPlanet, December 20, 2021, 11:58:35 AM

Previous topic - Next topic

SuddenPlanet

Quote from: WAS on December 21, 2021, 01:18:51 PMMaybe try 32bit TIF output instead of EXR? There isn't much benefit to a 32bit EXR as it doesn't contain extra data the format is used for.

Oh, that's right. TG still just does half-float. >.<
I'm not sure you are correct on that point about Terragen only supporting Half Float.  Has Matt said that?  Or have you seen that stated anywhere in the Documentation?

I did a ton of HDR Skyboxes for the last game I worked on and was doing a lot of technical image crunching to get the Dynamic Range of Terragen's output down because it was outputting HUGE ranges of brightness in the Sun Light (Sometimes over 100,000).

For example I just did a test render of the default scene, moving the sun down to 15 degrees, and leaving it at the default 5 for brightness.
I then saved the resulting image as 16-bit Tiff (32-bit Tiff not available in Terragen that I can find), 32-bit EXR, and 16-bit EXR (Half Float).

After loading the three images into photoshop and sampling the brightest spot of the sun I got substantially higher readings on the 32-bit EXR.
Here are the Results using the 32-bit Eye Dropper:

EXR 32-Bit: 73777
EXR 16-Bit:  5844
TIFF 16-Bit: 1.0

Does that make sense?

Derek

WAS

#16
I mentioned that on line 2, that there is no 32bit float. And I just mistook TG for having such a common format in DCC.

That's where I'm just like "Meh" with some of the logic put into TG. It's backwards. EXR is a waste when it's not even utilizing the standard method of use, incorporating channel information, etc, in one file.

And then we don't have 32bit float TIFF, which would be the proper exporting method here, since we are not utilizing the standard workflow of EXR.

Also stranger yet we have a reader that can read 32bit float TIFF, but not write it.

SuddenPlanet

Ah I see.  Well maybe we will get some of those things in Terragen 5. 

On a positive note I did get some of my 16k x 16k renders to finish and save.
One thing I did was turn off GI surface details, so maybe that was the problem.

Thanks for pointing me to potential GISD problems. 

Derek

SuddenPlanet

Oh, and I just remembered another reason I don't like Tiff.  Photoshop CS5 complains about not having enough memory to open large Tiff files, even though there is plenty of RAM.
Same file saved as an EXR loads just fine.  Maybe I need to upgrade to Creative Cloud (I'm guessing the latest version might be better at loading Tiff).  Just tired of more and more subscriptions.

WAS

Quote from: SuddenPlanet on December 21, 2021, 03:50:45 PMOh, and I just remembered another reason I don't like Tiff.  Photoshop CS5 complains about not having enough memory to open large Tiff files, even though there is plenty of RAM.
Same file saved as an EXR loads just fine.  Maybe I need to upgrade to Creative Cloud (I'm guessing the latest version might be better at loading Tiff).  Just tired of more and more subscriptions.

Photoshop 2021 trial is spoiling me. I don't want to go back... Lol But i can't afford a subscription to all this stuff. No way. I'm not a studio.

I haven't really had much issue except for the typical, files over 4gb won't load, regardless of EXR or not. I think that's a rasterization issue, as Affinity Photo doesn't have an issue.

WAS

PS Affinity Photo is a little strange coming from Photoshop, but it's well worth the price for it's 32bit support, and large file support.

I also think it's buy once. I bought it a little over a year ago now, I think, and it still gets updates to new versions from Windows store.

WAS

Sorry for another post. I just don't want it to get lost in other off-topic info.

It would be great if TG could export it's GISD pass as a layer, so in situations where GISD generation is too much, you can just do another render for the GISD itself and comp it in post. As far as I'm aware GISD is just doing an AO like effect which could be exported as a layer?

SuddenPlanet

Yea, maybe GISD could be added into Render Layers/Elements.

Oh and thanks for reminding me about Affinity Photo.  I bought it awhile back on sale for $25 and was able to convert my 16k tiff to other formats, so that was nice.  Thought I had wasted a render.  :)

noahding1

Increase memory. I have not had any crashes since I increased memory by 64GB

SuddenPlanet

Quote from: noahding1 on December 24, 2021, 03:58:03 AMIncrease memory. I have not had any crashes since I increased memory by 64GB

I have 128GB

Matt

Quote from: WAS on December 21, 2021, 01:18:51 PMOh, that's right. TG still just does half-float. >.<

Internally Terragen renders colours using 32-bit floats, and when saving EXRs you can choose between 16-bit and 32-bit float.
Just because milk is white doesn't mean that clouds are made of milk.

Matt

Quote from: SuddenPlanet on December 21, 2021, 03:28:36 PMOn a positive note I did get some of my 16k x 16k renders to finish and save.
One thing I did was turn off GI surface details, so maybe that was the problem.

GISD can take a while at such high resolutions, but I was still surprised to hear that it was taking hours for you. Was the GISD using default settings?
Just because milk is white doesn't mean that clouds are made of milk.

SuddenPlanet

Quote from: Matt on December 24, 2021, 12:52:32 PM
Quote from: SuddenPlanet on December 21, 2021, 03:28:36 PMOn a positive note I did get some of my 16k x 16k renders to finish and save.
One thing I did was turn off GI surface details, so maybe that was the problem.

GISD can take a while at such high resolutions, but I was still surprised to hear that it was taking hours for you. Was the GISD using default settings?

Hey Matt,

Yea, all Default Settings for GISD.  I never did definitively determine if that was the source of the hang at the end of the rendering though.  There were a couple other things I changed settings on like turning off Anti-aliasing bloom, and I might have changed shadow settings on the Sun.  I could more precisely determine the hang source with some more test renders if it is helpful.  

Derek

WAS

Next time it happens see if the task manager shows disk usage. If so it may be like I was mentioning and TG is trying to offload its assets to harddisk instead of RAM. I used to use scratch instead of RAM when I only had 4gb of RAM. While it worked mostly for me, it worked in slow motion, and was prone to crashing. But hey with 512gb allocated at the time I was able to do huge scenes that even this PC with 32gb crashes on. Just have no disk space available for scratch.

SuddenPlanet

#29
I always watch the Performance Monitor when I am rendering/building in various programs like Terragen, World Machine, etc.

I remember CPU activity was very low, like less than 1%.  No Disk activity, but I did have some Ethernet activity, which I assume was because I was using Remote Desktop to access my PC.

As far as disk swapping, I don't think Terragen or Windows would do that unless RAM was used up, and like I said before I was only at 75GB capacity of my 128GB (Windows 10 + Terragen).

I'll do some more test renders when I get time to see if I can nail this down to a specific setting.

Derek