Planetside Software Forums

Support => Terragen Support => Topic started by: WAS on May 16, 2020, 02:11:06 PM

Title: Terragen IOR
Post by: WAS on May 16, 2020, 02:11:06 PM
It's OK to be wrong, and again, why there are generalized tables for 3D graphics, that even TG defaults are based on, that don't match real-world constants of the same materials. "Too complicated". It's self-explanatory. You know you're just using the same formulas other engines use for IOR in 3D graphics. Jesus. These are all generalized, as even the charts admit.  If it wasn't, you'd assume real constants would translate, instead of known generalized ones for 3D - that being, more wide-spread, common, a standard. You're trying to say its something that it is not, in actual applied logic. No ifs about it.

I already experimented with this heavily for over a year making real-world materials procedurally. Almost no real-world IORs translate besides some generics like water at 20c, and stuff. Oshyan and you were even wondering why I was and it wouldn't be "right"  using IOR 1.12 for rock and how it may be better to just use 1.33 and lower reflectivity.
Title: Re: Terragen IOR
Post by: Matt on May 16, 2020, 04:08:06 PM
You misunderstood what I was saying.

"Too complicated" referred to splitting the topic without disrupting the on-topic conversation. I wasn't saying anything about IOR there.

Metals (conductors) need to be handled differently from other materials. Metals/conductors need to either use a metalness parameter or a separate underlying layer with high IOR to mimic their high reflectivity.

Most non-metals are dielectrics. With dielectrics I maintain that you can use real world IOR values in Terragen. Where confusion arises is with roughness. An IOR of 1.5 doesn't make it shiny if the roughness is high enough. If you have a material that looks too shiny when you plug in a real-world IOR, this is a very strong hint that you need to change the roughness.

IOR values in the real world correspond to settings in Terragen. IOR tables can and should be used, there are just some caveats to consider. This is a really useful tool to help make your materials mor realistic. When someone discovers this they make a leap in understanding. That's why I think care should be taken when starting a reply with a broad statement like "TG index of refraction values do not match scientific values at all FYI. Waaaay off from scanned readings for many materials, most especially being reflective." That's why I said you were generalizing. I don't want people to read that and start thinking that IOR values can't be used, because they can and they should be used. Discussion should focus on the exceptions to this rule, and how to handle those exceptions.

In Terragen 4.5 the Default Shader will have a metalness parameter. When you set this to 1, the base colour is used for the reflective colour of the metal. IOR values affect the transition towards glancing angles, so they are still important, but you don't need to cheat with high values anymore.

In versions prior to 4.5, I recommend mixing a normal IOR top layer with an underlying fake IOR for metals.

For non-metals, the primary way to control how shiny they are is the roughness parameter.

Of course these are just approximations of reality, and small tweaks may be needed. But if you find that IOR is "way off" for a dielectric and it can't be fixed with roughness, the next things to look at should be lighting, environment and render methods.



Title: Re: Terragen IOR
Post by: Matt on May 16, 2020, 04:20:06 PM
Quote from: WAS on May 16, 2020, 02:11:06 PMIt's OK to be wrong, and again, why there are generalized tables for 3D graphics, that even TG defaults are based on, that don't match real-world constants of the same materials. "Too complicated". It's self-explanatory. You know you're just using the same formulas other engines use for IOR in 3D graphics. Jesus. These are all generalized, as even the charts admit.  If it wasn't, you'd assume real constants would translate, instead of known generalized ones for 3D - that being, more wide-spread, common, a standard. You're trying to say its something that it is not, in actual applied logic. No ifs about it.

I don't know what this has to do with anything I've said. You seem to be taking the word "generalize" and applying it to a whole different point which I did not make.
Title: Re: Terragen IOR
Post by: WAS on May 16, 2020, 04:28:57 PM
Alright, the roughness makes sense, but what about when you are sampling the material raw on a smooth surface, to get the "sheen" right before ever working on roughness? There are a lot of materials that just did not translate well. For example feldspar and mica flakes. Take feldspare with a IOR 1.518–1.526, like glass. When I used this, I could never really get the twinkling shine you get in marble from the felspar and mica without ramping it up. This is just a basic "glass" like IOR effect, and not really "metalness", or is it? I even had issues with the quartz property, and had to mess with the IOR of powdered quartz rather than crystalline.
Title: Re: Terragen IOR
Post by: Matt on May 16, 2020, 05:09:39 PM
If you're not getting the reflectivity you expect from a dielectric, let's look at renders and discuss them to figure out why.
Title: Re: Terragen IOR
Post by: WAS on May 16, 2020, 05:18:44 PM
Quote from: Matt on May 16, 2020, 05:09:39 PMIf you're not getting the reflectivity you expect from a dielectric, let's look at renders and discuss them to figure out why.

I'll have pull a HDD from my old machine for that for non-corrupted files (my external I used to backup to move is messed up). Though Oshyan scrapped most these files as I shared them. Granite, marble traventino, etc. Was gathering them for your public assets release.

I know for the granite i based it on a rock here in my park, visited in sunny weather, cloudy, twilight, 5,000 lumen flashlight and phone flash light. I studied that thing to try and make realistic granite, and matching reflectivity wasn't just plugging real world data in. Maybe it was roughness?
Title: Re: Terragen IOR
Post by: Matt on May 16, 2020, 05:23:30 PM
Scrapped them from where?
Title: Re: Terragen IOR
Post by: Matt on May 16, 2020, 05:59:09 PM
A couple of ideas, maybe they are useful:

Base albedo is important, but it's difficult for humans to guess correctly. If your base is too high then it will make the glossy surface seem dim in comparison.

Something that we have to be careful about in CG is specular highlights on sharp corners and edges. The renderer can underestimate them or miss them entirely, resulting in an incorrect image.

Title: Re: Terragen IOR
Post by: WAS on May 17, 2020, 12:06:26 AM
Quote from: Matt on May 16, 2020, 05:59:09 PMSomething that we have to be careful about in CG is specular highlights on sharp corners and edges. The renderer can underestimate them or miss them entirely, resulting in an incorrect image.

This is tricky in a lot of circumstances, speckle where you don't want it from specular highlights. For example, rough terrain at a distance you may just a random twinkle here or there that's hard to defeat without changing the overall look. This is why I mentioned recently having specular being a feature to enable, disable, or even use alone (like in a lot of other programs to speed up rendering while maintaining a look of wetness/reflection. A good example of this is Jurassic park back in the day with the T-Rex at night. Actual reflections would have cripples their render budget, so they just used specular highlights alone, and that alone made Rexy look reflective and wet under point lighting in a dark scene. This would be wonderful for specular materials that shimmer, without the crippling effects of all the other calculations, especially under PT. You can't do interior lighting without a heavy impact, add reflective surfaces and it's kinda sad, really (take Hannes Indoor Lighting for example with just foreground lighting preset: https://planetside.co.uk/forums/index.php/topic,22817.msg230625.html#msg230625 )
Title: Re: Terragen IOR
Post by: Matt on May 17, 2020, 11:44:20 AM
1. Specular highlights can be enabled/disabled per light source. Not per material, but if increasing roughness doesn't do what you want and you don't want to turn it off on the light source itself then I think we are talking about a very narrow use case. I am going to add a new "direct highlight min spread" parameter to boost roughness of direct specular highlights. This should help you to keep sharp reflections of the environment while softening highlights of light sources. (Motivation being to catch highlights on sharper corners.)

2. For a render with only highlights from direct light sources (the Jurassic Park method), use the Standard renderer and don't enable ray traced reflections in your shaders. Why use PT in this case?

3. If Hannes scene from 2017 is using ray traced reflections in the Standard renderer and you want to turn them off, turn them off on the shaders where they are enabled. The shaders don't need any extra controls to do this. If you want to render this with PT but you only want diffuse reflections, I don't think turning off glossy reflections will be much faster, because diffuse reflections are also path traced.
Title: Re: Terragen IOR
Post by: Tangled-Universe on May 18, 2020, 10:40:11 AM
Quote from: WAS on May 16, 2020, 05:18:44 PM
Quote from: Matt on May 16, 2020, 05:09:39 PMIf you're not getting the reflectivity you expect from a dielectric, let's look at renders and discuss them to figure out why.

I'll have pull a HDD from my old machine for that for non-corrupted files (my external I used to backup to move is messed up). Though Oshyan scrapped most these files as I shared them. Granite, marble traventino, etc. Was gathering them for your public assets release.

I know for the granite i based it on a rock here in my park, visited in sunny weather, cloudy, twilight, 5,000 lumen flashlight and phone flash light. I studied that thing to try and make realistic granite, and matching reflectivity wasn't just plugging real world data in. Maybe it was roughness?

I considered trying similar things to capture real world characteristics of certain materials and did some reading on it.

A few things are worth considering:
Use grey cards or an X-rite colourchart (which includes a couple of whitebalance checkers as well) and capture a bracket of exposures with your camera.
Match the behavior of your camera's sensor to TG's lighting.
As Matt mentions base albedo is key to getting this right, but how exactly is difficult to tackle, because what's the albedo of a surface before it's lit? Sounds contradictory to me, but that's essentially/basically what TG is asking you and there's no answer.

With regards to reflectivity you would have to make the surface wet and shoot similar brackets to figure out how real world brightness compares to TG's specular reflections, while keeping track of sun angle and such.
Fresnel reflections can be extracted using polarizers in front of your camera. They are really important for 'liveliness' of your shaders.
Google for "everything has fresnel" (or along those lines) and you will find a lot of useful info.

All in all it's a pretty huge undertaking, but I'm sure there's a lot to learn from it... I'd love to have some more time to figure these things out as well, also just for mental exercise and curiosity.
Title: Re: Terragen IOR
Post by: WAS on May 18, 2020, 05:20:55 PM
Quote from: Tangled-Universe on May 18, 2020, 10:40:11 AM
Quote from: WAS on May 16, 2020, 05:18:44 PM
Quote from: Matt on May 16, 2020, 05:09:39 PMIf you're not getting the reflectivity you expect from a dielectric, let's look at renders and discuss them to figure out why.

I'll have pull a HDD from my old machine for that for non-corrupted files (my external I used to backup to move is messed up). Though Oshyan scrapped most these files as I shared them. Granite, marble traventino, etc. Was gathering them for your public assets release.

I know for the granite i based it on a rock here in my park, visited in sunny weather, cloudy, twilight, 5,000 lumen flashlight and phone flash light. I studied that thing to try and make realistic granite, and matching reflectivity wasn't just plugging real world data in. Maybe it was roughness?

I considered trying similar things to capture real world characteristics of certain materials and did some reading on it.

A few things are worth considering:
Use grey cards or an X-rite colourchart (which includes a couple of whitebalance checkers as well) and capture a bracket of exposures with your camera.
Match the behavior of your camera's sensor to TG's lighting.
As Matt mentions base albedo is key to getting this right, but how exactly is difficult to tackle, because what's the albedo of a surface before it's lit? Sounds contradictory to me, but that's essentially/basically what TG is asking you and there's no answer.

With regards to reflectivity you would have to make the surface wet and shoot similar brackets to figure out how real world brightness compares to TG's specular reflections, while keeping track of sun angle and such.
Fresnel reflections can be extracted using polarizers in front of your camera. They are really important for 'liveliness' of your shaders.
Google for "everything has fresnel" (or along those lines) and you will find a lot of useful info.

All in all it's a pretty huge undertaking, but I'm sure there's a lot to learn from it... I'd love to have some more time to figure these things out as well, also just for mental exercise and curiosity.

This is really useful information to me, actually. Thanks a lot Martin. grey cards and "x-rite colourchart" brought my to whole new areas of the net I haven't been. I'm honestly surprised none of this came up when doing materials and objects based on real world objects.

I guess I didn't share as many materials as I thought I did and only have a few to return to from here but I got new ideas for others. And may as well start fresh with no knowledge. I wanted to pull my other HDD but it has a OS on it, and my current PC has no chassis available, or SATA slot, which means unhooking the 5tb I'd use to back things up onto. Ugh

Edit: Might as well put this here; I found a few method of obtaining albedos, but I was wondering how I could translate this to the -0-1+ range in TG? The methods I keep finding give me large ranges for "ambient" and "reflected", and than albedo precentages. How can I use these methods to obtain useful data for Terragen? Or can I? It seems like it would be a good method in my mind.

This is a pretty comprehensive explanation: https://vixra.org/pdf/1110.0035v1.pdf

May seem silly I obsess over this but I really like the idea of accuracy and perfectionism, and I guess it's been noticed as I have gotten pitches based on my past materials I posted here.
Title: Re: Terragen IOR
Post by: WAS on May 18, 2020, 05:22:56 PM
Thanks for sticking it out guys, I see how TG can be handling real world data accurately, but easily bugged now. Actually, to that effect I was comparing some PBR materials and notice a lot of examples are just wrong, and it's not Terragen that's wrong.
Title: Re: Terragen IOR
Post by: KyL on May 18, 2020, 05:57:27 PM
Quote from: WAS on May 18, 2020, 05:22:56 PMActually, to that effect I was comparing some PBR materials and notice a lot of examples are just wrong, and it's not Terragen that's wrong.
Pretty much sums it all. PBR is a nice acronym but doesn't mean accurate data unfortunately. For example I find Megascan assets to be totally inconsistent with each others. You will have area with values above 0.8 in leaves textures or rocks. White balance seems off most of the time too, so in the end you cannot ever know how correct the files are.

One good source I found is the 3D scans from Textures.com. They seem to really care to properly expose and neutralize their textures, and most of the time provide a color chart/balls reference. I have been using it extensively and anytime I add a new textures they stay consistent with each others.

But I think that unless you are really working as a researcher in computer graphics, most of it is up to the eye of the artist at the end of the day! We see and judge things with our non-so-easy to fool minds, not as purely accurate machines.

And also a quite old but still very pertinent source on the subject: https://youtu.be/FzTHZRKUjjM (https://youtu.be/FzTHZRKUjjM)
Title: Re: Terragen IOR
Post by: WAS on May 19, 2020, 12:59:29 AM
Yeah actually I noticed this just last week messing around on my friends UE account. He has tooooons of Megascans, and I was checking out the icelandic scene, and it seems some of their assets, of the same rock-type, and location (seemingly) have different albedos. When I started adding them, I immediately had to play with the texture tints just to match each other, let alone the HDRI.
Title: Re: Terragen IOR
Post by: N-drju on May 19, 2020, 02:06:12 AM
My two cents would be - we need to base on something. If Matt has implemented the IOR feature and set water IOR value to 1.33 I have no reason to distrust him as a software designer.

I don't care that values are "off" or "in" as long as water looks like water and chrome looks like chrome.

A really basic way of looking at things on my part, I agree. But I have no time to delve into physics and molecular biology while working with Terragen. Enough said. Maybe I don't have enough of experience yet, but really... does anyone have a spectrometer and x-ray in their eyes to determine whether a texture is accurately represented or not?
Title: Re: Terragen IOR
Post by: WAS on May 19, 2020, 02:02:49 PM
Quote from: N-drju on May 19, 2020, 02:06:12 AMMy two cents would be - we need to base on something. If Matt has implemented the IOR feature and set water IOR value to 1.33 I have no reason to distrust him as a software designer.

This is something that is variable though. 20c low oxygenated tropical water has a IOR of 1.33. Head down south towards the Antarctic with high salinity and oxygen plus much lower temperatures and you're entering the realm of 1.397 for example. Doesn't seem like much but if you're looking onto the horizon of ocean in these two zones, you can see the difference in sheen.
Title: Re: Terragen IOR
Post by: Matt on May 19, 2020, 02:08:24 PM
That's one of the reasons you can change it :)  Do you have references for that 1.397? That's pretty high.
Title: Re: Terragen IOR
Post by: N-drju on May 19, 2020, 02:15:35 PM
Quote from: WAS on May 19, 2020, 02:02:49 PMThis is something that is variable though. 20c low oxygenated tropical water has a IOR of 1.33. Head down south towards the Antarctic with high salinity and oxygen plus much lower temperatures and you're entering the realm of 1.397 for example.

That's obvious, sure. It is up to an artist to decide about the "conditions" they have in a render, but one has to start somewhere.
Title: Re: Terragen IOR
Post by: WAS on May 19, 2020, 03:29:39 PM
Quote from: Matt on May 19, 2020, 02:08:24 PMThat's one of the reasons you can change it :)  Do you have references for that 1.397? That's pretty high.

I believe it was on the IOR info website? May have been another one. I also been reading individual studies, which is where I found IOR of water varies substantially with different salinities, oygenation and between liquid and solid phases (such as high salinity moving ocean water being kept liquid when in other conditions it would freeze). The IOR mentioned may have been a different wavelength as a lot of charts allow you to adjust, which we don't have in TG (and I'm not even sure what the default is or how it's modulated etc)