Planetside Software Forums

General => Terragen Discussion => Topic started by: keyframe on January 18, 2011, 03:41:01 PM

Title: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 18, 2011, 03:41:01 PM
Hello Everyone,

My apologies for the somewhat vague posting - i'm trying really hard to wrap my head around some of the concepts of the TG2 demo - and it seems that I'm failing.  Perhaps someone could shed some light on this for me.  I am familiar with PRman shader writing, and manipulating noise functions to create 'stuff'.

I have a series of questions that are somewhat related, but if you could bear with me, I would really appreciate it.

-What data type is returned by the displacement functions? (Alpine Fractal, for example)
-What data type is expected as input to the displacement functions (They seem to be able to operate both with, and without input -- does having an input imply Base_fractal+New_fractal?)
-Can that data type (Displacement? Vector?  Scalar? foo?) be manipulated by using the 'Functions' (for instance, can I multiply the result of Alpine Fractal, by 2), and still have it be a 'valid displacement'? (my tests seem to indicate that it cannot - the minute that I append any multiply node after Alpine Fractal, my terrain disappears).
-What data type is returned by the 'color functions' (Power Fractal for instance)?
-What is the relationship between the 'combo nodes' color and displacement parameters?  (PF, for instance, has both color AND displacement controls -- which suggests to me that they are handled separately -- gut instinct: displacement scalar, color vector)
-Where are the frequency controls? (I've noticed that some functions have a 'Tweak noise' parameter (PF), and some do not (AF)).

I've scoured the forums for clues - and the answer I most typically find is "What are you trying to achieve specifically" (people in this community seem very generous with their time and knowledge - this is fantastic to see).  In my case, i'm not really attempting to achieve anything (yet?!), I just want to understand the inner working of this system.

If someone has a few minutes to address my questions, I would really appreciate it.

Best,

G
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Tangled-Universe on January 18, 2011, 05:10:34 PM
Welcome to the forums :)

Quote from: keyframe on January 18, 2011, 03:41:01 PM

I've scoured the forums for clues - and the answer I most typically find is "What are you trying to achieve specifically" (people in this community seem very generous with their time and knowledge - this is fantastic to see).  In my case, i'm not really attempting to achieve anything (yet?!), I just want to understand the inner working of this system.


I understand your frustrations and feelings of "where the hell do I start". We all had to go through this, but admittedly, it shouldn't be so hard as it often is for many.

However, I think it is really the wrong way around to first want to understand the inner workings of TG2 and then start working from there.
In my opinion you can already learn a lot by playing around and especially doing a lot of mini-projects, which basically means trying to achieve one specific effect (cumulus cloud, detailed fake stone etc. etc.). This way you learn the workflow and the functions in small chewable steps.

At some point it is important to understand what kind of data each input accepts/requires exactly, but I think (almost want to assure you) that's the least of concern for someone starting TG2.

Anyway, I'll try to answer some of your questions. I always try to explain by examples so that you can reproduce the idea and understand it better. Also, it shows that through experimentation you certainly can deduce much of TG2's inner working as there's luckily enough logic in the system :)
It's clarity on things which lacks, so let's try to work on that from here:

Quote
What data type is returned by the displacement functions? (Alpine Fractal, for example)

First question, first exceptions in the answer already :)

The alpine fractal is an advanced fractal function like the powerfractal v3 shader and only generates displacement (vectors). However, the powerfractal v3 shader can also generate colour, hence the colour tab inside.
This way you can have colour (shading) match your displacements, for example.
The alpine fractal can not. The alpine fractal only outputs displacement.

The colour tab in the powerfractal v3 shader does NOT affect the displacement.
Open a new default project and generate a terrain from a powerfractal and enable high and low colour in the powerfractal.
Play with colour, contrast, offset etc. No effect.

So I discussed briefly an Alpine shader and powerfractal v3 shader, but there's also a displacement shader.
Here things change a bit.
The left input of the shader accepts displacement and colour and is meant to link the displacement shader in the network.
The right input (named "function") requires colour/scalar input from which it calculates displacement.
However, it does NOT accept displacement. So when you connect a powerfractal with colours unchecked, the displacement shader does nothing.
The strength of the displacement is determined by the displacement factor inside the displacement shader.
So, as it requires colour/scalar you can use a powerfractal with its colours enabled as (right!) input of the displacement shader.
Now the colour tweaks DO have effect. Change contrast, offset etc. and the displacement will look different.
That is because you tweak the colour function and thus tweak the displacement function consequently.

Quote
What data type is expected as input to the displacement functions (They seem to be able to operate both with, and without input -- does having an input imply Base_fractal+New_fractal?)

Generally the red nodes in TG2 have a main input to the right which always accepts displacement and colour. You need this in order to pass data through to the next node.
Most nodes perform an operation by default and thus affect outcome when they are connected in your network.
The displacement shader, for instance, does nothing if you don't have a function connected to its right "function" input.
Data just passes through the node, unaffected, then.

I'm not really sure to what you are referring to when you say "displacement functions". For instance, the surface layer also has a displacement function input port.
For the record, this input works similar to the displacement shader.

Quote
Can that data type (Displacement? Vector?  Scalar? foo?) be manipulated by using the 'Functions' (for instance, can I multiply the result of Alpine Fractal, by 2), and still have it be a 'valid displacement'? (my tests seem to indicate that it cannot - the minute that I append any multiply node after Alpine Fractal, my terrain disappears).

Certainly.
You would require to use a merge shader and conect that after your Alpine fractal. Inside the merge shader go to the "merge mode" tab and choose "multiply (Input * A's diffuse colour) as merge mode for displacement. As that is what you would like to affect.
Hey, wait a minute, "A's diffuse colour". It seems we need a constant colour value to multiply with. Create and connect a "constant colour" function to the "Shader A" input of the merge shader.
Give the constant colour any value you would like the fractal to be multiplied with.

Logically, you're now manipulating the displacement multiplier/amplitude.
The easiest way, without having to use any extra node, is to simply increase the "displacement amplitude" setting in the Alpine/Power fractal.
So, if you wanted to multiply with 1.5 then enter a 1.5x greater value as displacement amplitude.
You'll see that the result is exactly the same as using a merge shader in multiply mode with a constant colour.

You can create function networks which create displacement. It's pretty much to explain, but there's a nice tutorial about that here:
http://forums.planetside.co.uk/index.php?topic=1166.0 (http://forums.planetside.co.uk/index.php?topic=1166.0)

Quote
What data type is returned by the 'color functions' (Power Fractal for instance)?

This should be a bit more clear by now. The answer isn't very surprising, a colour function outputs colour.
You should keep in mind that you disable displacement and enable colour in a powerfractal when you explicitly only want to use its colour output.
I guess that would make sense.

Quote
What is the relationship between the 'combo nodes' color and displacement parameters?  (PF, for instance, has both color AND displacement controls -- which suggests to me that they are handled separately -- gut instinct: displacement scalar, color vector)

I also discussed that already a bit. May of your questions are inter-related. Yes, they are separate and do not affect each other.
As I mentioned it could be useful to have colours generated along with displacement, to get a match or other interesting effects.

Quote
Where are the frequency controls? (I've noticed that some functions have a 'Tweak noise' parameter (PF), and some do not (AF))

The frequency controls are in the "scale" tab and "tweak noise" tab. Both tabs control the noise-function generated by the fractal.

The scales are determined by a "feature scale", "lead-in scale" and "smallest scale".
These 3 values determine how many octaves of noise will be generated. Octaves mean how many iterations, thus the amount of detail.
Low octaves is less detail, many octaves is much detail.

The lowest limit of detail is determined by the smallest scale. The most dominant scale of shapes is determined by "feature scale" and the "lead-in scale" defines the total range of scales being used.

In the tweak tab you can choose which noise-flavour you would like to use, but also there are controls to modulate the frequencies/scales.
The precise workings of these functions have never been documented or explained by the staff.
With the noise variation you can kind of localize the noise to patches (by increasing the value quite a lot, say >5-10)
Or make it very evenly distributed in regard to scales, by making it <0-1.
The variation methods are a bit unclear, still.
Multi-scale modulator probably mixes an extra iteration of the same multifractal in some way, which you can control with the noise variation setting.
The other two, (un)clamped multi-fractal basically mean that you don't want to use the multi-scale modulator and either want to use the multi-fractal itself, being it clamped or unclamped.

The Alpine Fractal does not have these tweak parameters as you can consider the Alpine Fractal as an advanced powerfractal, probably a fancy combination of powerfractal noise with other clever elements. The UI for this shader is probably simplified to make it useable and probably also to avoid that the user will render TG2 useless as the shader already is very "heavy" without all of these (extra) fancy controls (with which you're likely to make it even worse if you don't know what you're doing).

Clamped means that the colour output is restricted to range from 0 to 1.
Unclamped means that it can go beyond these values, without limits. (probably there are, but not that I know).



Ok, this took a bit more than a few minutes, but hopefully things are a bit more clear now.
Time for a drink, might change little things later, but this is primarily how I think when using TG2.

Cheers,
Martin
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: airflamesred on January 18, 2011, 07:19:29 PM
Thats usefull info TU. Many thanks for your time
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 18, 2011, 08:37:27 PM
Wow Martin,

Thank you so much for the detailed discussion!

I think part of my confusion is that in 'our' world (Houdini), data is a little bit more fluid.

A function returns 'values' - how you map it, is completely up to the user.  You can use the result of perlin noise (whether 1D, 2D, or 3D), and map the results to displacement, color, or use as a basis for yet another function.  I suppose I was looking for an analogue in the TG world, but I suppose that's a bad approach on my part.

imagine:


x = perlin3d(P);                               // Compute the value of the perlin noise at (P) - returns a vector3
x2 = perlin1d(P.x);                            // Compute the value of the perlin noise at (P) - returns a scalar
x_ranged = fit(x, -0.5, 0.5, 0,1);        // Perlin returns noise typically found on [-0.5, 0.5]
x_clamp = clamp(x_ranged, 0.2, 0.8);  // Clamp the noise on [0.2, 0.8] - we try not clamp shader code as it introduces aliasing
P += x;                                          // Displace the micropolygon by x
P = P + N*x2;                                 // Displace the micropolygon by x2 in the direction of the normal
Cd = x_clamp;                                 // Finally stuff the clamped result into Colour


All this great information will obviously take me a little while to absorb, as I try to work through the examples that you've provided.
Hopefully, I'll be able to ask more intelligent questions in the future.

Thank you again for taking the time to share your experience.  Very nice of you.

G
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Volker Harun on January 19, 2011, 02:27:34 AM
In my opinion your questions were quite advanced ... and TU's answers even more ,-)
He pointed out the trying is a good approach ... I double this.

Your code looks interesting though I do nut fully understand the fit and clamp-commands for the colour information.
If you can show a resulting screenshot, it would be possible to underline TU's explanations with a workflow.

Volker
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 19, 2011, 10:40:40 AM
Quote from: Volker Harun on January 19, 2011, 02:27:34 AM
Your code looks interesting though I do nut fully understand the fit and clamp-commands for the colour information.

Heya Volker,

Thanks for the encouragement.

This is precisely the point I was attempting to make.  In my previous experience (which i'm happy to acknowledge is mostly useless in this context), the fit and clamp commands are context agnostic.
They operate on 'data' of some type - the value of the result doesn't 'become' colour until it has been assigned to the Cd global variable (final color).  Casting data from a color-vector, to a direction-vector, to a normal-vector is a trivial operation.

In this context though, they operate on a per-vector-component basis (since the result is initially a vector3).  The fit will take in the 'x' variable, break it into components (in this case x.x, x.y, and x.z - please forgive the poor choice of variable), adjust the values, and rebuild the vector.

This is definitely semantics (also the reason why I was asking about the data type that 'displacement' (in the TG context) returns).  I suppose I expected to be able to 'just multiply the result of AF by 0.5', and carry on going about my business.

Thanks again for the feedback - I will keep investigating.

Best,

G
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Tangled-Universe on January 19, 2011, 10:52:29 AM
You can perform per vector base component operations.
You would need a vector function and convert that to scalars by using the "X/Y/Z to scalar" nodes (under function node -> convert).
After that you can perform operations on each component separately and later rebuilt the vector with a "build vector" node.

These vector functions, as a base, are derived from function-node-based networks. In TG2 this means they are made of the blue function node.
As far as I know you can't extract vector specific data from a red node like the PF or AF. So breaking the output of those nodes into separate components is not possible as far as I know.

This is quite a technical discussion, so I hope Matt can chime in here some time soon to check if things thought by us (more advanced) users is still correct (though).
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 19, 2011, 11:04:31 AM
Quote from: Tangled-Universe on January 19, 2011, 10:52:29 AM
As far as I know you can't extract vector specific data from a red node like the PF or AF. So breaking the output of those nodes into separate components is not possible as far as I know.

This is quite a technical discussion, so I hope Matt can chime in here some time soon to check if things thought by us (more advanced) users is still correct (though).

Hello Martin,

Thanks again for keeping the disucssion going -- i'm learning lots.  You are very generous with your time.  If you ever decide to delve into the world of houdini, I'll be happy to return the favour.

I've recently started to suspect that the 'displacement' output of the 'red nodes' is in fact a scalar value [like a heightfield - the magnitude of displacement] (and not a vector [magnitude AND direction]).  I base this on the notion that certain nodes ask for a direction by which to displace the geometry (vertical, along normal, lateral, etc).  I could, of course, be on crack  :)

Will keep digging!

Best,

G
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Tangled-Universe on January 19, 2011, 11:45:33 AM
You're welcome, the pleasure is mine too.

Well, to be honest I never really understood the fundamental difference between a vector and scalar. Maybe you're wondering how the hell I know the other stuff then, well, please don't ask :) lol
Anyway, you've made that clear to me now, thanks :)

Your comment makes sense, but I'm not sure if the output would be scalar. I guess the internals are scalar, but being able to choose which direction you control the vector component it seems to me the output is not scalar only. The output would then be the scalar information + the direction defined by the settings you mention (you probably mean "along normal/vertical/lateral etc.")?
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: freelancah on January 19, 2011, 12:09:23 PM
This thread is very informative. Certainly sheds some light to things I've been wondering about.. I also wonder if there is a good book on the subject? Not terragen related but in general..

Keep it up guys!
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 19, 2011, 12:17:18 PM
Heya Freelancah,

One of bibles on this subject is: "Texturing and Modelling: A Procedural approach"

http://www.elsevierdirect.com/product.jsp?isbn=9781558608481
[edit: Much cheaper through Amazon: http://www.amazon.com/Texturing-Modeling-Third-Procedural-Approach/dp/1558608486/ ]
I HIGHLY recommend this book.  It's implementation agnostic for the most part, but does provide examples in Pixar's Renderman format.

Some of the author's name might sound familar  ;)

Best,

G

ps: Just as a bit of side note - it does presume a certain comfort level with math, but none of it is, pardon the reference, rocket science.

Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: cyphyr on January 19, 2011, 12:37:06 PM
Quote from: keyframe on January 19, 2011, 12:17:18 PM
Heya Freelancah,

One of bibles on this subject is: "Texturing and Modelling: A Procedural approach"

http://www.elsevierdirect.com/product.jsp?isbn=9781558608481
[edit: Much cheaper through Amazon: http://www.amazon.com/Texturing-Modeling-Third-Procedural-Approach/dp/1558608486/ ]
I HIGHLY recommend this book.  It's implementation agnostic for the most part, but does provide examples in Pixar's Renderman format.

Some of the author's name might sound familar  ;)

Best,

G

ps: Just as a bit of side note - it does presume a certain comfort level with math, but none of it is, pardon the reference, rocket science.



Pretty sure this book has been mentioned by the planetside guys :)
Thanks TU, very well answered, I kept "nearly" answering and then deleting the post, there's so much I assume and so little I actually know lol
Richard
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Volker Harun on January 19, 2011, 12:45:26 PM
 ;D

Okay, I try to add some informations based on the Perlin noise of TG2.
The node is called Perlin 3D scalar and it has three inputs.

1. The input called 'Input'. Here you normally plug in one of the 'Get position'-nodes, which give a vector of the certain point that is calculated.
You can use here a simple Powerfractal, too, as TG2 converts colour information to vectors where applicable. Of course a PF has a limited range of Colour information. Which is nice to have sometimes ,-) But just asume that we use a 'Get position' for having the 3D coordinates of the TG2 world.

2. The scale describes the main size of the Perlin noise. Using a contant scalar is nice, using a PF for a variety in size is nicer (change the colour inside the PF: Low Colour: 10, High Colour 20 and then adjaust the Colour roughness and contrast to your needs). If you feed in a vector, the Perlin noise uses the vector's length as a scalar.

3. The seed ... it needs integer numbers, any change here gives a heavy change in the Perlin noise.

The Output of the Perlin noise is nothing but a range of scalars from -1 to 1. There is no displacement anywhere, just scalars, that can be displayed as a grey-image (partly, as any scalar information below 0 is shown black). Your Perlin1d(P)

But you can plug this into a Displacement shader's Function input and change the displacement's amount by any (un-)reasonable number ;)
And of course you can use the output of the Perlin 3D noise for anything else ... plugging it into the inputs of any shader to give it a try.

To get use of the displacement but, you will need a Compute terrain node after the above mentioned displacement shader. And only a new 'Get position node will give you the vector of the Perlin 3D noise, depending on the amount of displacement in the displacement shader.
This will be your perlin3d(P).

,-) Volker
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 19, 2011, 01:02:54 PM
Quote from: Volker Harun on January 19, 2011, 12:45:26 PM
1. The input called 'Input'. Here you normally plug in one of the 'Get position'-nodes, which give a vector of the certain point that is calculated.
You can use here a simple Powerfractal, too, as TG2 converts colour information to vectors where applicable. Of course a PF has a limited range of Colour information. Which is nice to have sometimes ,-) But just asume that we use a 'Get position' for having the 3D coordinates of the TG2 world.

2. The scale describes the main size of the Perlin noise. Using a contant scalar is nice, using a PF for a variety in size is nicer (change the colour inside the PF: Low Colour: 10, High Colour 20 and then adjaust the Colour roughness and contrast to your needs). If you feed in a vector, the Perlin noise uses the vector's length as a scalar.

3. The seed ... it needs integer numbers, any change here gives a heavy change in the Perlin noise.

The Output of the Perlin noise is nothing but a range of scalars from -1 to 1. There is no displacement anywhere, just scalars, that can be displayed as a grey-image (partly, as any scalar information below 0 is shown black). Your Perlin1d(P)

But you can plug this into a Displacement shader's Function input and change the displacement's amount by any (un-)reasonable number ;)
And of course you can use the output of the Perlin 3D noise for anything else ... plugging it into the inputs of any shader to give it a try.

To get use of the displacement but, you will need a Compute terrain node after the above mentioned displacement shader. And only a new 'Get position node will give you the vector of the Perlin 3D noise, depending on the amount of displacement in the displacement shader.
This will be your perlin3d(P).

Soooo.... much... new... information...

Brain... exploding!  ;)

Thank you so much, Volker, for taking the time to add this valuable insight!  Much appreciated!

Best,

Gene
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 19, 2011, 02:50:29 PM
I think I might be getting somewhere ;)

I've attached a simple file showing a crater carved into an alpine terrain.

From what i'm able to gather at the moment:

- 'get position' returns X and Z coordinates by default
- one can separate the X and Z axis terms from 'get position' by using the 'x to scalar' and 'z to scalar' functions. (note that I've also attempted to use 'Red to scalar' and got the same result as 'x to scalar', which leads me to believe that vector data (color, position, normal, etc - is in fact one and same, despite having specific functions to deal with each)
- the Y term is 'undefined' at this stage (you can confirm this by plugging the output of 'y to scalar' into the first input of 'smooth step').

At some point Both Martin, and Volker may have mentions the 'compute terrain' node - which allows one access to the Y-term (among other useful things like normal, and slope (and i'm assuming surface derivatives dpds/dpst).  I've attempted to add it between 'AF' and 'merge shader', but it didn't seem to affect the result returned by 'get position'.

Can anyone clue me in?

Best,

Gene

[edit: just for completeness sake, in most systems the reason a distinction exists between different types of vectors (color-vector, point-vector, normal-vector, etc) is that certain operations 'transform' certain types of vectors (imagine a rotation of the terrain by 45deg).  You would want point, and normal vector to be transformed along with the operations, while 'other vectors' [ie: color] to be left alone.  I have no idea whether this is the case with TG or not.  Just speculating, and trying to fill in some blanks.   ;)
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Volker Harun on January 19, 2011, 03:18:59 PM
Quote from: keyframe on January 19, 2011, 02:50:29 PM
At some point Both Martin, and Volker may have mentions the 'compute terrain' node - which allows one access to the Y-term (among other useful things like normal, and slope (and i'm assuming surface derivatives dpds/dpst).  I've attempted to add it between 'AF' and 'merge shader', but it didn't seem to affect the result returned by 'get position'.

Well, the Compute terrain node would be nice to have below the Merge shader.
Between the Compute terrain and the Planet come the surface shaders and maybe additional functions with a separate 'Get position'.

By the way ... this is some unusual and interesting way to make a crater :)
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: keyframe on January 19, 2011, 03:25:06 PM
Quote from: Volker Harun on January 19, 2011, 03:18:59 PM
Well, the Compute terrain node would be nice to have below the Merge shader.

By the way ... this is some unusual and interesting way to make a crater :)

Heya Volker,

Is there a way to get the y-term (altitude?) that results from AF to be available BEFORE the merge shader?

G

ps: there is another issue puzzling me a little bit.  If you roll over the mouse in the 3d viewport over the inside of the crater - it seems that the height inside the crater is roughly -60m.  I would expect this height to be 0.  Any idea where the height difference is coming from?
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Volker Harun on January 19, 2011, 04:25:20 PM
I doubt that there is a convenient way to do it ... maybe there is no way at all.
One thing to try is:
Put a compute terrain below the AF and do a second connection from AF's output to this Compute terrain node. After this you can use the Get position.
Maybe this, maybe a simple Function node like 'Convert -> Length to scalar' does it. But I cannot give it a try myself right now.

The height of the crater ... the default setting of the merge shader gives you an average of the two inputs. The AF has its own height range, the Smooth step scalar gives you a colour range from 0 to 1. So the -60m is the average of the AF and the Smooth step scalar.
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Tangled-Universe on January 19, 2011, 05:08:25 PM
As far as I know you would need a "get position from texture" for the y-term to work as well.
Beware that if you use this other type of "get" function it will behave differently, which is logical as it provides different data.
There are of course also "get altitude (in texture)" functions, which only output the y-term.

FYI
Compute terrain computes the normals and texture coordinates ==> allows for slope and altitude restriction
Compute normal computes the normals but NOT texture coordinates ==> allows for slope but NOT altitude restriction
Tex coords from XYZ computes NOT the normals but only texture coordinates ==> allows for altitude restriction

Quote from: keyframe on January 19, 2011, 02:50:29 PM
At some point Both Martin, and Volker may have mentions the 'compute terrain' node - which allows one access to the Y-term (among other useful things like normal, and slope (and i'm assuming surface derivatives dpds/dpst).  I've attempted to add it between 'AF' and 'merge shader', but it didn't seem to affect the result returned by 'get position'.

I still have no idea though how to make use of these blue function nodes together with the normal red ones.

Quote from: keyframe on January 19, 2011, 02:50:29 PM
- one can separate the X and Z axis terms from 'get position' by using the 'x to scalar' and 'z to scalar' functions. (note that I've also attempted to use 'Red to scalar' and got the same result as 'x to scalar', which leads me to believe that vector data (color, position, normal, etc - is in fact one and same, despite having specific functions to deal with each)

I think this is intentional because this would allow vector data to be split in colour, position etc. so that you can use it with non-vector data and then rebuild it to a completely new function.
It adds flexibility/possibilities.

Quote from: keyframe on January 19, 2011, 02:50:29 PM
[edit: just for completeness sake, in most systems the reason a distinction exists between different types of vectors (color-vector, point-vector, normal-vector, etc) is that certain operations 'transform' certain types of vectors (imagine a rotation of the terrain by 45deg).  You would want point, and normal vector to be transformed along with the operations, while 'other vectors' [ie: color] to be left alone.  I have no idea whether this is the case with TG or not.  Just speculating, and trying to fill in some blanks.   ;)

You might find these interesting:
http://forums.planetside.co.uk/index.php?topic=2220.msg23499#msg23499 (http://forums.planetside.co.uk/index.php?topic=2220.msg23499#msg23499)
http://forums.planetside.co.uk/index.php?topic=4047.msg42267#msg42267 (http://forums.planetside.co.uk/index.php?topic=4047.msg42267#msg42267)
http://forums.planetside.co.uk/index.php?topic=3227.msg33372#msg33372 (http://forums.planetside.co.uk/index.php?topic=3227.msg33372#msg33372)
http://forums.planetside.co.uk/index.php?topic=5664.msg60374#msg60374 (http://forums.planetside.co.uk/index.php?topic=5664.msg60374#msg60374)
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Matt on January 30, 2011, 01:16:40 AM
Quote from: keyframe on January 18, 2011, 03:41:01 PM
Hello Everyone,

My apologies for the somewhat vague posting - i'm trying really hard to wrap my head around some of the concepts of the TG2 demo - and it seems that I'm failing.  Perhaps someone could shed some light on this for me.  I am familiar with PRman shader writing, and manipulating noise functions to create 'stuff'.

Hi keyframe,

Hope you're still checking in.

As you're coming from a shader-writing background I know I can skip past the basics and explain things a bit differently :)

Quote
-What data type is returned by the displacement functions? (Alpine Fractal, for example)
-What data type is returned by the 'color functions' (Power Fractal for instance)?

The shader nodes that are coloured red in the network view manipulate a wide band of data (the render/shading state), which includes things like P, Pg, Pt, N, Ng, Cd, etc. They are in some ways analogous to displacement shaders or surface shaders in PRMan, except that both displacement and surface shading are encapsulated in the same node. Some shaders, e.g. the Alpine Fractal, perform only displacement, while others such as the Lambert Shader perform only surface shading. Many do both, including the Power Fractal Shader and the Image Map Shader. (Displacement and shading are performed at different stages of the render pipeline, however.)

Low level functions are handled using the "function" nodes which are coloured dark blue in the network view. They don't become displacement or colour or anything else until they are fed into a shader, as only the shaders modify the shading state.

Since a lot of the interesting functionality of Terragen is wrapped up in the shaders without there being equivalent functions, shaders can also pass as functions, but only if they generate and store diffuse colour (Cd). Most shaders which are not solely displacement shaders will do this. For example the Power Fractal and the Image Map Shader can be used as functions, with only their colour settings and not their displacements being used. In various situations where one shader can call another shader or function, this has led to some inconsistency in terminology, with parameter names using "shader" and "function" interchangeably.

Quote
-What data type is expected as input to the displacement functions (They seem to be able to operate both with, and without input -- does having an input imply Base_fractal+New_fractal?)

When shaders are chained together through the main input labeled "input", roughly speaking they are executed in series. Most displacement shaders apply displacement additively, but this is entirely up to the shader. Compute Terrain and Compute Normal are a bit different in that they call the input displacement 3 times in order to calculate the normal.

Quote
-Can that data type (Displacement? Vector?  Scalar? foo?) be manipulated by using the 'Functions' (for instance, can I multiply the result of Alpine Fractal, by 2), and still have it be a 'valid displacement'? (my tests seem to indicate that it cannot - the minute that I append any multiply node after Alpine Fractal, my terrain disappears).

Yes, but not using the Multiply function. When you did that, the Multiply function would have cast your shader to a value, and the rule for this is to take the current diffuse colour. That would be whatever diffuse colour the upstream nodes had set.

Taking displacement shaders and manipulating them with functions like this is a current weakness of Terragen. You can do it, but it's not intuitive, because the design is biased towards chaining displacements in series. The basic idea is given here: http://forums.planetside.co.uk/index.php?topic=7548.msg80786#msg80786

Then again, if you build your displacement from scratch using functions, you can simply use them to displace the surface using the Displacement Shader. It takes a scalar function and displaces along the normal.

If you want to displace by a vector, that is possible using the Redirect Shader and three Displacement Shaders. Redirect Shader takes 3 other shaders and "redirects" them along each axis by giving them a modified normal to work with. Each of those shaders could be a Displacement Shader that takes a scalar.

Quote
-What is the relationship between the 'combo nodes' color and displacement parameters?  (PF, for instance, has both color AND displacement controls -- which suggests to me that they are handled separately -- gut instinct: displacement scalar, color vector)

Displacement can happen in any direction. A shader takes a point P and modifies it however it likes. The shader returns the modified shading state.

All displacement occurs before any shading is done. The shader tree is called twice. Because of this, the state of the surface being shaded might be different when performing shading than when performing displacement. For shaders which aim to link displacement with shading this can be a problem, but is often solved by choosing a suitable point in the displacement pipeline to calculate texture coordinates which will be used for shading. The "Tex Coords From XYZ' shader does this. The Compute Terrain shader does this too, and also calculates the normal of the incoming displacement.

A bit more info here:

http://forums.planetside.co.uk/index.php?topic=9745.msg101996#msg101996

I hope this helps. Please continue to ask questions.

Matt
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Matt on January 30, 2011, 03:37:33 PM
Quote from: keyframe on January 18, 2011, 08:37:27 PM
I think part of my confusion is that in 'our' world (Houdini), data is a little bit more fluid.

A function returns 'values' - how you map it, is completely up to the user.  You can use the result of perlin noise (whether 1D, 2D, or 3D), and map the results to displacement, color, or use as a basis for yet another function.  I suppose I was looking for an analogue in the TG world, but I suppose that's a bad approach on my part.

The same is possible using the function nodes, rather than the shaders. They can then be mapped to displacement, diffuse colour etc. using some of the shaders, e.g. the "Default Shader" has connections for diffuse, luminosity, two specularity attributes, displacement, opacity (the latter being limited to either on or off right now). The "Surface Layer" has some, but not all, of these attributes, but has additional features like "blend shader" (i.e. a mask) and some built-in slope-based and altitude-based masking features.
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Matt on January 30, 2011, 03:45:49 PM
Quote from: Tangled-Universe on January 19, 2011, 11:45:33 AM
Well, to be honest I never really understood the fundamental difference between a vector and scalar. Maybe you're wondering how the hell I know the other stuff then, well, please don't ask :) lol
Anyway, you've made that clear to me now, thanks :)

TU, a vector contains multiple values. In TG2 vectors are always 3-dimensional. The components of the 3D vector are x, y and z. As you know, we use this to represent positions or directions in 3D space.

A scalar is just a single value.
Title: Re: Functions, Colors And Displacement a.k.a confusion, pain, agony, and despair
Post by: Matt on January 30, 2011, 04:11:08 PM
Quote from: keyframe on January 19, 2011, 02:50:29 PM
I think I might be getting somewhere ;)

I've attached a simple file showing a crater carved into an alpine terrain.

From what i'm able to gather at the moment:

- 'get position' returns X and Z coordinates by default

It's also returning Y, but if you are starting with a nearly flat surface then the Y component will be very close to 0.

EDIT: Oh, in your case 'get position' Y would be changed by the Alpine Fractal, but I think the Merge Shader operates with two independent render states so that the results of the first branch don't pollute the second branch. If you want to modify the Alpine Fractal by a function that is aware of the result of the fractal, you might have to use the trick I posted in my first reply.

Quote
- one can separate the X and Z axis terms from 'get position' by using the 'x to scalar' and 'z to scalar' functions. (note that I've also attempted to use 'Red to scalar' and got the same result as 'x to scalar', which leads me to believe that vector data (color, position, normal, etc - is in fact one and same, despite having specific functions to deal with each)

Data can be cast between colours and vectors silently. However, the distinction exists so that the right thing is done when casting either of these to a scalar. For example, casting a colour to a scalar is done by taking the luminance of the colour using:


luminance = 0.2125f * r + 0.7152f * g + 0.0724f * b


A vector would be cast to a scalar by taking it magnitude or length.

The type of data output by a function is indicated by the last word in the function name, although I think there might be some nodes that omit that.

Quote
- the Y term is 'undefined' at this stage (you can confirm this by plugging the output of 'y to scalar' into the first input of 'smooth step').

It's not undefined, it's just very nearly 0 because of the shape of the surface you are shading.

Quote
At some point Both Martin, and Volker may have mentions the 'compute terrain' node - which allows one access to the Y-term (among other useful things like normal, and slope (and i'm assuming surface derivatives dpds/dpst).  I've attempted to add it between 'AF' and 'merge shader', but it didn't seem to affect the result returned by 'get position'.

The updated position generated by Compute Terrain is the texture position. This is retrieved using 'get position in texture'. Before Compute Terrain or Tex Coords From XYZ, 'get position in texture' will return the same as 'get position'.

As a general rule, you usually want to use the texture coordinates for any kind of texturing or shading. This also allows other shaders like the Transform Shader and Warp Shader to move your shader around.

EDIT: However, this still won't allow the second branch of the Merge Shader to see the results of the first branch.

Quote
[edit: just for completeness sake, in most systems the reason a distinction exists between different types of vectors (color-vector, point-vector, normal-vector, etc) is that certain operations 'transform' certain types of vectors (imagine a rotation of the terrain by 45deg).  You would want point, and normal vector to be transformed along with the operations, while 'other vectors' [ie: color] to be left alone.  I have no idea whether this is the case with TG or not.  Just speculating, and trying to fill in some blanks.   ;)

Yeah, with arbitrary attributes on some geometry this is useful. We don't really have anything like that. In the context of these function networks we only ever operate on one vector at a time, and it's nice to have it quietly cast to the type expected by the function requesting the data. The distinction between colour and spatial vector is kept so that these casts do the appropriate thing. Perhaps this will become an issue in future, I don't know.

Matt