Terragen's Node System

Started by PabloMack, October 17, 2013, 02:05:28 AM

Previous topic - Next topic

PabloMack

After using Terragen's node system for a while I thought I would share my understanding (or misunderstanding) with everybody who is interested. I am a very analytical person as many of you know from my rantings. Much of the official and wiki documentation that is available is targetted at artists who generally don't want to be troubled with theory of operation. Fred Barnard wrote that a picture is worth a thousand words. My correlary to that is that "a video is worth a thousand pictures". So in that vein I coined the expression "a principle is worth a thousand facts".

Have you ever wondered what is in that thin line that connects the "Compute Terrain" Node to the next node in the chain? What is in these connections and what is used by the nodes in these connections are some of the things I wonder about. I think of these lines as bundles of channels. Nodes can either tap into one or more of the channels or just pass them along to the next node unchanged. One of these lines can contain terrain displacement information, color information and other sorts of information. Color information alone can contain four semi-independent channels that are Red, Green, Blue and Alpha. And the kind of information that is in a channel is complicated by what its normal active domain is. For example, color information normally has end-points of 0 and 1 so you can think of a value as how full a tank is. But displacement has no pre-defined normal end-points and can even go negative. So trying to control color with a displacement value often might cause extreme unwanted results. When you attempt to connect one node to another, the different connectors often contain different sets of channels. Somehow the software knows what channels must be tapped into in another node and so this is done internally with little indication of the complex processes that are going on inside the nodes and lines. All you know is that you are connecting one terminal to another with a single line. Some of the node terminals don't contain any channels that are compatible with a certain terminal of another node so the software colors the line red telling you that what you are trying to do doesn't make any sense or that the two terminals have no compatible channels. I have also used Lightwave's node system and find it a bit easier to use because the terminals and nodes are broken out to have finer granularity. In TG, often times the data panels representing the nodes have "radio buttons" to tell the node how to interpret the data coming into them. This provides a level of flexibility to use channel information in more ways than was generially intended for use of a certain kind of channel information. One example I recently used in the Lightwave node system is to use the Y Coordinate of a surface to control the color of its surface. Here is an example of using distance to directly control the color of a surface.

archonforest

Thx for sharing your understanding about the subject. I saved it for reference. This is another piece of puzzle(data) that can help many people to understand better what is going on under the hood :)
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

Dune

I think I'm your total opposite; I don't read (so to speak of course), but learn by trial and error, and occasionally look something up if I can't find out. Intuitive, I guess. So what these lines do or contain is of no interest to me, I always thought of them as passing all information to the next node and that node will recompute the information it is able to use (so depends on what node you put there) to a next level of data. And so forth....

archonforest

I think nothing wrong with the trial and error system when u have a great CPU power and enough time to trial around :) Since I am short on both subject I want results kinda right now...Like I have an idea and I want to do it... so I will learn(manual) how to use the software and simple do what I want to achieve...
I think the lack of manual is the only reason that makes me I still stick with the free version.

Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

PabloMack

One thing I think was done better in TG is that the node system was designed up front to be the main way to control everything in your scene. Lightwave is a much older program and its node system is buried inside the surfacing part of the user interface and you have to explicitly turn it on if you want to use it. In TG it is the only way to do most of what you want done. But of course, TG is not a general purpose visualization system. It is specialized for doing realistic landscapes and it does that very well.

I can appreciate archonforest's perspective. When I have a good understanding of the tools I am using and something I want to do (often times that I have never done before), I can immediately implement it and it works exaclty as I expected the first time around. That is the sweet spot that I like to be in all of the time. But if that is all we do, we are not on the learning curve and we are not learning a new tool and that makes me appreciate Dune's perspective.  ;)

Matt

#5
The kind of data that is passed between nodes depends on the types of nodes. But we can start off by talking about shaders. Your model of how data is passed between shaders is reasonable and fairly accurate. There is a large band of data which is passed from one shader to a next. Each shader modifies one or more of those data. Exactly what it modifies depends on what shader it is (and perhaps some of the selected options). It is designed this way so that shaders can do all sorts of different tasks, and can do so in any order that the user wants.

Let's say you have a Planet with just one shader plugged into it, as in the following simple diagram:

Shader
     |
    \/
Planet

When you get down to it, the line connecting the Planet and the shader doesn't actually contain much data. It is simply a pointer back to the shader that tells the Planet "this is the shader that you should use to shade the surface". During rendering, Terragen uses that pointer to know which shader to use to modify the surface of the planet. While rendering, Terragen will render many points on the planet. For each point that it renders, there is a broad band of data associated with that point. This band of data is called the Render State. The Render State contains information about the position of the point, the surface normal, the surface colour, etc. For each point, the renderer calls the shader to modify the Render State. This is known as "shading" the Render State. After the Render State has been shaded, the information in the state is used to render the point to the screen.

http://www.planetside.co.uk/wiki/index.php?title=Render_State

So, during rendering you might think of it this way:

Unshaded Planet ==(Render State)==> Shader ==(Render State)==> Shaded Planet

Usually there is more than one shader affecting a planet. Usually, in the node network, they are chained one after the other via the "input node" plug. For example, let's say there's a shader called "Base Colours" connected to the Planet, and above this there is a "Fractal Terrain" shader plugged into the "input node" of "Base Colours", as follows:

Fractal Terrain
     |
    \/
Base Colours
     |
    \/
Planet.

This is a similar to the default project if you were to take out some of the other shaders. In this setup, "Base Colours" is used to shade each point on the planet. However, each time "Base Colours" is called, the first thing is does is call "Fractal Terrain", before doing any of its own modifications to the state. This causes Fractal Terrain to do all of its important work first, then pass the modified state to Base Colours, then Base Colours does all of its important work. The reason the renderer starts off by callling Base Colours is to give Base Colours the chance to modify the state before calling Fractal Terrain, if it wants to, thereby changing what Fractal Terrain will do. This sort of thing is done by Warp Shaders and other types of shader connections to change the texture coordinates used by the shaders that are input to them.

In the above example, you can think of the flow like this:

Unshaded Planet ==(Render State)==> Fractal Terrain ==(Render State)==> Base Colours ==(Render State)==> Shaded Planet

It's a bit more complicated with some shaders such as the Warp Shader which do some work to modify texture coordinates before they call their input shader:

Fractal Terrain
     |
    \/
Warp Input Shader
     |
    \/
Base Colours
     |
    \/
Planet

The above nodes cause a data flow like this:

Unshaded Planet ==(Render State with original texture coordinates)==> Warp Input Shader ==(Render State with warped texture coords)==> Fractal Terrain ==(Render State with warped texture coordinates)==> Warp Input Shader ==(Render State with original texture coordinates)==> Base Colours ==(Render State with original texture coordinates)==> Shaded Planet

That looks like it calls the shaders in a strange, unexpected order, but this is how the data have to flow to give the result you want when you put the Warp Input Shader between Fractal Terrain and Base Colours. It looks complicated here, but it's easy for the system to do, because the system always calls the very last shader and each shader can choose how to call its input and what to do at each step. Even so, the node layout is quite simple.

Now, with other types of nodes, the connections may be used for other things. For example, a populator node has a connection to an object node, as in this example:

OBJ Reader
     |
    \/
Populator v4

Obviously the OBJ Reader can transmit object data to the populator, but the connection does more than that. As with the shader connection to the planet, the connection is really just a simple pointer back to the object node, and the populator may use this connection in various ways. One of the things it does is to change certain attributes on the object node automatically, such as settings its "population mode" to "Member of a population". A connection is not just a pipe for a specific kind of information - it is a way for the user to tell the system which nodes are supposed to interact with each other, and the system uses this information in ways that are most appropriate for the given task.

I haven't covered Function nodes (blue nodes) here. In some ways they are much simpler because they usually only modify a single piece of data (a vector of 3 values for vectors and colours, or a single value for scalars), but the system can automatically convert between these simple values and complete Render States depending on how the nodes are used. But I'll link to this as a starting point:

http://www.planetside.co.uk/wiki/index.php?title=Node_Input_Type_Conversion

Matt
Just because milk is white doesn't mean that clouds are made of milk.

archonforest

Thx Matt, I do appreciate your write up...
Dell T5500 with Dual Hexa Xeon CPU 3Ghz, 32Gb ram, GTX 1080
Amiga 1200 8Mb ram, 8Gb ssd

PabloMack

#7
Thank you Matt. Some good information here. I'll have to study it in detail.