Planetside Software Forums

General => Terragen Discussion => Topic started by: bigben on February 26, 2008, 06:50:49 PM

Title: Rainbow
Post by: bigben on February 26, 2008, 06:50:49 PM
Hi All

I've been a bit busy lately but I've managed to still do a bit with TG. While having a break from sorting out my displacements for 90m SRTM terrains I've been working on a rainbow "model". I've actually had two models in mind. I started with my second idea as it was the simplest to construct but it didn't work  :(  I have my suspicions as to why (possibly a glitch with get camera that I experienced withmy population masking tests) so I'll revisit this next.

The image below is a proof of concept of my first model. The rainbow is deliberately too wide... that will be addressed later.  The result is very encouraging as the variation in lighting is really nice. 

The final model will consist of 4 duplicate cloud nodes (1 for rain, 3 for the rainbow), a single cloud density shader and four masks. If I get my second model working, it will provide realistic rainbow placement for animations where a cloud layer is used to simulate falling rain (or flying above clouds). The only thing it won't be able to handle accurately is losing the rainbow in the shadow of a cloud, but other than that it has a lot of potential. It can easily be expanded to produce a complete rainbow as well as sun dogs with a little tweaking.
Title: Re: Rainbow
Post by: monks on February 26, 2008, 07:05:02 PM
Very cool! It'd be interesting to see how this will look with all the bells and whistles on- could be sublime!

monks
Title: Re: Rainbow
Post by: dhavalmistry on February 26, 2008, 07:13:53 PM
cool bigben....is this all procedural??...based on your explanation...my guess is it IS procedural, but would like more explanation on this one...looks promising...keep up....
Title: Re: Rainbow
Post by: rcallicotte on February 26, 2008, 08:02:08 PM
Oh.  Oh.  Ooooh.
Title: Re: Rainbow
Post by: bigben on February 26, 2008, 08:13:05 PM
Both models are entirely procedural. This example is a relatively crude model and requires some manual hacking of settings to position accurately.

The second model I was working on might require a single hack as a workaround to a specific problem, but this may be remedied with the next update. The second model will be nerdishly accurate.

Construction of this model:
The basis of this model is a spherical gradient from 0-1 at a set distance from a specified point with a fixed width. The sphere is then sliced perepndicular to the camera to produce a torus-like shape (I used my terrain blending mask, but a couple of distance shaders would have done the trick)

This is then fed into 5 colour adjust shaders, 1 for each change in RGB values to make up the spectrum (2 red, 2 green, 1 blue). These are then combined to make 3 shaders representing the RGB values of the rainbow.

An additional colour adjust node sets the entire width of the rainbow to 1. This is used as a blending shader for the cloud density shader. The density shader is then fed into 3 surface shaders (as a child), using the R,G,B shaders above as blending shaders.

A cloud layer is then created for the rain and duplicated. In this case I just have the rainbow without additional rain, so there are 3 cloud layers. The colour of the cloud layers are set to red, green and blue, using the corresponding surface shader as a density fractal.

[attachthumb=#1]

This shows you the construction of this model. No rocket science here, just some crude (but creative) masking  ;)

The bottom 2/3 (what to do with the 0-1 gradient) of this is pretty much sorted  and is common to both models. There are a couple of extra bits left from previewing the result on the terrain as it's quicker to preview as a surface layer.

What's next

The second model will actually use the position of the sun to produce a conical mask spreading out from the camera. This will remove the need to manually position the centre of the sphere and then slice the sphere.

The rest is basically all thanks to the volumetric cloud model in TG2, and with some finetuning should produce some very realistic looking rainbows.

Both models should easily slip into an existing scene that uses clouds to simulate rainfall.
Title: Re: Rainbow
Post by: bigben on February 27, 2008, 04:49:38 AM
Turns out I didn't have a clue what I was doing with the second model  ;)  Back on track now after a quick refresher course...  http://en.wikipedia.org/wiki/Dot_product (http://en.wikipedia.org/wiki/Dot_product)

Here's the 0-1 gradient applied to a surface shader (now with the correct angles).  A second rainbow can be added by duplicating the network from the inclusion of the inner radius down. I'll construct the rest of the network next and from there the next thing to do will be to calculate the coordinates of a point opposite the sun using the camera position, sun heading and sun altittude.

At the moment this is manually calculated by turning the camera away from the sun, adding a sphere and then copying the coordinates of the sphere. Need a bit more refreshing of my math to find the simplest solution  ;) This will remove the need for any manual input for the rainbow!

I have a couple of higher quality renders of the first model still running including a close up of one end of the rainbow. Still on the pre-render  :-\
Title: Re: Rainbow
Post by: Matt on February 27, 2008, 08:16:12 AM
Quote from: bigben on February 26, 2008, 06:50:49 PM
The only thing it won't be able to handle accurately is losing the rainbow in the shadow of a cloud

If I understood your explanation correctly you're using 3 coloured clouds to simulate the red, green and blue parts of the rainbow. In that case it should darken automatically when it falls into shadow. The only problem is that it's also illuminated by other lights, not just the sunlight. If the only lights in the scene are a single Sunlight and Enviro Light, then all you need to is tell the rainbow clouds not to respond to Enviro Light. In the cloud layer's lighting tab you could set "Enviro light" to 0.

Matt
Title: Re: Rainbow
Post by: ghotir on February 27, 2008, 10:37:51 AM
Quote from: Matt on February 27, 2008, 08:16:12 AM

If I understood your explanation correctly you're using 3 coloured clouds to simulate the red, green and blue parts of the rainbow. In that case it should darken automatically...

Then wouldn't he have dark clouds instead of colored clouds.  It needs to go transparent when it is shadow.  That is what rainbows do. 

Title: Re: Rainbow
Post by: bigben on February 27, 2008, 01:53:16 PM
Yes the rainbow will darken when in shadow but it should disappear. How much it darkens will remain to be seen but any darkening will help to keep it looking realistic. Having envirolight settings for individual cloud nodess as you mentioned has been very handy and should help a bit.

Adding a secondary rainbow and Alexander's band (http://en.wikipedia.org/wiki/Alexander%27s_band (http://en.wikipedia.org/wiki/Alexander%27s_band)) which will help to make it loook even more realistic. I've also been tossing up whether to slightly lighten the first few degrees inside the primary rainbow.
Title: Re: Rainbow
Post by: bigben on February 27, 2008, 05:08:22 PM
The high quality renders of v1 are finally onto the rainbow. I'll post partial renders when there's enough to see.  I found a typo on one of the shaders which affected the red.

Below is a preview render of a double rainbow (another productive train ride to work, and as luck would have it there was a double rainbow outside for reference :) ). The combined RGB channels from the rainbow masks were simply added to a build colour node and applied to the terrain.

I had to reorder the masking sequence a bit as the existing sequence resulted in the area outside of the rainbow being purple which is OK when your masking a single rainbow, but screws things up when you add two rainbows together.

The even widths of the colour bands didn't look natural. Looking at the reference images the red end of the spectrum has wider bands. Rather than recalculate new values for the gradient values I simply added a colour adjust to the initial 0-1 gradient and then adjusted the gamma.  I also have to widen the violet band a little. Once I get the rainbow looking nice I'll recalculate the gradient steps for a minor efficiency improvement.

The secondary rainbow uses a separate set of colour adjust shaders to create the RGB gradients, with the black and white values being reversed from the primary rainbow (1 - primary value). Each channel is then multiplied by a darkening factor (0.3 in this image) to reduce its intensity.

I'll construct the extra bells and whistles tonight and then review the masking to try and find the simplest method. This will be a good test for boolean nodes ;)

The render times are long, but not as long as I had expected from some of my earlier cloud masking tests and it looks really cool which more than makes up for it. I will share the TGD eventually but I would at least like the chance to get one good image out of it first ;)
Title: Re: Rainbow
Post by: bigben on February 27, 2008, 05:58:53 PM
Here's a partial render of the end of the rainbow. The gap where yellow should be is the result of the typo I made. but the way the intensity changes depending on the background is sweeeeet.

The fall off at the red edge is partly due to the fact that I've used a rectangular mask to slice the rainbow, resulting in less cloud being visible at the edges. With the second model this isn't be a problem as the mask is projected radially from the camera.
Title: Re: Rainbow
Post by: Cyber-Angel on February 27, 2008, 06:52:18 PM
The only thing I'd say is that in nature the inner secondary bow tends to be much fainter than the primary outer bow and you are correct in your image of you secondary bow that the colors are reversed: nice experiments.  ;D

Regards to you.

Cyber-Angel     
Title: Re: Rainbow
Post by: bigben on February 27, 2008, 07:21:58 PM
I agree that it should be fainter. Luckily I built in a tweaking mechanism via a constant scalar. This could also be used to fade in the secondary rainbow during an animation...

There will also be a need to restrict the thickness of the rainbow with a distance shader (from the camera) or the build up over distance will result in a fairly solid rainbow.

The other bells and whistles tat I'll be adding will be a lightening of the rain inside the primary and outside the secondary rainbows, a darkening between the primary and secondary rainbow, and perhaps a single supernumerary.
Title: Re: Rainbow
Post by: Matt on February 27, 2008, 09:01:35 PM
Quote from: ghotir on February 27, 2008, 10:37:51 AM
Then wouldn't he have dark clouds instead of colored clouds.  It needs to go transparent when it is shadow.  That is what rainbows do. 

Ah... true. The cloud would need to be rendered additively for that to work. But an interesting trick you might try is to make the RGB colours much brighter and reduce the density. For example use a colour intensity of 100, and then divide your current density value by 100. That will make it almost entirely additive 8)  If that's not enough to make the shadowed regions invisible, use a larger factor.
Title: Re: Rainbow
Post by: Oshyan on February 27, 2008, 09:11:15 PM
I just want to say this is really cool and the results are surprisingly good. Go Ben! :)

- Oshyan
Title: Re: Rainbow
Post by: rcallicotte on February 27, 2008, 09:33:50 PM
Once again, the Lone Ben rides again!   ;D  Way to go!
Title: Re: Rainbow
Post by: bigben on February 27, 2008, 09:46:47 PM
Quote from: Matt on February 27, 2008, 09:01:35 PM
Quote from: ghotir on February 27, 2008, 10:37:51 AM
Then wouldn't he have dark clouds instead of colored clouds.  It needs to go transparent when it is shadow.  That is what rainbows do. 

Ah... true. The cloud would need to be rendered additively for that to work. But an interesting trick you might try is to make the RGB colours much brighter and reduce the density. For example use a colour intensity of 100, and then divide your current density value by 100. That will make it almost entirely additive 8)  If that's not enough to make the shadowed regions invisible, use a larger factor.

Thanks Matt

That sounds like a promising approach. A very quick test on the original render seems to produce a similar looking rainbow in sunlight.  From a previous test with the second model it looks like I'll have to find a balance between (horizontal) cloud thickness and density to maintain a certain level of transparency. I'll throw a shadow into the mix when I'm testing these combinations.

Quote from: Oshyan on February 27, 2008, 09:11:15 PM
I just want to say this is really cool and the results are surprisingly good. Go Ben! :)

- Oshyan

I'm starting to find the limitations of this hack but it's working much better than I first thought it would. I think adding the lightening inside the arc of the primary rainbow will be one of the biggest improvements towards a more realistic rainbow.
Title: Re: Rainbow
Post by: bigben on February 28, 2008, 02:42:43 PM
Another step forward. Replaced the conditional shaders with Boolean scalars for creating the edges of each region and removed all redundant clamping. This has been quite helpful, providing access to the original gradient to extend it either direction. This simpified the process of adding additional gradients to simulate lightening of the sky past the blue edge of the rainbow.

Each of the boolean scalars is also inverted by a boolean NOT to provide a complete set of edge masks. These are combined using a multiply or add nodes to mask different areas as required. I'll set something up on a cloud layer today to let it render over the weekend at work. The image below is still a surface preview.

PS my copy of XFrog arrived yesterday  :)
Title: Re: Rainbow
Post by: bigben on February 28, 2008, 08:31:11 PM
Matt is a genius  ;D

Here's a cropped render of a 300m thick double rainbow in shadow  8) 8) 8)
(well the effective thickness is only 200m, with fuzzy edges toward and away from the camera)

Cloud density 5e-4
RGB colour 100

It's going to be a balancing act between the total thickness of rain/distance shader high colour AND cloud colour/cloud density but I'm thinking that once I have the cloud/desnity colour sorted the placement of the rainbow will require keeping the cloud thickness (laterally from the camera) X distance mask colour constant to provide a constant density.

I have other nodes that could be used for adjusting the brightness of the rainbow in an animation.

Rendering a comparison image without the background clouds now so that you can see where the rainbow is. (You can partly work out it's location from the image above)
Title: Re: Rainbow
Post by: bigben on February 28, 2008, 11:17:34 PM
OK, apart from the levels/density etc needing a bucket load of fine tuning the general principle of the model is working. This is the same TGD as the previous post with the only change being the disabling of the clouds that were casting shadows on the terrain. It's going to take a little while to get familiar with the relationship between the different numbers and the way they render in the atmosphere.

Outer edges of rainbow may have to be softened (easy)
RGB gradients may be better as non linear to show secondary colours better (adjustment nodes already in place)
Not sure what happened with the inner and outer lightening, but they look like they got swapped around...  ??? Possibly from variation in the cloud's density shader which you can make out.

I sped up render times by simplifying the density shader for the cloud. 8 cloud samples for a quality of 1. There is some noise so I may need to adjust the cache/qualtiy settings some more.
Detail 0.5, aa 3, GI 2,2
Title: Re: Rainbow
Post by: bigben on February 29, 2008, 04:33:37 AM
The last image needed a lot of fixing...

Another little step closer... tweaking the primary rainbow for now. I probably should add the rest of the rain that's supposed to be producing the rain as well.  The sharp edges will definitely have to be softened, and maybe changing to slightly less saturated shades of the primary colours.

This image better illustrates the lighter region beyond the blue edge of the rainbow.

Title: Re: Rainbow
Post by: rcallicotte on February 29, 2008, 08:20:56 AM
Oh my God!  You are a busy man, you are.   :o
Title: Re: Rainbow
Post by: bigben on February 29, 2008, 06:04:29 PM
nah... just a nerd with a bone ;)  I tend to post a lot while doing stuff like this on the off chance that someone has an idea that will help (like Matt's wonderful suggestion) If I had waited until the end to post I would have had a lot of reworking to do.

Selecting less saturated colours is a) not possible with values >1  and b) not required as adding the surrounding rain tones things back a bit to a more realistic level. Replacing the Boolean masks with fuzzy-edged masks should just about finish the primary rainbow. The secondary rainbow should just be a case of transferring the new levels for the rainbow gradients and then finding a suitable intensity.

I'm tempted to add a single supernumerary rainbow just to show off what TG2 can do. I'll fade it with a horizontal gradient so it willl be stronger at the top of the rainbow and then fade as it goes left and right.

The whole process is basically an extension of my multiple masks method with the only difference being the the edge effects are required this time. After calculating the angle from the anti-sun vector the math is no more complicated than Y=mX+c to position gradients for the RGB values and additional masks
Title: Re: Rainbow
Post by: bigben on March 01, 2008, 11:24:30 PM
Finally made the obvious connections required to simplify the whole setup. I had a mixture of colour adjust nodes and manually generated gradients because I was only thinking of the colour adjust node within the normal context of adjusting colours rather than the way it actually functions. This approach also required additional clamp scalars and extra masks.

Adding soft edges to the rainbow made some of the masks redundant as all RGB values would drop to 0, but the process of creating the extra gradients seemed unnecessarily complicated. Eventually the obvious answer clicked.... I was doing things in two stages, while the colour adjust node could do the exact same thing in one easy step. All I had to do was calculate the actual angles for the colour changes and use these for the black and while levels in the colour adjust shaders, clamping to 0 and 1.  The node network pic shows the new node structure for te primary rainbow and the old structure of the secondary rainbow.

Added some "rain". The rainbow is still a bit too distinct, so an additional broader softening of the edges may be necessary.
Title: Re: Rainbow
Post by: Cyber-Angel on March 01, 2008, 11:48:27 PM
If it helps refine your colors any, I found this gallery of rainbow images that maybe of assistance http://www.capetownskies.com/rainbows.htm. If you can find away to soften your colors and the transition between bands that I this would make yours more realistic from a visual perspective (IMHO), but this is a WIP so I understand that there is still much work left to do and maybe you have already considered these factors.  ;D

It is a shame that Terragen doesn't support Refraction and Dispersion in its lighting model, but maybe it will at some point in future which would make life easier for trying to do these types of natural phenomena.

Regards to you.

Cyber-Angel   
Title: Re: Rainbow
Post by: bigben on March 02, 2008, 05:01:28 PM
There's lots of reference images around  ;)  tweaking is proving a little tricky, although I've pretty much stabilised the design of the model now so this should help.  As the construction is pretty much made up of distinct components, there are usually only one or two nodes that need to be adjusted at any one time.  I've added a fade shader to allow for fading of the rainbow from the center to the edges.

I also haven't tried a render with GI yet, and it will be interesting to see what effect the rainbow has on the surrounding rain (possibly not much for thick rainbows). This may help soften things up a little.

My trig was not as rusty as I had thought, once I got the angle calculation correct the automatic calculation of the anti-sun vector worked (I had dropped it earlier it for a manual vector just to be sure). I've left the manual node in as a fallback in case it provides a minor efficiency boost for a stationary sun). The coordinates are calculated based on a 1 meter vector in the XY plane pointing towards the sun. At least now the rainbow is effectively attached to the camera so you can go chasing rainbows around your scene.

Below is the current node structure, which is  simplified a fair bit from earlier versions despite doing a lot more. It's also easier to locate specific controlling nodes. The bulk of it consists of just colour adjust nodes that are added or multiplied together.

I'm running another test render at the moment of a complete double rainbow with "rain" and background clouds but it's going to take a little while. I can already see things that need tweaking but it will be good to see what else is working...
Title: Re: Rainbow
Post by: Cyber-Angel on March 02, 2008, 06:44:37 PM
As to reference images yes there is a large amount, I found that gallery just from a cursory search of Google Images which I posted only to be of help. Other than that I really like the work your doing here which shows just how powerful the node network can be.

Regards to you.

Cyber-Angel  ;D
Title: Re: Rainbow
Post by: bigben on March 05, 2008, 05:30:56 PM
A little more progress although unfortunately no images to show at this time. I have a completed render but I can't get it off the laptop as Explorer has spat the dummy and locked up on an error message (not related to TG).  The 2 TG renders are still running fine though, so I'm waiting for them to finish before I give it a "kick".

The one thing that bugged me with some of the tests was that the faded rainbows seemed to lose the secondary colours. On reflection this is probably related to the hack of using a low cloud density with high colour values. Simply multiplying a linear mask by a fixed factor to reduce its density will drop some of the lower densities to a point where they disappear.... so that the more a rainbow is "faded" the more the colours will be reduced to just red, green and blue. To get around this I thnk I'll have to set a minimum density for the rainbow masks and then multiply the remainder by the mask value.

Normal rainbow = min value + mask value x (1 - min value)
Faded rainbow = min value + fade factor x mask value x (1 - min value)

The value for 'min value' will depend on the cloud density/colour values and will require some testing to find out. All of the other tweaks I tried had problems that were related to this, so hopefully this should just about complete the model.
Title: Re: Rainbow
Post by: moodflow on March 05, 2008, 06:36:35 PM
Woah, those are some serious connections!  Any more and we'll need a machete to get through those nodes. 

Nice work BTW!   8)
Title: Re: Rainbow
Post by: FrankB on March 06, 2008, 05:11:31 AM
I'm just amazed about the level of creativity you are demonstating, Ben! Respect!

Cheers,
Frank
Title: Re: Rainbow
Post by: bigben on March 06, 2008, 03:34:58 PM
Thanks guys

Here are the renders I had been waiting for.
[attachthumb=#1]
This is a full double rainbow with an intersecting cloud. As the sun is behind the camera you don't really see any efects from the clouds shadow on the rainbow.  The secondary ilustrates the problem I mentioned above. The rainbow has been reduced to just the primary colours. The lightening effect inside the rainbow is getting better.

[attachthumb=#2]
This is a close up of the lower left end of the rainbow showing the gap between the red and green bands in the secondary rainbow.  I'll have to finalise the density settings first before I deal with the noise. Quality settings were set reasonably high for this (hence the long wait). Fixing the minimum density should also make the colour blending much smoother across the primary rainbow.

[attachthumb=#3]
This was more of a positioning test than anything else. The wall'o'rain was restricted to just in front of the camera rather than having the camera in a sphere'o'rain so that the full effect of the sunlight could be seen. There are shadows from the background cloud layer cutting off the rainbow at the top and you can just make out the secondary rainbow.

Having the rainbow attached to the camera makes positioning a breeze. The output of the RGB masks were fed into a Build colour node which was then connected as a child to a surface layer. This gives a quick preview of where the rainbow will appear on the terrain, and it was then just a case of moving the rain to the right distance/position. Just a "quick test" on a Mt St Helens terrain.

Now to figure out the various density settings, especially the minimum density for the cloud masks.
Title: Re: Rainbow
Post by: rcallicotte on March 06, 2008, 06:27:12 PM
Excellent work.  Who would have even thought this was possible last year?  I didn't.
Title: Re: Rainbow
Post by: bigben on March 08, 2008, 02:16:25 AM
This is definitely heading in the right direction.  This image uses a greyscale step wedge image to trial a series of numbers for the minimum value of the RGB density masks (faster than rendering an animated sequence) They range from 0.5 at the top to 0 at the bottom in .05 increments. As you can see it gets smoother towards the top although the overall density seems to have dropped a little as well. This can easily be fixed by increasing the cloud colour to compensate.

I had also considered that some of the variation may be due to the fact that the total sum of the three mask values varies from 1-2 (e.g. red =1, green = 1, yellow=2) but this does not match the pattern of gaps so I'm still not really sure what is going on. I will try a render with mask values adjusted so that the sum of the RGB mask values always = 1 but I'm not sure whether it will improve things (although it should make a smoother colour gradation).
Title: Re: Rainbow
Post by: bigben on March 08, 2008, 04:54:27 PM
Set the minimum density to 0.5 and used the stepwedge on the cloud fractal this time so see the changes in the rainbow with density.  As you can see in this there is a sudden drop off at 0.4, so the density fractal could use this as it's low colour.

Changing the gradient to provide a sum of densities of 1 was a flawed idea as the gaps return too quickly. Reverted back to the previous gradient, but there may still be the problem of the rainbow degrading to just RGB. A worst case scenario may mean having to add cloud nodes for the secondary colours as well but I want to avoid that if possible.

Running a new test with some cloud tweaks to reduce the noise.

The build up of density outside of the rainbow is due to the combined RGB (minimum) densities. I haven't masked out the edges of the rainbow yet as this is providing me with some useful information for now.
Title: Re: Rainbow
Post by: j meyer on March 08, 2008, 05:06:21 PM
Hi-ho Silver,ride on! ;D
Title: Re: Rainbow
Post by: bigben on March 10, 2008, 07:07:55 AM
Well I got a little side-tracked when I adjusted 1 or 2 cloud settings too many (at once) and broke things with a weird result. This wasted a bit of time but I've gone back to my last successful test and am moving forward from there.  The next test render is still running, using quality settings more along the lines of what I would use with a nearby tree in the scene. It's looking relatively good.

There's still a bit of noise but I think it adds to the scene and I won't try to remove it entirely. Still tossing around ideas on how to fade the density nicely without the rainbow falling apart. Adding a minimum value helped a lot. I had experimented with tweaking the gamma of the gradients but this is flawed because this involves increasing the slope of each part of the gradient which exacerbates the problem. The latest possibility I'm considering is increasing the range of the gradient by extending each black level beyond its theoretical cutoff. This will mean that the secondary colours will actually have a total density greater than the primary colours. This will only be a small difference but it may be enough to retain the secondary colours long enough to keep them visible as the rainbow fades.

Other than that I think I've just about exhausted the possibilities and it may be a case of putting up with it or adding cloud nodes for the secondary colours as well, but that's a frightening prospect from the point of view of render times. Hopefully another image tomorrow  ;)
Title: Re: Rainbow
Post by: Matt on March 10, 2008, 05:12:07 PM
Hi Ben,

I'm not sure I fully understand the density fading problem you're talking about, but maybe it has something to do with the built-in density profiles in TG2's cloud layer. They reduce density in the upper and lower parts of the cloud layer, and there is a variable cutoff threshold which could cause problems. You might need to use a much deeper cloud layer that's centred around the altitude you're most interested.

I think you need a much wider band for each of the channels. I'd expect to see a lot more yellow, and maybe some more cyan. The human eye response to red and green overlaps a great deal.

http://en.wikipedia.org/wiki/Cone_cell

Obviously there are other factors to consider, such as the fact that the RGB components coming from your monitor don't match the eyes cones' responses, but I'd start with an approximation of the eye's RGB response. Unfortunately that won't give you violet because the blue of a computer monitor doesn't look violet, but I think that's the only special case you need to worry about (by adding a touch of red as the blue fades out). Does anyone know how it is that digital cameras are able to reproduce violet in rainbows? Do they perform a similar trick to remap pure blue to something that appears more violet in RGB space?

Matt
Title: Re: Rainbow
Post by: bigben on March 10, 2008, 07:06:20 PM
Thanks for the feedback Matt

The density fading problem is related to the hack of using a very low cloud density and high colour values to have the rainbow disappear in shadows.  This also makes it disappear at lower cloud density values, and because the secondary colours are made up of reduced density masks applied to separate cloud layers, these transitions are the first to disappear. It's a trade off then to keep the mask density high enough to keep the rainbow visible while still providing the variation for the colour gradient.  It's not a problem with TG2 as such, more a drawback of the model I'm using to build the rainbow. 

In the previous image you can see how the rainbow is almost straight RGB (density starting point was a bit too low to properly show secondary colours), and as the cloud density decreases further you get gaps appearing between the bands until everything disappears before the density reaches 0.

If you could apply a colour function to a cloud it would simplify things a whole lot  ;)

I'm dealing with violet by adding a reduced red band at the end. Still a bit too bright, but getting there. I had also noticed in pics of rainbows that the bands got wider towards the outer edge. The angles I've used for the gradients provide a uniform increase in width for each colour band.

My laptop crashed halfway through (no GI), but the render I left running at work (with GI) is nearly finished. I've taken a screengrab just in case although it's only using 700Mb  of memory (180 RAM, 530 VM) so I'm not expecting any problems at this late stage.

Detail 1, AA 6, GI 1,3 (0 GI on rainbow nodes)
  [edit] render's finished now [/edit]
Title: Re: Rainbow
Post by: bigben on March 10, 2008, 08:16:50 PM
Also following the Cloud Noise at distance thread and learning a bit more about clouds...

Making some headway with cloud scattering colour (was set way too low, probably the main cause of the noise(?))
Lowering edge sharpness also seems to be helping but will have to run a high res render to check...

Moved the rainbow to a new file with a "real" terrain and new cloud to see how portable it is. Mostly works fine. Including the cloud nodes for the rainbow wasn't as easy as duplicating the existing cloud node and then tweaking each colour. It seems to be more successful customising it to suit the original cloud node that it is being added to, but the process is pretty straight forward.

Running a medium res render which will include a terrain shadow across the rainbow demonstrating how the rainbow builds up with distance from the camera.
Title: Re: Rainbow
Post by: bigben on March 11, 2008, 04:30:06 PM
Forgot to connect the cloud density shader to the rainbow, so this shows a full density rainbow. The noise is greatly reduced, and the transitions are a lot smoother. The hacks to bump up the minimum density have resulted in a lighter edge to the rainbow which suggests that I can bring the gradients towards their original forms.  So it's another step backwards for now to revise and rebuild based on the latest tests.

I've played around with a few different cloud settings and the latest process goes something like this:

Duplicate cloud node, attach rainbow mask X cloud density shader X distance restriction
Light propagation = 20
Cloud Colour = R,G or B and increased to 100
Scattering colour = 50-100% of Cloud colour (still testing)
Edge sharpness = 1% of original value
[edit] and Enviro light (& tint) = 0 [/edit]

The rest of the cloud settings (including Cloud density) are left at their original value (but this may need to be changed later)

The depth of the rainbow has been increased 10x in this image from previous tests (now 3km)  Some preliminary tests indicate that the darkening of the secondary rainbow is working much better as well. Back later when I have something slick to show.

Title: Re: Rainbow
Post by: Mohawk20 on March 11, 2008, 04:50:17 PM
It's looking pretty good allready! This is going somewhere.... somewhere nice!  ;D
Title: Re: Rainbow
Post by: rcallicotte on March 11, 2008, 05:41:13 PM
This is nice, Ben.  So...educational, beyond my brain energy level.  For now.

Thanks for letting us in on all of this.  I come back to your explanations from time to time, when I get stuck.  It's all been very helpful.
Title: Re: Rainbow
Post by: bigben on March 13, 2008, 07:01:50 PM
Quote from: bigben on March 11, 2008, 04:30:06 PM
... Back later when I have something slick to show.

I lied ;) I has started a render with GI but I accidentally disconnected one of the primary rainbow masks which made the entire rainbow disappear :(  The pre-render took a long time.

Here is an incomplete render of the new, improved double rainbow. Detail 1, AA 2, GI 0,0. The primary rainbow has the original (theoretical) gradients. 

The secondary rainbow has two extra tweaks;
The minimum density of each has been increased by multiplying the gradient by 0.5 and then adding 0.5 to the result
The black levels of the internal gradients (not going to the edges) have been shifted to the next secondary colour (red = 0 at cyan insted of green, blue = 0 at yellow instead of green, green = 0 at violet instead of blue).

The secondary rainbow seem to be holding together much better at lower densities and I may apply the second tweak to the primary rainbow before starting a render with GI.  I may have to extend the red and green black levels beyond the red edge of the rainbow to lighten the yellow as it's gone a little orange. See... I knew there was a reason for posting this... Stopped the render to apply the revised black levels to the primary rainbow and discovered that there were some old tweaks to the gamma of the gradients on the secondary rainbow. These would have been largely responsible for the reduced amount of yellow.

The gap in the primary rainbow looks a little odd although I'll have to reserve my judgement until after seeing a render with GI which will lighten the shadow. It's caused by a shadow, but the thin bit of rainbow in front of the shadow has disappeared. The difference in intensities is probably too great due to the depth of the rainbow (3km). Using a thinner rainbow would fix this... probably around 300-500m, although this would make it a little harder to position the rainbow.

There is no darkening between the two rainbows in this example, although I have a mask that can be used for this. This area will look darker anyway due to the lightening applied inside the primary and outside the secondary rainbow.
Title: Re: Rainbow
Post by: bigben on March 14, 2008, 12:43:22 AM
Here's a sneak peek at a larger render with GI.


.... OK so it's the end of the week and it's bloody hot here... must be dehydrating   ::) ;)
Title: Re: Rainbow
Post by: rcallicotte on March 14, 2008, 07:55:01 AM
 ;D
Title: Re: Rainbow
Post by: bigben on March 14, 2008, 08:36:21 PM
Here's the double rainbow so far.  The primary doesn't have a minimum density set this time. I prefer the version with a minimum density as it's a more subtle.  Render quality settings also make a difference.  Increasing AA makes the rainbow a lot smoother and increases it's transparency... (larger cropped render in progress)

This image Detail 1, AA 2. Next test is Detail 0.5, AA 6 and is looking much nicer. Also tweaked some of the gradients to try and improve the colour distribution
Title: Re: Rainbow
Post by: Mr_Lamppost on April 06, 2008, 09:03:44 PM
I sort of let this one slip past while I was engaged on other things.

Amazing results. When are we going to see a completed scene? Is ther light at the end of the rainbow?  ;D :-[
Title: Re: Rainbow
Post by: rcallicotte on April 07, 2008, 09:32:49 AM
This could be very useful.  I would never have thought such a thing was even possible in TG2.
Title: Re: Rainbow
Post by: bigben on April 29, 2008, 05:52:19 PM
Hi All

My relative silence here has been due to life taking precedence over tinkering, but I have managed to squeeze some time back in.  I may tinker a bit with the alpha on this one but I have a few things to sort out first.  Had a play with clouds again and have what I think may be some suitable clouds to go above the rainbow. 

I think the gap in this project has been useful in that I have to rethink what was going on, and some of the cloud/fractal settings are making more sense to me and I seem to have some control to change the clouds to what I want ;)

I'll hopefully have something to show after the weekend.
Title: Re: Rainbow
Post by: Mohawk20 on April 29, 2008, 05:59:57 PM
I hope so, you're doing a great job!
Title: Re: Rainbow
Post by: bigben on April 30, 2008, 07:14:46 PM
I did a test render of on of the latest TGDs (http://forums.planetside.co.uk/index.php?topic=3517.msg37111#msg37111 (http://forums.planetside.co.uk/index.php?topic=3517.msg37111#msg37111)) in the alpha and the increase in render speed was amazing.  This has been the first time that I've seen my rainbow rendered with detail 1, AA 6 and GI (not applied to rainbow nodes). The complexity of multiple cloud layers had really slowed things down but now it's actually practical to use higher quality settings.  I'll benchmark it later when I have some time just to make sure it wasn't just my fading memory  ;).
Title: Re: Rainbow
Post by: bigben on May 05, 2008, 05:02:10 PM
[attachimg=#1]

Detail 1, AA 6, GI 1,2 (alpha).... and no, it doesn't go any bigger  ;) Short of the odd nap while I had to do some work, the render time is somewhere around 20 hours... somewhat longer than the previous quick test.

This has a new cloud layer and I restricted the rain to the same depth as the rainbow (all restricted by two distance shaders). Needs to be darkened a bit. 

Tweaking the intensity is somewhat difficult to predict at the moment, but this test probably shows the upper and lower limits. The primary rainbow needs to be darkened a little and the difference between the primary and secondary rainbow needs to be decreased a little (controlled by a single multiplication factor)

The only other thing to test will be the sharpness of the edges... and maybe checking the relationsip between cloud samples and the noise of the rainbow as it "fades". Time for some cropped larger render tests



Title: Re: Rainbow
Post by: Tangled-Universe on May 05, 2008, 05:27:39 PM
Amazing work Ben, admire your node-creativity and knowledge :)
It's a pity these complex setups take so long to render, but that's inherent.

Martin
Title: Re: Rainbow
Post by: bigben on May 05, 2008, 05:50:59 PM
In this case it's not entirely the fault of the complex network.  It looks complex but it's mathematically quite simple. Applied to a surface it renders quite painlessly  ;). In this case, it's probably due more to the fact that there are 5 cloud layers that mix together to form the rain/rainbow, while having a large cloud with ray traced shadows visible through them. That makes for some pretty heavy calculations.

What was also interesting was watching the two squares rendering at the same time. In other renders each thread renders at about the same speed but in this one the split was 1:3, with one thread rendering the first and last square (out of 8) The top right square 1/8 took almost as long to render as the rest of the image.
Title: Re: Rainbow
Post by: Oshyan on May 06, 2008, 12:07:10 AM
Wow, a pity it takes so long to render, but the results are gorgeous! Once the memory issues are sorted out with multithreading I'd love to throw one of my quads at this if you're interested. ;D

- Oshyan
Title: Re: Rainbow
Post by: bigben on May 06, 2008, 02:40:50 AM
I may take you up on that. I'll release the clips for the rainbow after I finish them and render something nice
Title: Re: Rainbow
Post by: rcallicotte on May 06, 2008, 09:13:02 AM
Pretty cool lookin', BB.  Excellent work.
Title: Re: Rainbow
Post by: nikita on May 24, 2008, 11:40:55 AM
Quote from: bigben on March 10, 2008, 07:06:20 PM
If you could apply a colour function to a cloud it would simplify things a whole lot  ;)
I'm not sure if this has already been mentioned somewhere in the thread but: I think you can. Use it as your density shader.
Title: Re: Rainbow
Post by: j meyer on May 24, 2008, 01:08:54 PM
nikita - sounds interesting,could you please post an example?
Title: Re: Rainbow
Post by: nikita on May 24, 2008, 09:32:32 PM
I attached an example, a screenshot of it's network and a thumb of the texture used.
It's fairly simple... just multiply the color function, in this case the image map, and your density shader and that's it. Just make sure you're using multiply color (or multiply vector).

I haven't had the chance to do more testing yet, but as far as I know now, the brightness of the supplied color function (the length of the vector) is used as the density function. (This means you can't vary the brightness of the cloud using a function.)
The color set in the cloud's preferences affects the final outcome of the colors in the rendered image too. I guess they are multiplied as usual.

My test uses a single texture for the whole rainbow, projected onto a thin cloud sphere. It's far from perfect, but I think you could make a nice rainbow if you use all the other techniques described and/or used in bigben's project. Most important: reading colors from an image is basically the fastest method of "computation". And you don't need 3 cloud layers anymore. This could speed things up. :)
Title: Re: Rainbow
Post by: j meyer on May 25, 2008, 10:58:57 AM
Thanks nikita,going to try that. :)
Title: Re: Rainbow
Post by: bigben on September 01, 2008, 10:14:24 AM
There may be some merit in this, but the main problem I can see to begin with is that the rainbow appears as a dark band when shaded from the sun when it should actually disappear.  This has given me a few ideas which might be worth pursuing though...
Title: Re: Rainbow
Post by: bigben on November 06, 2008, 07:20:53 AM
I'm pleased to say that I've made some substantial progress with this.  I went back to an earlier version and tried an alternative simplification of the cloud layer.

I now have it down to a single cloud layer using the same masks but rearranged in a slightly different way. 

A lot of the noise was due to the settings in the cloud fractal resulting in a low number of cloud samples even though the quality was set high.  Cloud samples now set to 64 and it's nice and smooth.

The main change has been in the building of the cloud mask. Rather than use 3 cloud layers (Red, Green and Blue) I built the rainbow colour first, added it to a surface shader and set the blending shader for this to the cloud fractal (which in turn is masked by distance restrictions to limit the depth of the rainbow).  The cloud colour is now white, and set to 100, so it still behaves correctly in shadows.

This makes the colour much more predictable, and simplifies the process of constructing composite rainbows. The secondary colours also stand up to fading much better.

The biggest advantage is in the improved render times which is pretty exciting.  The sample below took only 20 minutes with detail 1, AA 4, no GI (no acceleration in the cloud node).  For the previous versions this would have taken up to a day to render.  The screengrab shows the simplified cloud network. Everything else is pretty much the same.  The rainbow travels with the camera, with the only input required being the sun's heading and elevation, which can be easily scripted in an animation.

Now I just have to fix up the mixing of colours in the rainbow to suit the output of the new model.
Title: Re: Rainbow
Post by: Mohawk20 on November 06, 2008, 10:44:43 AM
Sweet!
Title: Re: Rainbow
Post by: dandelO on November 06, 2008, 11:16:07 AM
The best yet, this looks great. Well done on your perseverance with this!
Title: Re: Rainbow
Post by: rcallicotte on November 06, 2008, 02:15:59 PM
I agree.  This is the best implementation so far.
Title: Re: Rainbow
Post by: bigben on April 23, 2009, 10:18:36 PM
Left my usb stick at home so the morning's train trip was about getting this rainbow to the next stage. I downloaded a picture of a rainbow and calculated the positions of the primary and secondary colours to give a better distribution of colours.  On a surface preview it's pretty accurate, but on a cloud the amount/gradation of secondary colours depends on what's behind the rainbow.  A little more experimentation with cloud settings to go for that one.  The other tweak was an improvement (simplification) of the cloud model.

The network looks quite complicated but there are very few nodes that actually control input and output, so most of it can be hidden internally.

Required controls

Everything else is calculated on the camera's position (i.e. it moves like a real rainbow)

Optional controls
Title: Re: Rainbow
Post by: Mohawk20 on April 24, 2009, 04:14:07 AM
Good to see you on this again. Hope you get to finish it soon.