Embedding multiple masks in an image

[bigben]

Here's a 500 pixel render from orbit. South island contains masked surfaces, north island contains blue marble texture.  Even though the render's a bit small, you can still pick out several lakes over the island and a long riverbed running across the island to the coast (cyan arrow).

The yellow arrow indicates the position and direction of the camera for the previous render.

Running a cropped 1000 pixel render now to show a bit more detail.  I got errors attempting 2000 and 3000 pixels renders. I'll have to track down which image is causing it, although I suspect it may be the mask image.

Just remember that the mask creation was only a quick one based only on the area for the render above. This mask only took about an hour to make including manually drawing some rivers. Taking that into consideration I'd say it's worked fairly well, which is probably more luck than anything. The next render is looking very cool 


[edit] here 'tis. You can see the riverbeds I was talking about at the top right.  Running a 1500 pixel render of the whole scene with better AA overnight (+). Compare this to the blue marble texture in http://forums.planetside.co.uk/index.php?topic=2512.msg24788#msg24788

Now for the long process of creating a proper mask... [/edit]

[monks]

Won-der-ful.

monks

[bigben]

Thanks.

Now that the initial enthusiasm has slowed down a bit I can get back to being a bit more methodical 

Attached a zip file with a test image and the node setup for these tests.  Colouring by numbers using a single greyscale image.
[attachthumb=#3]

As mentioned previously there are edge effects if image smoothing is turned on, but whether this is detrimental to the final image will depend on a number of factors.
[attachthumb=#2]
This demo shows a worst case scenario where masked areas are surrounded by black. If you zoom in really close on the edges you can see the banding resulting from image smoothing.

Turn image smoothing off and there are no edge effects.
[attachthumb=#1]

The best bit is, with "Data is linear" set in the image map's colour section the numbers matched the greyscale % set in Photoshop. You could potentially set even more masks per image, but that would depend on your uses for them.

Now to add some tonal masks to the mix....

[cyphyr]

Is that a clamp function (I'm guessing clamp colour?) in there?
Richard

[bigben]

The colour adjust shader includes a 0,1 clamp. Black and white levels are placed close together just below the greyscale value of the region to be masked. You could set them just above the previous greyscale used, which would give you a slightly wider masked area. I've used this to create "beaches" for rivers by simply using colour adjust shaders with different limits.

You could also use 2 colour adjust nodes for each limit, using the "wide" mask for one mask, and subtracting the "narrow" mask from the adjacent tonal value. This would reduce edge effects on smoothed mask images but may have a detrimental impact on render time.  The performance of this kind of thing is one thing that has improved with the latest update. Some of the things I tried previously would simply crash TG in under a second, but now they render nicely.

So many ideas still spinning around... I'm definitely glad to have found a way around the limitations of my earlier masking projects in TG0.9 and TG2, but it will also mean many more hours creating the masks.  My Grampians project in TG0.9 was something of a personal milestone, but that benchmark has been moved a very long way with TG2... even taking objects out of the equation. Having almost as much fun as the guys working on displacement spires 

Attached another test, this time of a best case scenario where tonal regions of the mask are in sequential order. Image is smoothed and there's only one edge with edge effects (left hand edge where white is adjacent to black)

[nvseal]

This is looking so good! Can't wait for the next render.

[cyphyr]

In you "best case" scenario could you use a "junk" colour that would have the edge effects occur on it but since your not using that colour it would not render, sort of like having a padding colour. Bit out of my depth but hey sometimes (read a lot) I just cant keep my trap shut
Richard

[bigben]

This is looking so good! Can't wait for the next render.

Alas, we're at the "waiting" render size  ... although there's just the one cloud layer left in so it's progressing reasonable well.  Thanks for the clip.  I'll have a spare machine at work in a couple of weeks. I think it might have to render an orbital cloud animation test. Animating a single cloud fractal works really well in TG2. Animating multiple clouds with multiple fractals controlling blending and warping could produce some very interesting results.

In you "best case" scenario could you use a "junk" colour that would have the edge effects occur on it but since your not using that colour it would not render, sort of like having a padding colour. Bit out of my depth but hey sometimes (read a lot) I just cant keep my trap shut
Richard

The colours in this case are simply to illustrate the distributions of the masks. I needed exaggerated colours so that I could easily see whether distriubtions were correct, and to highlight any potential glitches. 

There are several factors that would influence the final result and if/how much these edge effects would be noticeable so I wouldn't worry too much about them here. I've just mentioned them because they should be taken into consideration when selecting which greyscale value in the mask to assign to which surface. e.g. reflective water would really destroy a render if it shows up in odd places, so water surfaces should always be assigned to black or white so that there is no overlap on boundaries between other surfaces.

I have a couple more tests to do and then I'll show a real example, which will help put this into perspective. It's not necessarily ideal for all image-based masking but it certainly has a lot of potential for saving memorey usage by reducing the number of masks required, thus allowing higher resolution masks to be used ... which in turn results in more realistic renders.

Keep the trap open... explaining things helps me sort things out more clearly for myself, which is partly why this post is so verbose... thinking out loud 

[bigben]

Spent the night working on the technical processes to correct the mistakes I made in the intial masks for NZ and I now have a clear idea of how to fix it.... then I got to work this morning and saw the "buggy" render about 2/3 complete. Looking very cool 

I've nutted out a process for using a range of mask greyscale values to create tonal variation within a masked area... or a gradated blending of the surface. I'll post a demo tonight and then start some serious work on the real masks for this project.

[bigben]

Here's the next proof of concept... using TG's image anti-aliasing to create intermediate tones in an image with a limited pixel value range.

3 blocks of the step wedge image were filled with a gradation covering the same pixel value range as the blocks.  Using the previous method, the total distribution of the masked area is defined by  creating a "binary" image of mask using the high and low pixel values of the range and then subtracting them to get the difference.

An additional colour adjust shader is then used to increase the contrast of the gradation to whatever range is required... e.g. nomralised to 0 - 1 for use as a fractal breakup, or a smaller range for use as a colour function (as in this example) This provides extra flexibility for blending shaders using the same portion of a mask as well as creating extra variation within a surface.

Once again, the choice of mask subjects and their order is critical to dealing with any possible edge effects, but it is quite feasible to get 2-3 tonal masks and 2-6 binary masks out of a single mask image.

The sample render shows the original mask image showing the gradation in the middle, the effect of the step wedge surface shaders on this gradation, and a masked portion of just the gradation, with a contrast adjusted version used as a colour shader. The transform shaders are simply to avoid loading the image multiple times.

Used carefully this technique could be used to greatly extend the detail in masked renders using real life data in a reatively efficient manner, leaving extra resources for better surfacing and large populations.

I also did some testing to see why I got errors attempting to render the NZ scene at 2000+ pixels output. I removed the images 1 by 1 to no avail and then started on the terrains. Removing the largest TER file got a 3000 pixel render started so I may have to split this terrain into two parts. Posting a 1500 pixel orbital render in the image sharing section soon...

[bigben]

OK enough of the pretty coloured squares. "What's the point?" you may ask. The example in the first post shows multiple "binary" masks using a single mask image. This in itself is pretty cool. Now let's take the lakes as an example.  In my render they are all the same colour. In all satellite images, there are two large lakes in the centre of the south island that show up as being much lighter (shallower). In higher resolution imagery, you can also see some subsurface detail in many lakes giving you a sense of the depth of the water.

Starting with the satellite image and a slope map I first created a mask for the boundaries of the lakes. Using this mask as a selection, I copied the RGB image of the lakes to a new layer in Photoshop, extracted the blue channel, normalised it's pixel values and then set the black output level to 225 so that the lakes' pixel values ranged from 225-255. The rest of the image was filled with black, but would normally contain the other masks.

Now into TG2.
Load the image and use a colour adjust shader to set the black and white levels to 0.8, and feed this into the fractal breakup of a surface layer, creating the boundary of the lakes.

Link the image to a second colour adjust node, this time setting just the black level to 0.8, giving you a full greyscale image of the greyscale values from the RGB image. While you might expect stretching 30 pixels values out to a 255 pixel value range would contain significant banding, you have to remember than anti-aliasing was applied prior to the colour adjust, smoothing the pixel boundaries with intermediate tones. The result is instead incredibly smooth.

Apply this to the fractal breakup of a surface layer to act as a blending shader between a light and dark water colour, tweaking the gamma in the colour adjust node to get the desired effect. The end result is a hack of inland water bathymetry.  There is an edge effect in the render below, and I need to invert the mask greyscale values to reverse the effect so that the edges appear as shallow water. The edge effect will still be there, but it will be cunningly disguised 

I'm always impressed by just how flexible the colour adjustments are in TG2. Given that I've started with an 8 bit image and applied some pretty severe adjsutments to a very small tonal range, the results are *very* smooth. This "feature" helps to make this technique more flexible by allowing more tonal masks to be squeezed in to a single image.

[Oshyan]

Very nice results and cool in-depth manipulations of the masks. But are you sure those lighter lakes are just shallower? I rather suspect it's some kind of area-specific sediment or bottom coloration, like the bright blue or green alpine lakes you see sometimes: http://www.myroyaltyfreehd.com/assets/images/bright_blue_lake_in_alberta.jpg As you can see it does not appear to be a function of depth.

- Oshyan

[bigben]

They did seem to be unnaturally light... variations like this could be a combination of several things such as this or the colour of the material forming the lake bed. We have the reverse situation here in some places where the lakes are on very light sand but tannins released from leaves falling into the water make the water look almost black.  The cool thing is that it doesn't matter what the cause of variation is, it gets picked up by using a mask based on the actual colour of the lake.  An extra bonus   

[bigben]

This is sort of becoming a diary for me for this project... more thinking out loud and a bit of progress...

The new mask is coming along nicely. The landsat images for NZ are fairly consistent which is making things easier. The first mask I made was an extrememly simplified technique. One of the biggest weaknesses of was that there was no allowance made for shadows. You can't notice it much in the render because the lighting is similar to the angle of the sun in the satellite images. 

This time, however, I went back to my more accurate hack to remove shadows...  Convert the satellite image to lab colour, fill the lightness channel with 50% grey and normalise the other two channels. Convert back to RGB and then run a curves adjustment to put some contrast back into the image. Add some adjustment layers to tweak the contrast/hues temporarily for highlighting certain regions, and then use colour slection to select the area.

Then it's a case of copying and pasting the relevant channel to a new layer and adjusting the levels to fit the range in the final mask image... simple really... kind of. Selecting the right colours to generate each mask can be very tricky.  [edit] Just had a look at the images you linked Oshyan and it seems I was a lot closer than I thought.  There are some minor adjustments to be made but it's going to be easier rather than harder. Those images were very helpful. [/edit]

The sample of mask image attached also shows some coastal subsurface detail. I'm going to try to include this around the north east corner of the north island. As the mask will be in a shader separated from all of the other regions by an altitude restriction, I can leave it as a full range greyscale image.This will be used to further lighten the subsurface detail "hacked" from 1km bathymetry data and should look really cool.

From my other resolution tests it looks like I'll be drawing a lot of rivers. Things have changed somewhat frommy original ideas, but the more I toy around with this the better it's looking, and it's a lot simpler than my previous masking efforts from satellite images. My last renders are starting to look pretty ordinary

[bigben]

Did a bit more work on the masks and applied it to my project.  I started with the coastal subsurface detail which worked pretty well but required some careful finetuning of several shaders to remove edge effects along the beach/water edge. There were essentially two main issues.

1. Antialiasing produced reduced pixel values along the beach/water edge over the water. This in turn resulted in the lake surface (darker, more saturated blue)  appearing in the shallowest water. This was fixed by adding an altitude restriction to the lake surface *and* reducing the white point to spread the amount of white across the margin.

2. Antialiasing also offsets the texture slightly as was apparent when I tried to combine a smoothed and unsmoothed image in the same mask setup. With some more experimentation I may be able to find an offset amount to realign the smoothed image. Such an offset is typical of image processing routines, and is usually too small to be noticed, but in this application it is critical.

The extra coastal subsurface detail looks pretty cool   Terrain surfacing was only partially reconstructed in this render.

[attachthumb=#1]

Reworking the surfaces to the new mask it's starting to look a little sexier than the original version.

• Ocean subsurface detail: greyscale bathymetry data + full greyscale range from the primary surfacing mask image (only on upper right coast in this view)
• Lakes: Distribution and colour variation extracted from primary surfacing mask image
• Rock: distribution only (needs to be converted to include colour variation)
• Dry Grass: distribution + colour variation
• Green Grass: distribution + colour variation
• Trees: distribution + colour variation
• Snow: Altitude restriction, 2m displacement, favour ridges.

Some aspects of this have improved, others have been degraded...although these only appear at closer camera angles.  Some more experimentation is needed to see if I can work around these problems, but even if I can't it's still a nifty method if used within its limitations. There are still a couple of gaps in the mask image I have to fill after I interpret what the colours are referring to.  There are no rivers drawn in the mask yet, so that will be another improvement to come.