Planetside Software Forums

General => Image Sharing => Topic started by: bigben on October 16, 2007, 04:31:54 PM

Title: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 16, 2007, 04:31:54 PM
I haven't surfaced the North Island yet (uses a blue marble image texture), there's a major screw up with one surface and I have short list of fixes to do... but I still went wow when I saw this at full size.
Title: Re: NZ orbital WIP
Post by: Will on October 16, 2007, 04:47:22 PM
how did you get the water like that Ben? I've doing something similar (but with my own continents) but I'm trying to figure out how to get the rippl;y wave like effects on the water. great work though, keep it up :)
Title: Re: NZ orbital WIP
Post by: Oshyan on October 16, 2007, 04:52:09 PM
Great work so far Ben, fantastic detail and amazing scale! I'm sure you have all this in mind already, but I'll make a few critiques nonetheless. ;D

The most obvious is the cloud scale (and possibly color). A better cloud setup overall is going to help a lot in terms of realism and in this case I'd just as soon have left the clouds out and save render time while you're still tweaking the texturing, etc.

As far as texturing itself goes, the masks look great, it seems like just some coloration for the South island needs adjustment - more browns from what I can see, on the mountain chains particularly:
http://www.4x4adventures.co.nz/p/space.jpg
http://earthobservatory.nasa.gov/Newsroom/NewImages/Images/NewZealand_TMO2002296_lrg.jpg
(I presume both of those are not too color adjusted to be valuable, and I know you already have appropriate references)

Lastly the color of some the South island interior lakes, which again I imagine you're aware of.

All in all this is coming together really nicely. I wonder at the level of input vs. output reward with all the multiple masking techniques and whatnot, but it's definitely stuff worth exploring. Keep it up!

- Oshyan
Title: Re: NZ orbital WIP
Post by: cyphyr on October 16, 2007, 05:03:59 PM
This is real. I mean really brilliant. You've sorted out the lakes and rivers nicely. I wonder how long it takes to render at these resolution. I guess its got to be at 1500 x1500 just to see the detail. What settings are you outputting at and what system are you running on (if I'm not being to impertinent ::D)
Richard
Title: Re: NZ orbital WIP
Post by: nvseal on October 16, 2007, 05:14:34 PM
One of the most amazing space shots I've seen.  :o I can't stop looking at it.
Title: Re: NZ orbital WIP
Post by: bigben on October 16, 2007, 05:22:10 PM
hmm... let's see  

nvseal and I have been exchanging material on orbital renders so lookout!

The two worst surfaces were brown (distribution completely screwed) and bare rock (colour sucks) so you managed to pick most of them  ;)  I have some potentially cool stuff coming for the lakes.

You can never have too many references. Thanks for the links

And finally... I want to render this much larger, which will show a far greater reward. On this image it is particularly interesting to see 4-5 large rivers clearly defined (they all have very wide riverbeds) based on a colour selection made on a single river (partly luck, partly true to theory)

The level of input in this case is not that great. The mask image is at the same resolution as the terrain (90m) and is under 8000 pixels. Even if I add a mask for the north island that's still fewer mask pixels than my old Grampians project which used 3 x 8000 pixels masks and 2 x 2000 pixel masks. While my new technique allows me to create even more masks, I'm struggling to use them all as it's also allowed me to simplify the masking process.  For this scene it will probably boil down to masks for grass (dead and green in a single mask), trees, bare rock, sand, snow (undecided on that one yet) and inland water (shallow and deep (hacked)).

Manually drawing rivers is pointless at this distance though and that will save me a LOT of time, so I'll probably stick with this distance to fine tune the technique and then start another project closer in.

TG seems to be handling the multiple versions of the image pretty well, and for now I'm just exploring how far it can be pushed before it breaks ... just so I know  ;)
Title: Re: NZ orbital WIP
Post by: bigben on October 16, 2007, 05:26:05 PM
Render time about 26 hours. Not too bad for the render size.

Detail 1, AA 4, no GI
can't remember specs of machine but it's about 3 years old so not the fastest.
Title: Re: NZ orbital WIP
Post by: bigben on October 16, 2007, 08:48:59 PM
Here's a tile from a 3000 pixel wide render. (Damn, you can see the rivers I drew manually) I did a hack of a beach for the lakes and rivers that I won't repeat at this resolution. The water layer was given a 1 pixel stroke around the edge, but this is too wide for the rivers.

At this size you can start to see tree-lined roads and when I fix the grass mask up you'll begin to make out paddocks, although it will require an even larger render size to make them out properly.  You can see even more riverbeds now... I find it quite staggering that it's worked so well for such a quick mask creation.

I think I'll tile the render (so I don't have to breakup the larger terrain) and go a bit bigger again for the final.  One last tile coming, twice the resolution again. I'll use that to see how much work I need to do when I create a new image mask.
Title: Re: NZ orbital WIP (& hi res tile)
Post by: sjefen on October 16, 2007, 09:35:30 PM
I like your work bigben. This is just stunning.
Title: Re: NZ orbital WIP (& hi res tile)
Post by: bigben on October 16, 2007, 11:18:46 PM
Thanks.

Here's the tile from a 6000 pixel render from the same camera position (no post work). The mask:render pixel ratio is about 2:1 which is about as big as I'd want to take it without increasing the resoltuion of the mask or using additional blending to disguise the mask borders. There is a fractal child layer providing the colour for each surface but each masked region is quite well-defined as can be seen in close up renders in my masking tests.

I may try a cloud free composite of the final at this resolution. Render times for the tiles are about an hour each, with this res requiring 36 tiles (they will probably take longer near the edge of the globe). I dare say clouds will increase times too much to consider this size (initially).
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: SeerBlue on October 17, 2007, 12:40:16 AM
Have you thought about running your dems thru a gis analysis software such as SAGA Gis (free) to extract masks for the rivers and such, with some experimenting you can get some pretty defined masks. Though I don't know if you can get down to 1 pixel streams, perhaps by running it thru an  colour adjust shader.SeerBlue
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 17, 2007, 01:17:06 AM
It's part of my job to find ways to speed up the tedious  ;)

The terrain is only 90m SRTM3 so the resolution isn't really that useful for really clever stuff, especially in flatter parts of the terrain. There are also several issues with landsat images (JPEG compression artefacts, inconsistent sample times, terrain mis-alignment etc...) which limit how clever you can be choosing rivers "automtaically". The biggest killer is the false colour which will put blue/black pixels in other areas making a colour-based mask unusable.

With higher resolution terrains and full res landsat images I have been able to use a combination of slope map as a layer mask and magic wand selection in Photoshop to quickly select 95% of most rivers. The extra 5% require manually drawing the river to tidy things up, but the results are well worth the effort. With the higher res images you also get more variation in the river width, which can give a river it's own unique charactersitics.

In this case it's going to be manually tracing rivers with a 1 pixel pencil, but as it's only a low res hack I won't have to worry about getting to tricky with the shape of the river.  I may do a full res mask of a smaller area for demo purposes.

In my Grampians project the hardest mask to generate was (and will be) a mask for bare rock. In that specific situation the bare rock is the same colour as lush grass and burnt out forest in the landsat image and drawing a mask that looks good up (relatively) close is extremely difficult.  Editing a 3D mask in 2D is really hard  ;)

That said, it's still worth a look as I'm sure I could find a use for it, so thanks for the suggestion. I did download SAGA once but never got around to using it.  I should also have a look at what else I can do with GlobalMapper now that I've upgraded to the new version.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 17, 2007, 06:48:49 PM
OK, the last one for this file... double the resolution again (that's the first render at 12,000 pixels wide!). For me this is pushing the limit a little too far for this setup, but I had to find out where it was ;)

Not bad if you don't look too close... but you can start to see just how basic the colouring is. There's 8 main colours on the surfaces with very little variation and bugger all blending. I already have an improved mask for lakes (http://forums.planetside.co.uk/index.php?topic=2519.msg25355#msg25355 (http://forums.planetside.co.uk/index.php?topic=2519.msg25355#msg25355)) which alone would make a substantial improvement, but I'll wait until I have finished the new set of masks before posting more. There is some potential for adding some extra detail inthe shallow water of the sea but this will won't be possible to do around the entire coastline due to clouds and reflections in the landsat images. The middle east coast has a lot of potential though.

I'll give the grass surfaces the same treatment as the lakes which should look awesome and then look at fixing the other surfaces.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: rcallicotte on October 18, 2007, 11:13:07 AM
To quote Gomer Pyle, "G-o-o-o-lly!"
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: zionner on October 18, 2007, 11:16:13 AM
Wow!

Nice Man! I love All these!

Can I ask...did you use a DEM for this?

If you did..What one? cause i've been wanting to do some NZ work myself.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: RArcher on October 18, 2007, 11:29:45 AM
I'm sure you've thought of this, but, for your bare rock mask that you are having trouble with, have you tried processing the landsat images in a 4/3/2 band image instead of the usual 3/2/1 band natural colour image?  The 4/3/2 should give you bright red for the vegetation and hopefully white/gray for the rock. 
(http://www.archer-designs.com/satellite-data/images/bands432-large.jpg)
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: efflux on October 18, 2007, 12:35:30 PM
That's spectacularly large scale in feel. Maybe some more variation in the water regions and cloud issues have been mentioned but other than that, great progress. Awesome work.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 19, 2007, 08:05:39 PM
Quote from: zionner on October 18, 2007, 11:16:13 AM
Wow!

Nice Man! I love All these!

Can I ask...did you use a DEM for this?

If you did..What one? cause i've been wanting to do some NZ work myself.
Its all DEM data.... 4000x4000km of it! The terrain is only SRTM3 so it needs a lot of "help" for up close work.

Quote from: RArcher on October 18, 2007, 11:29:45 AM
I'm sure you've thought of this, but, for your bare rock mask that you are having trouble with, have you tried processing the landsat images in a 4/3/2 band image instead of the usual 3/2/1 band natural colour image?  The 4/3/2 should give you bright red for the vegetation and hopefully white/gray for the rock. 


No I hadn't  ;) Looking at the image I think there would still be some problems, but it would certainly be a lot easier, especially if placed below the grass layers (where it should be anyway).  Do you have a source URL? I found a site before but didn't bookmark it.  The main problem with rock occurs when you start getting closer. It's the first mask to become obvious due to the lack of resolution and it's difficult to add terrain-related variations because it can occur over a wide range of terrain features.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: RArcher on October 20, 2007, 12:37:04 AM
My source for raw Landsat stuff is here:

http://glcfapp.umiacs.umd.edu:8080/esdi/index.jsp (http://glcfapp.umiacs.umd.edu:8080/esdi/index.jsp)

You can search by a variety of different options, and everything comes in raw grayscale tiff format so you can combine whichever bands of data you want to get a variety of different remote sensing information.  Here are the steps I use for processing the bands into a 15m resolution Panchromatic False Colour image:

For a natural colour image:

1. Open Bands 1, 2 and 3 in photoshop
2. Adjust the levels of each image – move the left slider to the lowest visible black point, move the right slider to the highest visible white
3. Under Channels choose – merge channels, choose RGB.
4. For Red use Band 3, for Green use Band 2, for Blue use Band 1 - when done this is your 30m sample, for 15m keep going down the list
5. Once it is combined, change colour to LAB
6. Double the image size
7. Under channels choose – split channels
8. Close the Brightness channel, leave channel A and channel B open
9. Open Band 8, adjust levels as before
10. Under channels choose – Merge channels, choose LAB
11. Use Band 8 for Lightness, Channel A for A, Channel B for B
12. Adjust colours, contrast, levels, sharpness etc.
13. Save and recalibrate if desired

For other varieties, just change the band sequence to highlight different features.

Here are a few varieties, but there are many more:

http://www.archer-designs.com/satellite-data/satellite-image-gallery.php (http://www.archer-designs.com/satellite-data/satellite-image-gallery.php)

You can also put the landsat bands together in Global Mapper, but it is pretty difficult to get good results.

Here is the 15m tile I converted while writing these out to make sure I got the steps right (It is around 170mb so it may take awhile to download, but I think it is far better than the google earth landsat images)

http://www.archer-designs.com/temp/p044r02520010914z11.rar (http://www.archer-designs.com/temp/p044r02520010914z11.rar)

Full resolution crop:

(http://www.archer-designs.com/temp/lsat-crop.jpg)

Once you have everything converted to a contrasting colour palette, you should be able to do a colour selection or replace colour step to change the rocks to whichever colour you want.


Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 20, 2007, 11:01:11 AM
That was the site I was thinking of... and thanks for the tips.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: dhavalmistry on October 21, 2007, 12:22:36 PM
hey RArcher....is that satellite imagery???....incredible detail....
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: zionner on October 21, 2007, 04:14:17 PM
Wow!

That is incredibly High Res, Nice Find!

BigBen...Could I chance it and ask you where you got that DEM? lol

I have used smaller DEMs before..but nothing like that.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: RArcher on October 22, 2007, 09:28:13 AM
Dhavalmistry:  Yes, that is just standard landsat 7 imagery.  The only difference between that image and google earth, is that I processed it by hand which tends to retain better detail.  Using the link at the top of the post, you can download raw images for anywhere in the world and then just follow the steps to get a similar result.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 23, 2007, 12:24:30 AM
Quote from: zionner on October 21, 2007, 04:14:17 PM
Wow!

That is incredibly High Res, Nice Find!

BigBen...Could I chance it and ask you where you got that DEM? lol

I have used smaller DEMs before..but nothing like that.

DEMs are SRTM3: http://srtm.csi.cgiar.org/ (http://srtm.csi.cgiar.org/)
It took a while but I finally have the full set.

Landsat references: https://zulu.ssc.nasa.gov/mrsid/ (https://zulu.ssc.nasa.gov/mrsid/)
plodding through them....

Bathymetry data: http://topex.ucsd.edu/WWW_html/srtm30_plus.html (http://topex.ucsd.edu/WWW_html/srtm30_plus.html)

From these I created 6 TER files to cover NZ plus a couple of smaller ones to include the larger offshore island... The bathymetry is a highly adjusted TIFF file derived from the bathymetry data (covers 4,000x4,000km). I could have included it in the TERs but it was simpler to fale the transparency and have a flat terrain for the ocean surface.

I may recreate the TERs as there is one large one on the south island that is limiting the size of a single render to 1500x1500.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: jo on October 23, 2007, 09:56:23 PM
Hi Ben,

This is excellent stuff. I live roughly under the red arrow :-) ( can't see the arrow today, it's too cloudy ).

Regards,

Jo
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: nvseal on October 23, 2007, 10:42:44 PM
Jo, can I live where you live?  ;D
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: Seth on October 24, 2007, 01:21:44 AM
Quote from: jo on October 23, 2007, 09:56:23 PM
Hi Ben,

This is excellent stuff. I live roughly under the red arrow :-) ( can't see the arrow today, it's too cloudy ).

Regards,

Jo



damn ! lucky you are for sure ^^
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: zionner on October 24, 2007, 12:57:14 PM
Thanks Ben :P
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on October 24, 2007, 11:04:27 PM
Quote from: jo on October 23, 2007, 09:56:23 PM
Hi Ben,

This is excellent stuff. I live roughly under the red arrow :-) ( can't see the arrow today, it's too cloudy ).

Regards,

Jo


Hi Jo

I might fly past in the animation (since I have some cool subsurface detail along the cost) but I'll be "landing" near the lakes at the bottom of the image. More clouds are on their way...
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: dhavalmistry on October 25, 2007, 12:01:08 AM
Quote from: jo on October 23, 2007, 09:56:23 PM
Hi Ben,

This is excellent stuff. I live roughly under the red arrow :-) ( can't see the arrow today, it's too cloudy ).

Regards,

Jo

It must be beautiful down there....
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: lightning on November 01, 2007, 03:58:29 AM
cool one mate nz rox i live there 2 ha! in a  little town called Geraldine about half a hour away from timaru south canterbury .by the looks of it you live somewhere round mt cook am i right?
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on November 01, 2007, 07:02:34 PM
Here's a rough WIP of a smaller high res mask I'm preparing for a zoom animation from orbit. Camera height is about 2km (camera height in the previous render is 670km). Thought I'd post it here to give it a bit more perspective with the detail in the original.

There are only two main surfaces in this (and a crude snow), but at this camera height I should be able to make a fairly convincing looking scene with the other surfaces and some added detail. Going any lower will require a lot more work but I'll try and get this level of detail done first.
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: rcallicotte on November 02, 2007, 03:49:38 PM
Like it.

Someday, I'll figure out what you're doing.   ;D
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: king_tiger_666 on November 02, 2007, 07:32:17 PM
great stuff.. always good seeing our country being rendered:D
Title: Re: NZ orbital WIP (& hi res tiles)
Post by: bigben on November 03, 2007, 12:37:52 AM
Quote from: calico on November 02, 2007, 03:49:38 PM
Like it.

Someday, I'll figure out what you're doing.   ;D

I'm contemplating writing some tutorials on my masking methods (and masking in general).  Working my way through Volker's tutorials at the moment to fill some of the gaps in my knowledge (I still classify myself as a noob in some areas). Proving very helpful so far, so I may be able to set a lower minimum altitude later on.