Hi All
Figured I'd make a post for this as you know how I like to think out loud while I'm working on something ;) At Oshyan's suggestion I'm building a basic framework for a model of the earth in TG that will ultimately be released for everyone to use. I did a few quick tests of some of the available data to the point of adding a river/lake layer. https://flic.kr/p/ra5iq5 (https://flic.kr/p/ra5iq5)
For now, I'm crunching the elevation data to get that bit right to start with as this is also proving to be the most time consuming. Dealing with some of the problems in the data has confirmed for me the need for such a project.
- SRTM data is great, but it only covers latitudes between +/-60° and has issues around the Black/Caspian sea
- GMTED2010 has enough resolution with global coverage. but has a gap between latitudes -56° to -70° and the Himalayas contain some large areas of ugly hole filling
- GEBCO-30 has global coverage with some very nice bathymetry but it's only 30 arc sec resolution and I'm looking at a max resolution of 19 arc sec
No single dataset has what we need, so it's a case of picking the best available data from each. I currently have 500gb of raw data and will hopefully finish crunching the SRTM data down to size by tomorrow so I can start piecing it all together...
Stay tuned ;)
Perhaps you may find what you need at Jonathan de Ferrantis excellent site:
http://viewfinderpanoramas.org/
He has made available global 3 arc second data, partly just because SRTM has it's flaws with gaps. He also provides 1 arc second data for several mountainranges. A truly amazing feat and resource!
Good luck!
Robert
Looks like a promising resource Robert. Might save Ben some work, hopefully so! There's lots more to the proposed model though - color map, specular, subsurface water (as needed), clouds, and some other ideas.
- Oshyan
Thanks Robert.
I've just finished the base terrain, just a bit of bathymetry to go, but I'll still have a look at this data (nice accessible links for harvesting)
At the moment I've stalled a little bit on the ocean model. Getting some odd glitches in the shallow water which vary based on the angle of the sun. Ocean is a second planet with a water shader, decay distance 1000. Seems to be a linked to masking the surface. In the last image I removed the mask. The green surface shows the position of the mask used for the water surface... any ideas for what to look for?
Try turning off Cast Shadows in the 2nd planet.
- Oshyan
That option isn't available on a planet(?) I used a planet-sized sphere and that made a big difference to the water. After a little juggling I also got the opacity working to stop rendering water under the terrain (and flooding low lying land) Possibly still some minor glitches at the boundary of the two objects, but I'll deal with that later. Image is 1000m decay distance over default base colours.
OK, so this brings us to beta 2.Water settings can be tweaked, this is just a demo. Decay depth is 5000m... unrealistic, but an interesting visualisation. Ocean floor is just a light sandy colour. Blue marble texture is restricted to inland areas. This creates a few dark artefacts along the water's edge. This is an inescapable problem with the source image along with the loss of several small islands. Both of those issues will get looked at later as there are no quick fixes for them, but I have a couple of ideas.
Rendering without atmosphere at the moment so I can see what's happening, using a blue background instead. Lakes and rivers next..
Your progress is amazing, looks really good. Too bad about the coastal artifacts. The ocean floor can perhaps be made procedurally, it's probably dirt mostly anyway (xcept for the great barrier reef ;) ).
I also think you should make some more diversity in the waves, but you already had that planned I guess.
Yes, to start with I just want to keep the surface shaders simple as it's important to see where things don't work (and do work) While it's only a small number of components there'll be a fair bit of masking going on.
When I talk of creating a realistic Earth
model, I did not think there could be
terrain models with such resolution!
you done a great job so far (or I'm a little surprised) :o
however I'm curious .. how much memory the dataset sacrifice?
Let's just say that when I go home I edit the TGD to change the resolution of the TIFs from 0.01° to 0.02°/px. (8Gb RAM at home, 32Gb at work.. 64Gb on work VM). I have elevation data down to 0.005°/px in 4 quadrants which kicks of TG at 11Gb RAM usage, although I can load a single quadrant at home without much drama. The files may be big, but rendering doesn't increase RAM usage that much.
Quote from: bigben on February 23, 2015, 03:36:01 PM
Let's just say that when I go home I edit the TGD to change the resolution of the TIFs from 0.01° to 0.02°/px. (8Gb RAM at home, 32Gb at work.. 64Gb on work VM). I have elevation data down to 0.005°/px in 4 quadrants which kicks of TG at 11Gb RAM usage, although I can load a single quadrant at home without much drama. The files may be big, but rendering doesn't increase RAM usage that much.
Good to hear you can load them with 8G RAM as that's all I have...Looking forward to this product more than I can express. I've directed all my change to go to NWDA as that is a very cool store....and reasonable prices as well. Adding this to my wish list. Great work, no; mind blowing at least to me, work...
Yeah, 5 year old computer at home, so I'll have at least one version that won't be too hard on resources
Quote from: bigben on February 25, 2015, 01:01:17 AM
Yeah, 5 year old computer at home, so I'll have at least one version that won't be too hard on resources
Thank you for that ;)
Puget Sound is very recognizable, great work! will I get to see Alaska too?
Will this be a purchased product?
Russ
Oshyan can clarify this, but AFAIK it will be free from a TG resource site. The source data is all public
Doing a fantastic work here!! Thx for all your efforts ben ;)
And it will be free to play with it? Wow that is very cool.
Well I sure hope Ben gets rewarded for the efforts.
I am working on animating a sounding rocket lift off, and I want to have the camera follow it up to very high altitude.
Would be so great to have the land mass recognizable. As it is, the Earth model looks already useful.
You go!
r.
Ah right, that's the reason to use a sphere instead of planet, of course. Glad you figured that out. The sub-surface is looking great now. What's the source of the black shore artifacts hough?
And yes, the intention is that the data be hosted by Planetside and available for free. Earth as a subject is a common interest and need for many, and the base data is already freely available. The problem has been that the data needs some work in various ways, not just for TG but for almost any renderer. To get a really good result it takes some work, and frankly I don't think anyone has achieved the best possible quality yet. So this project is intended to tackle two main issues. We want to eliminate the duplication of effort (reinventing the wheel, so to speak) that people have had to do in setting up their Earth models, which I've seen people do at least 10 times just here in the forums alone. And then we also want to increase the actual overall resolution and quality of the data that's available and easy to use. The end goal is an easy-to-use Earth model with the best possible quality.
- Oshyan
I honestly have no idea on the artefacts, or indeed the subsurface detail. There is a substantial difference between using a planet and a sphere in this case, especially regarding decay depth. I'll run a render tonight with both using the same shaders to demonstrate but the two sets of images in this post have very similar shaders.
Quote from: Oshyan on February 26, 2015, 01:21:19 PM
The end goal is an easy-to-use Earth model with the best possible quality.
- Oshyan
for me, that would be a utopia made reality
If this is successful, I would like share my effort with this project
I'm work in a "semi-procedural" clouds layers
that would be the 1st part of my personal project
the 2nd part would be the terrain, and share here when it finish
but with the effort of users like Big Ben, this is not necessary now
I am anxious to see the result and help in whatever with this project
I wish you the best of luck Mr Ben
Ben, did you disable shadow casting in the sphere?
Ariel, yes I think your clouds could become a useful part of the project. There will be one or more base cloud masks (image-based), but we will want procedural detail to fill it in. If you have useful cloud masking techniques (e.g. continent-based, as you have shown), then you could potentially provide your own additional setups for that.
- Oshyan
Yes, shadows were disabled on the sphere. I must have also changed the decay depth. Rendered both ocean models with the same settings: planet and sphere with cast shadows on look the same.
Got to spend a bit more time on this. Basic city lights this time. I've set up a mask for the shaded side of the earth based on the sun's position, providing a gradated border to fade them in. The image below was based on a preview without the atmosphere so it looks like the lights come on a little early. The mask is adjusted via a colour adjust shader specifying the ON and OFF limits in degrees from a vector opposite the sun's position.
This looks really sophisticated, Ben. Awesome work!
Thanks.
Most of the nodes can probably be buried into internal networks once it's done. I'm trying to keep it functional for animations and also relatively straightforward for people to add their own tweaks.
Here is a rivers and lakes test. Based on an image map derived from vector data for lakes and rivers (and islands). The offset from the image may take a bit of editing of the texture map, although I'm hoping most of it is due to resizing in Photoshop rather than exporting from GlobalMapper at lower resolutions. I've checked without any other surfacing and the water IS sitting on the surface of the planet. The width of the rivers is a bit unrealistic, but I've narrowed them as much as possible using a colour adjust shader on the mask. The water is just a masked surface with a water shader. To keep the lakes flat the DEM is loaded into 2 heightfield nodes, one with fractal detail and one without, using the water mask to apply one or the other.
How will you deal with areas that are below sea level, but still dry? Mask? The western half of my country is well below sea level :-\
Separate coastal mask for the sphere that forms the ocean surface. Didn't want to flood Lake Eyre either. ;) I have 3 water-related masks in a single RGB image, pulled out using convert to R,G,B nodes although I think one of them is unused at the moment
Must check what happens at the Netherlands coast. Hopefully the ocean isn't floating above
Getting more excited about this with every up date. Awesome work Ben!
Great to see further progress on this, especially the city lights setup. The dark side of the Earth is rather bright in general though, isn't it?
Also, still worried about the appearance of shadow from the lakes which makes it look like they're hovering above the land. Why is that? Almost like it's "embossed" or something (simple Photoshop effect). But the amount of detail is great!
- Oshyan
There is a fill light on the dark side in that render... wanted to see where stuff was still.
As for the lake edges, I've georeferenced the blue marble image (quite easy with a global image) and am exporting the lower resolution versions from that. Checked the new texture maps with the masks and there are some 1px differences. This would also be exacerbated by trying to sharpen the edge of the lake/river mask with a colour adjust shader. I might have to try a content-aware deletion of the lake areas from the blue marble image to try and "shrink" the lakes.
Ah, thanks for the additional info. Good to know.
- Oshyan
Exploring an idea on using the image texture colour to mask TG shaders to rebuild the image. Comparing a reference colour to colour in the image using the angle between two vectors constructed from the RGB values... Sample images are the image map masked for a particular colour. The idea is to replace the image map entirely with TG shaders using the image map to control distribution and blending between the surfaces.
This looks totally brilliant but I'll need a math education to grok it. Well done/doing Ben....
:o :o brilliant!! how is that not even occurred to me?!
this really works?
requires a great texture to correctly define the coastlines?
If you really want to explore the math:.. From "Maths is Fun" http://www.mathsisfun.com/algebra/vectors-dot-product.html (http://www.mathsisfun.com/algebra/vectors-dot-product.html) This is actually quite a useful little function for lots of things.
But you don't really need to understand it to use it in this case. Input is an image and a reference colour, output is a greyscale mask of colour similarity in the image to the reference colour (not including intensity). I'd started to explore this idea using Colour Select in Photoshop to create greyscale mask images, but that has obvious implications for RAM usage. This will sacrifice a little realism for (hopefully) a nicer looking render once you surpass the resolution of the image texture (e.g. the previous lake image with a very low res texture underneath)
Apart from altitude restrictions, I have high res mask images for coasts and all water boundaries.
I see this link later (I have a little sleep to think matematicas now)
regarding the method, I suspect think I could still be a good way to get "extra details", especially for (as suggested Oshyan) create a more "versatile" model of the Earth
and of course, the realism is lost a little when the resolution of the texture is exceeded. I should experiment on my own for talk a bit more about that
Onwards to the pole... Blue marble has no sea ice. Here is the monthly average coverage of sea ice for April... Colour adjust shader can get you the max or min coverage depending on what you want, as well as monthly data to go with the Blue Marble images. 8) I'll build a basic ice sheet but leave it to others to rough it up. Data is for both poles, and is only a 1800x900px image map.
I'm super excited about the procedural detail masks!
- Oshyan
Quote from: Oshyan on March 14, 2015, 03:05:56 PM
I'm super excited about the procedural detail masks!
- Oshyan
I'm rather lost(don't worry, I'll just watch), but it seems like it's reverse anti-aliasing the low-res image. Very cool whatever it is.
8)
Quote from: lat 64 on March 14, 2015, 04:38:51 PM
I'm rather lost(don't worry, I'll just watch), but it seems like it's reverse anti-aliasing the low-res image. Very cool whatever it is.
8)
It's more like posterising the image into fewer colours and then replacing each colour in the image with a procedural surface. Here's a very quick demo using the orange example above.
Once you get multiple colour-based masks in action they will also overlap providing additional variation/transitions and you can also rough up the edges of the mask.
And a clearer example of the ice cap mask
Quote from: Oshyan on March 14, 2015, 03:05:56 PM
I'm super excited about the procedural detail masks!
- Oshyan
I hadn't gone that far yet as there was one piece of the puzzle missing. The mask is essentially just a hue similarity, so it doesn't distinguish between dark green and light green for example. I've been hovering around the answer a couple of times in coming up with this model and it finally came to me. The lightness of a colour can be put into a useable form by converting it to a vector (RGB in the range of 0-1) and then dividing the length of that vector by the square root of 3 (the length of that vector in a 0-1 range) Most of this is already in the previous mask model so it's only a couple of extra nodes.
Now I'm super excited too ;D
Still baffled by the maths but generally blown away by your results....
Smart thinking, Ben! I'm getting very excited as well.
Here's a clip file for the mask if people want to play with it. It takes a little experimenting with to get used to, and I'd strongly recommend using a preview shader while tweaking the numbers in the colour adjust nodes for the final output. For the reference colour, I'd suggest using either a light or dark variant of the reference colour which will make it a bit easier to use the lightness mask.
Screengab shows how I've connected it up. Input is a convert colour to vector node. I haven't included it in the clip as it would most likely lead to unnecessary duplication of that node.
I usually build up my procedural colours from dark to light, so on this case I'm using the lightness mask to differentiate between tones, and the colour mask to control the total distribution of the parent shader.
Scary ...cool...thanks for the peek under the hood as it were.
I'll release this more formally in the file sharing section later with a more complete example. This is so far the best solution I've come up with for this particular problem. You may remember some of my earlier attempts, particularly this (scarily old) one: http://www.planetside.co.uk/forums/index.php/topic,2519.0.html (http://www.planetside.co.uk/forums/index.php/topic,2519.0.html).
Applications include pretty much anything that uses real world imagery as texture, and it provides some possibilities for tweaking the input images to make better masks that would otherwise have seemed unrealistic e.g. greatly lightening shadows for high res images or content-aware deletion of shadows from lower res imagery. Also allows for re-colouring of false colour satellite images directly in TG.... lots of potential
Incorporated the sea ice mask onto the ocean sphere. Here's a sample low enough to see the 2.5m thick edge of the ice shelf off Greenland. Tossing up ideas to rough up the edge. Might go with a masked PF to provide smaller ice chunks in the surrounding water.
Ice shelf! Coooool. :D This is turning out even better than I had hoped, thanks to you and your creative, inquisitive mind Ben. :)
I see a ground-to-orbit Earth video in our future (without transitions).
- Oshyan
Yes, you know the ground to orbit animation is always at the back of my mind when working on this sort of stuff. It's looking more feasible now. I've been looking at the coastal bathymetric data and much of it appears to be already incorporated into the global dataset I'm using, albeit at a slightly lower resolution. I'll add it in to a later release of the elevation model as I have to do some intermediate cleanup and they're too big to work on with all of the other data loaded at the same time.
I'll release a few other components to get some ideas from others on some of the masking challenges. E.g. City lights are challenging as the data is technically a mix of lights, atmospheric glow and lens/sensor flare. I'm thinking some square'ish blocks might help break things up a bit more. I have them as 1m circles at the moment but they still look like amorphous blobs from a distance.
Getting close to a beta
Had to do an ice shelf image with context ;).... a little lower, atmosphere and dirty (dark texture) polar bear for scale
That polar bear is filthy. ;)
- Oshyan
If you could take the (edge of the) shelf through a surface shader, you can warp it into loose chunks here and there and/or use the breakup with an adapted breakup fractal (other fracture noise and using not only color) to break pieces of ice of. Just an idea.
Awesome progress!
Roughing up the border is on the cards. There is actually plenty of greyscale mask to play with as the map represents the historical range of ice cover for a given month. Rather than provide 12 image maps I'm thinking of combining them into a single map for the year so that the ice cover can be animated more easily. That could also be used as a kind of density map for loose sea ice.
So you can teach an old dog new tricks My first warped mask thanks to an example TGD in the files section by some guy calling himself Dune. ;)
The dark line at the ice/sea border was annoying me but it was actually correct for the structure of the model. Rebuilt the sea ice model and eventually ended up with a second planet with a negative displacement for the bottom edge of the ice and a masked displacement to build the sides and top. The rest of the planet surface is transparent, so you see the ocean floor where there is no ice. For the floating bits of ice I'm thinking of masks using the fake stone shaders, using the gradation in the ice cover map to control density and thickness. Just don't take the camera under the ice or they'll see it's fake.
heh heh heh...and it looks really good...
This is crazy. You are crazy.
I like it.
Given the adult polar bear size, your ice edge seems to be about 5 meters showing above the water. That would make the ice edge there about 45 meters thick!
Mean first-year sea ice thickness in the last ten years has been less than two meters. Depending on the season, Most ice edge extent is first-year or seasonal ice. Ice piles up in ridges and makes some impressive cliffs, but mostly I think its a small drop to the water from the ice edge.
Old ice like the Ross Shelf in Antarctica and multi-year ice that does not melt can look just like your renders.
I think it would/will be awesome to have control over this as users of your model. Thick where we have old ice and thin for first-year ice.
I getting exited about the prospect of playing with it.
Do we get the Polar bear too ;D
Forge on,
Russ
Those sort of things were in the back of my mind while doing this but I've just used random numbers for now and haven't researched ice formations yet... Thanks for the kick start. I'm just building the basic model for now but keeping in mind the need to control settings and create masks for additional features such as you have mentioned. The massive ice shelves in the Antarctic are included in the DEM and this ice map adds in the seasonal sea ice. By combining the monthly coverage maps into a single greyscale image I think it would be reasonable to use this to also generate masks for different ice formations based on the age of the ice.
PS. The polar bear was a freebie off T3D. You can have the bear if you really want ;) Must check the scale of the bear. Ice is 2m above, 7m below in this
I usually read too much into things, so I am trying to temper my expectations here. But am I wrong to be really excited about what can *probably* be done with this?
What should I really be expecting out of the box here? From mid to high orbit (at least) will this be photo real?
I was thinking that was the point, and then the user would have to do detail work at ground level. But watching you get into the ice here, Now I am thinking even large swaths of the terrain will be photo real, say as close as 30meters or so. Even better? No?
especially when people post the link to that "wanderers" short film, in the other thread, and then Oshyan says the challenge is to do even better. Is this going to be better? What will be the distance between the finished model, and a finished high quality render, do you suppose? If that is even a fair way to pose the question.
:o
Quote from: TheBadger on March 16, 2015, 10:38:42 PM
What should I really be expecting out of the box here?
An excellent question... with a few answers, partly due to the way TG allows the combination of data (images) and procedural surfaces.
1. There will be a few different resolutions of data available. For the most part, this will simply mean pointing an image or DEM loader at a different file, but the TG file itself will remain the same. This is good for people with not much RAM, and also good for speeding things up for previewing work.
2. The maximum resolution of the terrain will be a bit higher than a 36K bump map (0.01°/px) along with 0.025° and 0.05°/px. At the absolute top end will be a set of 4 DEMs in 90x90° chunks (can't remember the exact resolution) but these require 6Gb of RAM just to load the DEMs. Modified Blue Marble texture will be available in the same resolutions as well as 0.005°/px and the original resolution (85K) ... with antialiasing, the Blue Marble image will be the first thing to go so that's the limit of how close you can get.
3. IF (and I'm kinda hopeful here) the idea of masking colours in the Blue Marble images and replacing them with coloured TG surfaces works, then this should get you at least twice as close as just the image alone... closer depending on how creative you get with the TG surfaces.
4. ... and this is where TG really shines IMO... You don't need the entire globe at insanely high res to get to the surface. Having geotiff support makes it really easy to produce multiple LoD data and load it without having to do anything special. This model will be the foundation on which you could add additional data where it's needed, whether it's for stills or an animation.
In GlobalMapper for example, I can export only the data I see on screen at the equivalent screen resolution. I run a screen res of around 1400px wide so for the USGS 10m DEM I get a 14km terrain at 10m/px. Zoom out and I get 28km DEM at 20m/px.... and so on. So it's quite easy to prepare data for a scene with extensive line of sight using relatively small data files. To avoid the need for masking terrains I always start with my global DEM in Global Mapper and then load in additional data on top of that. Then in TG it really is as simple as just loading all of the DEMS from low to high res. Repeat that with images and yes, in theory, it should be possible to do a kick arse ground to orbit animation. Horizontal flight obviously increases the need for more data, but the vertical transition would nevertheless be an impressive proof of concept.
You might want to take care with the use of fake stones for masking ice flows, as the borders are absolutely sharp. Unless it's a very thin sheet with no real displacement above the ocean, it's better (IMHO) to use a quite hard billow (warped by quite angular PF's if you will). But you will most likely know this, as an old dog ;)
I'm more than happy to take suggestions. The vast majority of my tinkering has been of a technical nature and I can assure you there's a lot I'm not very familiar with ;) I was leaning towards PFs as well, although for different reasons. Animating the extent of the ice over a year with fake stones would probably result in a lot of popping as the density map changed.
Quote from: bigben on March 16, 2015, 06:17:09 PM
Those sort of things were in the back of my mind while doing this but I've just used random numbers for now and haven't researched ice formations yet... Thanks for the kick start. I'm just building the basic model for now but keeping in mind the need to control settings and create masks for additional features such as you have mentioned. The massive ice shelves in the Antarctic are included in the DEM and this ice map adds in the seasonal sea ice. By combining the monthly coverage maps into a single greyscale image I think it would be reasonable to use this to also generate masks for different ice formations based on the age of the ice.
PS. The polar bear was a freebie off T3D. You can have the bear if you really want ;) Must check the scale of the bear. Ice is 2m above, 7m below in this
I should have known, you are way ahead of me. 8) I'll sit on my hands and watch this thing being born. My old mac has only 6 gigs of ram so Im delighted to learn you may have a "DEM lite" version to play with too.
I'm off to hunt polar bears now,
Russ
Added in a "flat" ocean model. Having real bathymetry and using a sphere for the ocean surface was working pretty well but at really high altitudes (1/2 - 2/3 of the planet visible) sections of the sphere were starting to disappear where the water was relatively shallow
DEM is still the same (including bathymetry) but the ocean bed is masked to be flat. The water colour is used to simulate depth (mask consisting of Displacement to scalar -> Colour adjust). This render is a simple 2 colour shader, but I'll dig up version I used for this: http://www.planetside.co.uk/forums/index.php/topic,2502.msg24609.html (http://www.planetside.co.uk/forums/index.php/topic,2502.msg24609.html) It should look better than that old project as the bathymetry data has improved since then.
Also found a glitch in the Black sea bathymetry in my model... back to the drawing board for that one, but I think it's an easy fix.
A quick terrain test using the 4 small geotiffs of 10, 20, 40 and 80m (total 16mb) resolution from another post with a 0.05° resolution global DEM. Last image swapped in the 0.01° resolution global DEM (2gb)
In the 4th image you can see the difference between the 80m DEM and the global DEM. The 80m DEM could be doubled in area, adding another 12mb or there is also a global DEM 0.005° resolution for those that have a lot of RAM. I don't think there'd be any problem with going from orbit to ground on the terrain front if you can get the data.
The rendering away from the top of the planet has certainly improved a lot. This is with the north pole at the apex.
Nice illustration Ben. The 0.01° data makes a noticeable difference and seems worth the memory use to me.
- Oshyan
It makes a big difference to the image as it should and I'd probably double the area of the smaller DEMs as well (only increasing to 64mb). The 2gb global DEM slows down the loading of the project into TG quite a bit on my home PC (open file, get cup of coffee) but once it's loaded performance is not too bad. I'll run some comparison renders at work on the full DEM set. The 0.005 degree DEM also makes a noticeable difference but it takes my home PC pretty close to max RAM usage.
Exporting a new DEM. Most of the SRTM data had no data over sea surfaces except for the Black and Caspian Seas. These have large blocks of identical altitude extending into the water, probably due to the presence of land at lower levels in the SRTM tiles (Black Sea altitude 0m, minimum altitude -89m; Caspian Sea -29m, minimum altitude -176m)
I'm going to ignore the Caspian sea for a little while so that I can release a mini beta to get some input from others here...
[edit]I think it will be easiest to release an additional geotiff to patch the Caspain Sea rather than figure out how to flatten it in TG (AND rotate it with the rest of the globe) .... oh crap... also figured out how to restore the SRTM data to low lying areas for the previous fix so that will be another 16hr export of the high res data.
OK, here's a little Easter egg for those following this thread. Attached is a VERY basic TGD with a mask image. It should uncompress to a TGD and a "data" folder with an image mask.
To this, you will also need a DEM to load into the empty Geog heightfield load. Here are the 2 low res versions I've made: https://cloudstor.aarnet.edu.au/sender/?vid=73861ca8-e40e-4f68-da38-00006512e7fb (https://cloudstor.aarnet.edu.au/sender/?vid=73861ca8-e40e-4f68-da38-00006512e7fb) (600mb, link expires 14/4/2015)
... and a texture map of your choosing for the Geog image map shader. I'm using the Blue marble images from here: http://visibleearth.nasa.gov/view_cat.php?categoryID=1484 (http://visibleearth.nasa.gov/view_cat.php?categoryID=1484) You want the images WITHOUT topography and bathymetry. ... but any full globe image will do. The final model will use a modified Blue marble image.
The other bits that I've shown in this thread are still at the proof of concept stage. Now that the DEM is done I can get back to polishing these into shape.
If you're doing renders from orbit, try increasing the Height multiplier in the displacement tab of Global DEM to 1.7 for some extra relief
Holy Smoke Ben those are huge, So are they applied to the planet or an added sphere?
he he, these are the small ones ;) Full release will include 0.01° resolution @ 2.1gb and 0.005° which is 4 tiles of that size.... and that is only 20% of the original resolution used to create it.
If you're setting up from scratch, Terrain > Heightfield Load DEM. Must turn off stitchable borders or you will get a groove from pole to pole at long 180°.
Quote from: bigben on March 24, 2015, 11:12:56 PM
he he, these are the small ones ;) Full release will include 0.01° resolution @ 2.1gb and 0.005° which is 4 tiles of that size.
If you're setting up from scratch, Terrain > Heightfield Load DEM. Must turn off stitchable borders or you will get a groove from pole to pole at long 180°.
muchos gracias
Thanks so much Ben. They are huge indeed, but I must try this, so your bits and bytes are traveling across the globe as of now.
They're big, but nowhere near as big as the source data they're derived from.
[attach=1]
Looking at those numbers now, I'm amazed at how GlobalMapper actually let me load it all at once and export the data with only 8Gb of RAM.
Too rich for my blood.
I'm out fellas. Your full house beats my two pair (of cores).
My best data is about 45 kb/sec.
I do, however, look forward to seeing some nice renders though.
Time for a cigar
As I'm setting up the masks I thought I might ask for some feedback... As a lot of this model relies on the use of masks there is the potential for the network window to become quite tangled. To help alleviate this, and to help making things modular, I'm adding Null shaders to the internal network of the output nodes of key masks and settings. This should also help to make things modular, as I can make a clip for a specific feature e.g. night lights and include links to related masks e.g. rivers and lakes (no city lights on top of water), without having to include the related masks.
Separate clips will be interchangeable for key components comprising masks and images, terrain, scene parameters and of course the different shaders.
It does,however, make it a little slower to follow the links as you have to open a node to see where the link goes to but I think it will be well worth it in the end. Whether or not you follow that convention with your own tweaks will of course be up to you.
Any thoughts ?
I like this kind of setup. I mostly work that way too, cleaning up into 'hidden' nodes after the main hurdles are taken, or even from the start. My only 'complaint' (which I mentioned earlier) is that you can't see where an outgoing arrow goes; a popup with a line to follow would be of great help.
I think I need a tutorial on the how to create and use of "hidden nodes"....please, searched the WIKI and nada there except references to them, but no how to.
It's very easy; say you have a surface shader with a lot of children and mask and breakup attached... CTRL+X those. Then rightclick on the now empty surface shader and choose internal. CTRL+V your nodes there. Then go back outside, hover your mouse over the child input, and choose the surface shader (you'll see 2 above eachother, the lower one with a link sign in front), open the lower one and you are able to choose the one node you want linked, which of course should be the last in the child line). The child input will then dissappear off the surface shader. Then hover above the mask input, get into the surface shader again and choose the mask shader. And finally hover above the breakup input, etc. This will hide all additives of that surface shader inside it. From inside you can also attach external links, which may be hidden inside other shaders.
Once you get the hang of it, it's easy. Like with everything ;)
Many thanks Ulco. I have this copied and pasted to a note pad and will see if I can grok this. I sure do hate the messy node screen I;m presented with after a while of making surfaces in the tree structure which I find easier at the out set. Off to test my comprehension.
Quote from: bobbystahr on March 27, 2015, 09:49:23 AM
Many thanks Ulco. I have this copied and pasted to a note pad and will see if I can grok this. I sure do hate the messy node screen I;m presented with after a while of making surfaces in the tree structure which I find easier at the out set. Off to test my comprehension.
Got it! ...a real
D'oh moment for me. Somehow the terminology confused me...thanks again.
The real secret to using these 'hidden nodes' is descriptive naming of the parent Surface shader of each hidden node so when it comes to 'tweak time' you have an easier time navigating.
Quote from: bobbystahr on March 27, 2015, 09:58:32 AM
The real secret to using these 'hidden nodes' is descriptive naming of the parent Surface shader of each hidden node so when it comes to 'tweak time' you have an easier time navigating.
Definitely, I've used a few capitalised keywords like PARAM, IMAGE and MASK at the start of the parent nodes to a) group them together and b) make them easier to spot in the list (except, of course, on the node in the example image above ;) )
Yes, naming is paramount! And if there are nodes that you'll use more often to link to put an A in front. Especially with a few hundred :o ), the list to scroll down is long.
Quote from: Dune on March 28, 2015, 03:11:13 AM
Yes, naming is paramount! And if there are nodes that you'll use more often to link to put an A in front. Especially with a few hundred :o ), the list to scroll down is long.
In making Distribution shaders which I often re use, when I name them I put an
! in front of the name and they're always top of the Right click>Assign shader list.
Hi Ben,
Is this what your underwater Red Sea DEM should look like? I need to displace it quite considerably to see some elevation (took the size from the file, but located near North Pole for ease of work). There's a strange band of height just off the land part as you can see.
It certainly shouldn't need that much. I'll check the file when I get home tonight. There are a few artefacts in the elevation data, both from my merging and the merging done in the bathymetry for GEBCO. There's no real escaping them but they shouldn't be too bad unless you're multiplying the elevation by 1000 ;)
Any updates on your progress?