Planetside Software Forums

General => Terragen Discussion => Topic started by: Ariel DK on January 14, 2015, 12:36:59 AM

Title: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on January 14, 2015, 12:36:59 AM
hello everyone! this is my first posting on Planetside Forums
I began to use Terragen more than a year ago because since I study art multimedia and 3D graphics, I always wanted to recreate the Earth from space
but soon I realized how hard it was ... many people underestimated how hard it is to recreate our planet, regardless Use software package for it (I tried with C4D, 3ds Max, and Vue ), but only terragen proved to be the most convincing to achieve my goal
I've seen hundreds of videos from NASA and beauty is extremely difficult to recreate (or at least so I thought until I saw the film "Gravity")
here are obviously tons of shader written for a renderer called Arnold
however can be the basis of this knowledge be used in TG3?
and if so ... how?
I tried using textures from NASA, but are only useful for the surface (86k textures). for clouds, there is only one map 43k (very bad at altitudes of 300 km)
I found it impossible to recreate some type of similar cloud whose pattern is similar to the real clouds seen from space (or at least similar to clouds texture pattern 43k)
I know this is an old topic, but never has been taken so seriously. I just want to create an earth that seem real and I am convinced that Terragen is the right tool for this job
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on January 14, 2015, 12:42:55 AM
here a test render of my work
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Dune on January 14, 2015, 03:05:27 AM
Welcome to the forum. You might want to check out some files available at NWDA; there's global clouds and global worlds. Saves you a lot of work.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on January 14, 2015, 03:17:35 AM
This is also a strong interest of mine, so I'm happy to support your work as much as I can. I can say that of course the NASA data is the best source, but I have found it to be inconsistent in resolution and quality. I have been hoping to work with another forum member, BigBen, on a project to create the best possible, easily used, ready to apply out-of-the-box texture set of Earth data, including color, height, clouds, city lights, specular, water, maybe even bathymmetry, but that project hasn't get been started.

In general you can get good basic results just by using the existing NASA data. But as you found, it works best from higher altitudes. It is inevitable that global data will be limited in resolution because as resolution increases, data size increases, and memory use also increases. So even if you have 64GB of RAM, you will soon be out of memory working with huge 50,000 pixel square images/textures/displacement maps/etc.

As for the clouds, I think the best thing to do is work on a setup where the NASA map is a mask or other modifier for the Density Shader of a Cloud Layer in Terragen 3. This allows you to add procedural detail to the NASA data, which works on a smaller scale and thus at closer camera distances. The trick is in getting the right procedural function scales and patterns to complement your NASA cloud map and the right densities, depth, altitude, etc. as well. It may be best with multiple cloud layers all with varying settings, but using the same or similar variations of the NASA cloud mask.

There are a number of threads around here on the forums where several people have achieved varying degrees of success with Earth renders. It's worth looking around and trying a few different searches to see.

And, as Dune mentioned, there are some more procedurally-oriented full planet scenes and presets available at NWDA, which could help with some aspects:
http://www.nwdastore.com/

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on January 14, 2015, 03:24:23 PM
thanks for your answers, I agree with Oshyan that the solution is given on finding the right way to mix some texture for clouds with procedural details, or maybe with that package clouds in NWDA
the truth is that I've already tried, but the main problem I had was that this method worked well for the vast cloudy areas, but not very well for small clouds (the smallest completely disappeared), the result was that had vast cloudy areas, and others areas completely clear
I will try with those clouds in NWDA
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: mhaze on January 15, 2015, 04:33:05 AM
You could combine several cloud layers to achieve the small details.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on January 16, 2015, 01:21:58 AM
you are right, but I never tried.
refers to connected a cloud texture like a Final Density Modulator, and use a Power Fractal Shader maybe?
unfortunately I really have not had time for this
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 16, 2015, 04:26:28 PM
while I have not been to this topic .. now the Iceland contest is ending I have time to go in this..
in fact I have new promising results
I decided change the mask for a "semi- procedural" cloud layer
I used a Alpha image of continents for create cumulus, cirrus and others
here is a fast preview
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on February 16, 2015, 04:28:36 PM
That looks very promising. Are you using other image-based data for cloud positioning/shape at all, or just the continent mask?

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 16, 2015, 04:33:46 PM
is only a alpha mask with many Power Fractal Shader
had previously proven with a relief image (black and withe) and 24 bits per channel
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 16, 2015, 04:38:10 PM
not work very well :-\
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 17, 2015, 01:08:07 AM
after a laborious night I'm on the right track
the "semi-procedural" clouds seem to adapt well to the geographic patterns (especially cirrus clouds)
but this is only the beginning .. I will need more tests soon
I go to sleep, good night to all
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 17, 2015, 01:09:54 PM
a new update
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 17, 2015, 01:17:40 PM
the next step is a little highly detailed cumulus layer
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on February 17, 2015, 06:06:53 PM
This is coming along really nice, you're on track for sure!
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on February 18, 2015, 04:02:12 AM
Yes, your clouds are coming along very nicely.  I'm still downloading data to patch together. Half way through the tiles for GMTED2010 when I discovered how crappy the Himalayas are..  It's beeen a few years since I put up with Earth Explorer's "thou shall have only 4 simultaneous connections and thou shall not automatically harvest data"  It's a good thing nerds have a lot of patience  ;)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 18, 2015, 08:48:40 AM
Well, I'm also having a stupid problem with a dataset too
in my case it is nothing less than the dataset NASA earth 86k color + topo
I have already downloaded the entire southern hemisphere, but I can not load other plates with Chrome
I increased my speed internet, but happen exactly the same: the textures never load
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 18, 2015, 08:54:42 AM
Another update
I conformed with this but there are still areas where clouds seem to behave oddly
300km above Sumatra
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on February 18, 2015, 02:23:20 PM
Quote from: Ariel DKMultimedia on February 18, 2015, 08:48:40 AM
Well, I'm also having a stupid problem with a dataset too
in my case it is nothing less than the dataset NASA earth 86k color + topo
I have already downloaded the entire southern hemisphere, but I can not load other plates with Chrome
I increased my speed internet, but happen exactly the same: the textures never load

I generally use Firefox with the Download Them All extension. It gets the files in multiple segments at the same time which helps speed things up and it's pretty good at resuming stalled downloads.

I'd suggest using the images without topo as they have shading which can look quite odd closer up if the sun position doesn't match it's shadows. Send me a PM with your email and I'll send you mine.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on February 18, 2015, 02:34:23 PM
And on the masking side of things.... I'd suggest using a colour adjust shader after each key component of your mask. I find it much easier to fine tune distribution masks with this.  And as clouds take a while to render, use the same mask on a surface shader. It also makes it a little easier to see if a problem is located in the cloud settings or the mask.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 19, 2015, 03:26:03 PM
Thanks, use surface shader is a good advice bigben, I'll try it later
for now I'm focused on creating clouds with a "semi-procedural" aspect
with an alpha image (to restrict the clouds to certain areas),with a little noise variation.
the best of this method is that the resolution of the image should not be at all high
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 21, 2015, 12:08:44 PM
I bring good news about masking clouds
I have found with enough research on internet,
there are satellite images calls AMO.
constantly shows the Color of the Earth and clouds
in total there are at least 2 million of these images, each with a resolution of 6k or less
although the frames are different days and years, they can be easily edited and combined
into potoshop, to create the first map of clouds at 100K resolution!!!
Unfortunately, I don't know if the best solution
to a manageable model of the Earth  :-\
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on February 21, 2015, 01:30:52 PM
I've seen a few low res texture maps for clouds that were "artistic composites".  They produced a few different versions by patching in local sections, e.g. A version with relatively clear skies and another version with some of the gaps filled with cyclones.  As long as you keep the latitude for an image section the same, you can move it east/west to suit.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 21, 2015, 05:28:39 PM
Quote from: bigben on February 21, 2015, 01:30:52 PM
I've seen a few low res texture maps for clouds that were "artistic composites".
Again you are in the correct Bigben
but this technique should expect
because I've gotten amazing results
using only "semi-procedural" layered clouds
pd: ignore the resolution of the textures, I'm downloading new data ;)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on February 21, 2015, 08:10:56 PM
The last time I played around with cloud masks was with Hurricane Katrina. One of the approaches I used there was to create a mask image in ImageJ for the small dense cumulus formations by filtering by particle size.  By playing around with contrast and particle size you can pull out a few of the different cloud types.  If you can get your hands on images at multiple wavelegths you can mix and match to get some other combinations. I think I'll be aiming for 3 cloud masks combined in an RGB file (just to keep the file count down) but I'll see when I get there.

I also have a water mask for continent outlines. A blurred version of this could be used to mask by distance from the coast which might be useful.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on February 21, 2015, 10:02:58 PM
thanks again, but for now I'm just focused
in the atmosphere settings
However, I foresee a complication when I it begins
to load large data sets, because I have only 8GB RAM
will this be enough??
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on March 12, 2015, 11:15:15 AM
more news..
who is this guy??  :o
http://www.turbosquid.com/3d-models/earth-clouds-256k-x/828437
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on March 12, 2015, 03:21:02 PM
Quote from: Ariel DKMultimedia on March 12, 2015, 11:15:15 AM
more news..
who is this guy??  :o
http://www.turbosquid.com/3d-models/earth-clouds-256k-x/828437

at $1300.00 approx. for the model; no one I'm ever likely to know, hee hee hee
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on March 12, 2015, 05:53:57 PM
I don't even see any mention of Displacement Maps. Without that a million poly sphere isn't going to have nearly enough detail for any decent height representation, so it becomes just a flat Earth with nice textures. I'm not certain of the source of his data, but if I recall correctly Blue Marble has max resolution data that might stitch together to be that resolution or higher. There are some errors or visual artifacts in some of his example images too. Wonder if anyone is paying that much... eek.

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on March 12, 2015, 10:58:13 PM
Blue Marble stitches together to 85k. The resolution on these images looks about right for the 256K mark, but it's not as clean as Blue Marble. Significant banding between satellite passes in the Amazon example.
Here's a comparison with one of the Blue Marble JPEGs (coincidentally a 1:1 crop)

...and yes, no sign of any terrain. "The best results near the surface ever seen" if you call 50km near ;)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on March 12, 2015, 11:36:26 PM
Quote from: bigben on March 12, 2015, 10:58:13 PM
. "The best results near the surface ever seen" if you call 50km near ;)


hee hee hee
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on March 12, 2015, 11:41:26 PM
He has an el cheepo? version at $299.00:
http://www.turbosquid.com/3d-models/solar-32k-3d-3ds/631809
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on March 12, 2015, 11:42:57 PM
In jis defence he does do some pretty damn nice air craft and the odd auto:
http://www.turbosquid.com/Search/Artists/Galva858
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on March 13, 2015, 09:40:24 AM
if I pay that price, would only by huge textures
but I'd say it would not be good business ...
if any ask me, this is the best earth model I see in the web
in this case, the artist use Houdini
ironically the only problem of the artist, are the huge textures (my same problem)
http://project-eden.blogspot.com.ar/
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on March 13, 2015, 07:09:23 PM
Sure Ben, banding. But he has set the bar. Can we beat him? ;)

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on March 13, 2015, 08:55:10 PM
I'd need to find the data source to start with. I'm assuming it's a public dataset. I don't think it would be practical to use that resolution at a global level as well as a high res DEM, but if you're rendering at a viewpoint where you can see the difference, you won't be looking at a full globe anyway.  Unlike 3DS we're using a georeferenced planet and not just a sphere so we have a lot more flexibility for mixing global and local data sets. 

On the elevation front, I'll see if I can find the link again, but there is a project to do for Earth what HiRise has done for Mars... 2m DEM via photogrammetry.One of their focus areas is Greenland glaciers.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on March 14, 2015, 01:06:13 AM
Quote from: bigben on March 13, 2015, 08:55:10 PM
On the elevation front, I'll see if I can find the link again, but there is a project to do for Earth what HiRise has done for Mars... 2m DEM via photogrammetry.One of their focus areas is Greenland glaciers.

hmmm do not forget that Mars is smaller than Earth
satellite orbits are much lower
is no a surprise to find 2m DEM of mars
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on March 17, 2015, 05:05:05 PM
a small upgrade
this is how they look some cumulus clouds in my model of Earth clouds
about 70 km altitude (Note that these clouds were not set for this altitude)
I am facing new problems, such as the elevation
and the manner in which the surface seems to "swallow" the clouds, in areas like the Himalayas
as well as also, a new level of orbital realism by adding clouds located in different areas or "sceneries" of Earth
I am also thinking of adding another layer of "localized clouds"
for shots from the ground to the space
and of course, as always, any advice is welcome
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on March 18, 2015, 07:13:46 AM
Altitude software might help for the range of altitudes
http://www.planetside.co.uk/wiki/index.php?title=Cloud_Layer_v2_-_Functions_Tab (http://www.planetside.co.uk/wiki/index.php?title=Cloud_Layer_v2_-_Functions_Tab)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on March 18, 2015, 10:37:19 AM
Quote from: bigben on March 18, 2015, 07:13:46 AM
Altitude software might help for the range of altitudes
http://www.planetside.co.uk/wiki/index.php?title=Cloud_Layer_v2_-_Functions_Tab (http://www.planetside.co.uk/wiki/index.php?title=Cloud_Layer_v2_-_Functions_Tab)

great page of movies...I'd not seen that yet. Thanks Ben.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on March 18, 2015, 04:48:08 PM
Yes, it's a good reference for settings.  Not sure how "offset" got changed to "software" in my post. I'll blame TOS ;)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bobbystahr on March 19, 2015, 02:06:12 AM
Quote from: bigben on March 18, 2015, 04:48:08 PM
Yes, it's a good reference for settings.  Not sure how "offset" got changed to "software" in my post. I'll blame TOS ;)

hee hee hee
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on March 29, 2015, 01:23:45 PM
have overcome the problem of altitude with clouds, and now it looks fantastic
I only need add more realistic patterns, large-scale
this is in the same scene
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on March 29, 2015, 02:24:31 PM
Very promising!

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on March 29, 2015, 05:50:26 PM
Looking good. Here's a TGD that demonstartes how I'd add the full res blue marble images (First 2 tiles, but you get the rest)  Further to my PM, you can use the PNG images without converting.  The colour adjust and adjust saturation node do nothing in this particular case, but they provide most of the controls you'd need to tweak the colour of the images
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on March 29, 2015, 06:34:43 PM
I also had a look at the cloud clip you shared.  For using the blue marble image to restrict the clouds to over the land can I suggest adding a Colour to greyscale colour function and a Colour adjust function after the image(s).This will give you a bit more control over the distribution e.g. by using a negative black level you can add a lower density of the cloud over the water.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: TheBadger on March 29, 2015, 07:57:30 PM
 8)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on March 29, 2015, 10:16:29 PM
I go to try  ::)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on April 01, 2015, 03:18:12 PM
NEW APPROACH
hi all, long ago, I did not see real images of our planet.
I found a new (and perhaps obsessive), problem here.
I will see that TG is capable of creating extremely detailed clouds
(I can go from the orbit to the surface without problems), but the reality is visually different to what we can show in terragen... why?
It is very simple: our planet is full of extremely complex patterns of clouds ... and it's not a joke
These patterns are very visible and cause a huge visual impact on the orbital landscape of the planet
however, all these patterns can be recreated in TG: http://gannaingh32.deviantart.com/art/Orbit-152516845 or  http://www.planetside.co.uk/forums/index.php/topic,8624.0.html
but, to all this immense work, we must add the fact that these patterns are closely related to land masses, since clouds tend to "adapt" or respect the coastal patterns.
Also, certain types clouds are formed only in specific areas such as peninsular areas and large islands (florida and oceania, for example)
IMHO, it makes no sense to create a realistic planet Earth, if in the end, you end up giving yourself artistic license with something as important as the clouds.
some ideas to achieve these "Located clouds patterns"?
This is an example of what I'm saying:
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on April 01, 2015, 04:48:54 PM
Well, I think your idea(s) of creating masks based on continent shapes is a good start. Some of BigBen's work on procedurally-generated masks should help. There may be ways of creating other "selectors" in procedural ways, e.g. edge selector for coast-appropriate cloud formations. That's something you could look into. Ben is pretty good with those kinds of techniques. You could also work on creating procedural noise functions to act as a mask for these cloud types, and then just aim for macro shapes in these procedural noises that emulate the kinds of large-scale shapes and distributions you see for different cloud types (as shown in your example image).

However you go about it, it will take a lot of work. But it seems possible to get some really good, realistic results, and it's definitely an important part of the "realistic Earth" concept and project.

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on April 01, 2015, 05:27:34 PM
It is indeed complex, but quest for attempting this can also be very illuminating. The same problem exists with the surface of the planet. Short of city lights, my model has no cities or other human infrastructure.  For the atmosphere though, there are some possibilities to extend the concept further.  My Earth model includes a 36k mask of the land masses for masking the ocean and beyond that low resolution image maps of climate-influencing factors could be used to provide additional "true to life" variations. 

The tricky part is describing the variations in ways that can be implemented, but let's take mountain ranges as rain shadows as an example....  You would need masks based on slope direction and altitude - straightforward, and a vector for the trade wind direction ... a little trickier but possibly describe the average direction of the wind based on latitude. Combine those 2 and you may be able to have clouds forming more on one side of a mountain range than the other.   

It is indeed a complex challenge but there is much to be learnt in its pursuit ;)
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on April 01, 2015, 06:09:31 PM
Quote from: bigben on April 01, 2015, 05:27:34 PM
low resolution image maps of climate-influencing factors could be used to provide additional "true to life" variations. 
all is interesting! refers to something like this?
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: bigben on April 02, 2015, 11:52:00 PM
That depends on what it's mapping ;)  I'm thinking things like solar irradiance
(http://i.imgur.com/xg1JFKj.png)

or rainfall
(http://www.climate-charts.com/images/world-rainfall-map.png)

or rainfall rate
(http://oceanworld.tamu.edu/resources/environment-book/Images/globalrainmap.jpg)

etc...
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on May 26, 2015, 11:21:00 PM
FINISHED?

I decided to revive this thread for last time...
and well, this is also my last post for some time
I'm a bit disappointed with my final result, because I wanted to create a semi-procedural clouds (or fully procedural) for my earth model
but the final result was a high detailed masked clouds, very effective at 500 km altitude, but not as much under 150 km altitude.
there's always room for improvement, so any advice is welcome.
the results speak for me:
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Oshyan on May 27, 2015, 04:14:16 PM
I think your results look rather good. What is the source of the mask?

Remember that this is a very hard problem. No one else has fully solved it, in Terragen or any other program. I think your results here are some of the best I have seen in terms of accuracy and realism in an orbital view. Some of your previous images that I think were entirely procedural were also very good. So you have made some good progress, you have "pushed forward the state of the art". There is still work to be done, but this is a step closer. :)

- Oshyan
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Dune on May 28, 2015, 02:20:35 AM
Welcome to the forum, Allen. And thanks for posting your work. I'm sure a lot of users will find it useful.
Title: Re: it is possible to accurately recreate Earth in Terragen 3?
Post by: Ariel DK on May 28, 2015, 11:35:06 AM
Hi Oshyan, well, this is only the standard 43k cloud map by NASA, (blue marble collection). I've just created three detailed clouds layers, with a high contrast in the density node. nothing special. ;)

and welcome to the forums, Allen. I really appreciate your interest and any support.
I'll also post the TGC file, in file sharing.