Planetside Software Forums

General => Terragen Discussion => Topic started by: Ariel DK on October 23, 2015, 06:37:27 PM

Title: An ambitious project: Hawaii
Post by: Ariel DK on October 23, 2015, 06:37:27 PM
this will be another long and crazy project that I'll start, so do not be surprised if I update this thread once a week
and YES, i want to recreate the entire island. crazy, really? :D so let's open the drawing board:

1- I've downloaded the latest data about the island, and have been successfully loaded in TG. I can easily find places from real references

2- About SHADERS... here come the hard work :( but before I frustrate, I start to think that perhaps there were ways of texturing that I still  do not know, right?
    so I search in the forum something (or somebody) that can help me. but I could not find anything useful, at least, no directly...
    http://www.planetside.co.uk/forums/index.php/topic,7231.msg77061.html#msg77061

basically: How I can texturing certain areas, regardless of the altitude of the terrain? for, an example, get natural patterns for populations
I am talking about ecosystems

here some examples: the first one is the detailed DEM, the second one is the same (real) place that you can see how intrincated the pops can be.
thanks in advance for any help

Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 23, 2015, 08:17:15 PM
Most of what you see in the reference image really does derive from simple height and slope aspects. Most of the rest is based on hydrology, where the water flows, how moist or dry it is in given areas. This stuff you can simulate with imported or even natively generated masks. For example you can generate erosion with the built-in erosion shader and then do a sort of "difference" function to figure out where the major flow channels are and essentially create a basic flow mask *in* TG. Dune has shown some methods for this I think. Alternatively if you have a terrain editor like World Machine or World Creator you can erode the terrain there and just use the flow, deposition, and other masks, but on your original, un-eroded terrain. The shapes should be similar enough.

You could also potentially derive real vegetation masks from other real-world data that is available. BigBen has experimented with this stuff a fair amount, I think. In fact he has lots of threads with related ideas, like masks for trees by age of tree for more realistic forest distributions:
http://www.planetside.co.uk/forums/index.php/topic,5049.0.html

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 23, 2015, 11:12:31 PM
firstly, thanks for the answer and the link
Quote from: Oshyan on October 23, 2015, 08:17:15 PM
...This stuff you can simulate with imported or even natively generated masks. For example you can generate erosion with the built-in erosion shader and then do a sort of "difference" function to figure out where the major flow channels are and essentially create a basic flow mask *in* TG. Dune has shown some methods for this I think.

i am not sure what you're talking about here, you could be more specific about this? I gonna try to talk with Dune in anyway

about the BigBen´s file, as usual their work is fantastic, but doesn't work for all the island, I mean when I achieve that the pops works in a specific area, in other zones simply doesn't. so, maybe the best way here, will combining this mask, with another real dataset. I not try this yet, but i will do it in the next days. I'm also going to come back here often, so any other advice from any other user is very welcome. thanks ;)
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 24, 2015, 04:52:20 PM
just a funny update. in others places of the island, these same pops stink
Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 24, 2015, 04:55:40 PM
Good lord, why are you attaching a 3.2MB PNG!? With PNG *compression* it becomes just 800KB, and with JPG (because PNG is really unnecessary here, especially for a test), it is only 160KB with still great quality. Please, spare our servers and Internet connections. ;)

That being said, it's a promising test image. I just had to download it to even see it...

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 24, 2015, 05:00:44 PM
yeah but the file is not 3MB! strange... maybe I selected other image? anyway sorry, reuploaded ;)
Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 24, 2015, 05:12:34 PM
Quote from: Ariel DKMultimedia on October 24, 2015, 05:00:44 PM
yeah but the file is not 3MB! strange... maybe I selected other image? anyway sorry, reuploaded ;)

Better but still huge. Why use PNG? :D

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 24, 2015, 10:09:02 PM
Oshyan, indeed, there is no reason really, is just a PS preference when I save images automatically (usually I use PNG because I work with vectorial graphics, and this format preserve most of the quality)

Here is my first decent attempt at realistic shading denoting beaches, rocky areas and highlands. about the color data, I can't find nothing useful yet.
if anyone knows a website that I don't know, please let me know. :-\
Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 24, 2015, 10:19:37 PM
The National Map viewer has lots of satellite imagery ("orthoimagery"), Tree Canopy data (in "Small-scale datasets"), and "Woodland Tint":
http://viewer.nationalmap.gov/basic/

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 26, 2015, 12:03:10 AM
Hello, I sorry the delay, I downloaded The orthoimagery files in a ".Gz" extension. I dont know this type of extension.
What am I supposed to do with the files? more specifically, how I can load it in TG or PS? ???
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 26, 2015, 12:25:52 PM
Update
a very basic intent to create foam in a existent terrain, thanks to Dune for the basic information.
honestly, I not have much interesting in this yet, I am more focussed in the shading and pops
for now, but will be interesting to work in this in the future. ;)
Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 26, 2015, 01:45:49 PM
.gz is usually a compressed archive format. You can open/extract it with e.g. 7-zip.

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: WAS on October 26, 2015, 02:35:59 PM
Quote from: Oshyan on October 24, 2015, 04:55:40 PM
Good lord, why are you attaching a 3.2MB PNG!? With PNG *compression* it becomes just 800KB, and with JPG (because PNG is really unnecessary here, especially for a test), it is only 160KB with still great quality. Please, spare our servers and Internet connections. ;)

That being said, it's a promising test image. I just had to download it to even see it...

- Oshyan

This is why I use off-site hosts. Because one JPEG is not a good format for viewing renders and quality of said renders (and possible artifacts) and it's also just bastardizes images color and anti-aliasing. PNG/TIFF is the usual standard in ANY art forum for viewing things that need peer review. JPEG is for new users without comprehension of quality on a small scale.

For example, when I view art here on the forums I am constantly zooming in for fine detail. On most everyone images, that fine detail is ruined and not even offering a good viewing experience. Especially in image sharing where I expect from completed scenes, high quality renders and complementary standard formats.

JPEG is for camera shots on your phone. :P

This is 2015. I run 105mbp/s, no images here ever even have time to load visibly. Even my screenshot programs work off PNG. Even on my phone with 4G there is no real load time (even for his 2.5mb image)

Anyways, this is a good start. I'm working on similar with custom terrain from scratch. Very similar to Hawaii's peaks. I am actually taking a break from land detail and working on white peaks on my water too. How Ironic. Here are some examples.

(http://i.imgur.com/WDDNogb.png)

(http://i.imgur.com/KpBtzX8.png)






Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 26, 2015, 02:49:44 PM
JPG is a fine format if used properly and with a good compressor (there are a lot of bad JPG compressors out there though, it must be said). XnView has a great built-in JPG export engine and is free. I will put $10 on the line that you can't tell the difference between a JPG and PNG of an average Terragen image if compressed with the right settings (and viewed at 100%, i.e. not zoomed in to pixel level). And with those same settings the JPG will almost always be half the size *or smaller* vs. the PNG.

I typically use a Quality of 90, Optimize Huffman Table, the Float DCT method, and Subsampling of 1x1. At that setting a JPG is about 1/5 the size of PNG. At Quality 98 it's still half the size. At 98 quality, if I do a "difference" image in Photoshop it's black. Only if I crank up the contrast to 95 do I see anything (and only using the Legacy method). At Quality 90 the difference image has barely perceptible outlines, meaning the difference in the actual images (compressed vs. uncompressed) is still imperceptible to most people even side-by-side.

Just because we have high bandwidth connections and storage space is cheap doesn't mean it's best to use needlessly large files. If you want your art/images to have the broadest audience, think about more than just what *you* have available, think about what other people use to access and view your images. Think about metered connections, expensive and limited server space, *mobile*, etc. Those are all still factors in this modern age, and people are still working on better and better image compression schemes (lossy and otherwise) for good reason.

Zooming in to see "detail" to where you see pixels as larger than a pixel on your monitor doesn't seem that useful to me. *shrug*

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: WAS on October 26, 2015, 03:07:11 PM
Quote from: Oshyan on October 26, 2015, 02:49:44 PM
JPG is a fine format if used properly and with a good compressor (there are a lot of bad JPG compressors out there though, it must be said). XnView has a great built-in JPG export engine and is free. I will put $10 on the line that you can't tell the difference between a JPG and PNG of an average Terragen image if compressed with the right settings (and viewed at 100%, i.e. not zoomed in to pixel level). And with those same settings the JPG will almost always be half the size *or smaller* vs. the PNG.

I typically use a Quality of 90, Optimize Huffman Table, the Float DCT method, and Subsampling of 1x1. At that setting a JPG is about 1/5 the size of PNG. At Quality 98 it's still half the size. At 98 quality, if I do a "difference" image in Photoshop it's black. Only if I crank up the contrast to 95 do I see anything (and only using the Legacy method). At Quality 90 the difference image has barely perceptible outlines, meaning the difference in the actual images (compressed vs. uncompressed) is still imperceptible to most people even side-by-side.

Just because we have high bandwidth connections and storage space is cheap doesn't mean it's best to use needlessly large files. If you want your art/images to have the broadest audience, think about more than just what *you* have available, think about what other people use to access and view your images. Think about metered connections, expensive and limited server space, *mobile*, etc. Those are all still factors in this modern age, and people are still working on better and better image compression schemes (lossy and otherwise) for good reason.

Zooming in to see "detail" to where you see pixels as larger than a pixel on your monitor doesn't seem that useful to me. *shrug*

- Oshyan

Yeah, that's why I link to my own hosts to spare the site. And if you work in image augmentation and manipulation as long as I have you will see where it's crucial. First thing they do in a meeting is check for layer seams, bad pixels, color burning, JPEG artifacts, etc. And to do so they isolate areas they think look good, and check them on a greater scale. As when you create posters, prints, t-shirts, these images are blown up or down scaled depending on size. And if they are blown up, any artifacts from JPEG becomes very visible.  I'd be fired from most places if I thought JPEG was ok to use.  Most places don't even trust your PNG/TIFF settings and want the raw files that I've worked at.
Title: Re: An ambitious project: Hawaii
Post by: Oshyan on October 26, 2015, 03:19:01 PM
I don't want this thread to go off-topic for too long, so this will be my last response here. We can continue in an Open Discussion thread if you like.

I'm not saying that JPG is appropriate for everything, not at all. I am merely saying that it is absolutely a useful format and it has its place, which is to say, places where it is *more* appropriate to use than other options, including PNG, TIFF, etc. One of the places where JPG often makes more sense than other formats is on the web. This is especially true when the image being displayed is not intended for printing or other close scrutiny. That is likely to be true, for example, of work-in-progress images like all of the ones shown so far in this thread, and many others here on the forums. There's nothing wrong with posting the occasional lossless PNG image to show people your final, high quality resulting image, but it's pointless to avoid JPG merely on principle and use PNG for every image, even WIP.

The mark of truly knowing your image formats is knowing when to use each of them appropriately. They all have their place, even, to some degree still, GIF. ;)

- Oshyan
Title: Re: An ambitious project: Hawaii
Post by: WAS on October 26, 2015, 03:27:49 PM
I will agree with you there, but I would have you try to understand that in 2015 we assume (in most data-centers I also work at) that everyone is on High Speed internet. Those that aren't, are knowingly putting themselves at a inconvenience. For example, all the software I use to capture my screen and share with others uses PNG technology. The screenshot program I use uploads and saves in BMP, PNG, and TIFF. Because in this day and age we just prefer "HD" over anything else. This is where I casually go about linking PNGs and don't see it as something anyone should really gripe about. If it's really a issue with PNG's I can add a filter for you to exclude large PNGs in favor of JPEG uploads for the site! Not trying to say "You are wrong!" just saying it's become rather standard now.

Anyways, yes let's keep on topic.
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 26, 2015, 06:00:02 PM
about the format discussion, I can't said nothing that no was said here, WASasquatch have their point, but also Oshyan, when he
said that "The mark of truly knowing your image formats is knowing when to use each of them appropriately". ;)

but keeping all on topic: I must say that your work is very interesting, WASasquatch. is a DEM? or, what do you meant with "custom terrain"?
Title: Re: An ambitious project: Hawaii
Post by: WAS on October 26, 2015, 08:59:24 PM
Quote from: Ariel DKMultimedia on October 26, 2015, 06:00:02 PM
about the format discussion, I can't said nothing that no was said here, WASasquatch have their point, but also Oshyan, when he
said that "The mark of truly knowing your image formats is knowing when to use each of them appropriately". ;)

but keeping all on topic: I must say that your work is very interesting, WASasquatch. is a DEM? or, what do you meant with "custom terrain"?

It's a custom .ter file from L3DT. It's huuuge, took awhile to render out all the erosion levels. The tree population in that image is 10,000 meters. So I may need to resize the island a bit. :P
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 27, 2015, 02:23:58 AM
Quote from: WASasquatch on October 26, 2015, 08:59:24 PM
"...The tree population in that image is 10,000 meters. So I may need to resize the island a bit. :P
hehe my pops are indeed, much more bigger than the yours, but "clip on camera" is activated (except for test images obviously), my advice for you, if you ask me: resize the pops, never the island ;)

I've been a little busy today, but returning to the game, I unzipped the .gz data (thanks to Oshyan for the tip), and for now, I'm using it as a good reference for when I add the procedural shaders. I also plan maybe to use this as a base for these shaders, you know... step by step
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 27, 2015, 06:36:40 PM
just a test of the procedural shaders. all the detail is there, even though that this is based only on simple masks, looks well from ground. so, well, the funny part starts here... 8)
Title: Re: An ambitious project: Hawaii
Post by: WAS on October 27, 2015, 07:08:35 PM
Wow, those are very nice! Very well done! Definitely can tell Isla Nublar's inspiration came from here.

I really especially love these three spots. The detail looks so well done.

(http://i.imgur.com/SwgWO2e.png)
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 28, 2015, 12:42:13 AM
thanks, I can say that soon there will be all kinds of plants there. and the size of the most biggest population, have now 130k x 160k, likely the size of the island.
however, some zones (especially south and southwest) looks more like Iceland with tropical tone :o

meanwhile, i need to tweak some things here and there
Title: Re: An ambitious project: Hawaii
Post by: Hannes on October 28, 2015, 03:40:03 AM
This is really cool!
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on October 29, 2015, 10:37:02 PM
populating palm trees, and all kind of plants native from Hawaii

Unfortunately, I can not deal with all the vast populations in the scene. the memory of my PC 16 gb RAM, does not seem to be enough to do the job. the best I can do now is to simply, placing the camera where I want to render, and start to populate from there.

now, looking forward to render volcanic coasts, and black sand beaches.

hey Hannes, you can imagine to Vader vacationing over here? :D
Title: Re: An ambitious project: Hawaii
Post by: AP on October 29, 2015, 10:48:13 PM
There is a thread here about Black Sand.

http://www.planetside.co.uk/forums/index.php/topic,19164.0.html

It is looking quite interesting. 

Vader would need some serious AC and dehumidifying controls in his suit. 

Title: Re: An ambitious project: Hawaii
Post by: TheBadger on October 29, 2015, 11:21:49 PM
following along. Very cool!
Title: Re: An ambitious project: Hawaii
Post by: Hannes on October 30, 2015, 02:51:34 AM
Quote from: Ariel DKMultimedia on October 29, 2015, 10:37:02 PM
hey Hannes, you can imagine to Vader vacationing over here? :D

;D ;D ;D Yes, Vader AND me!
Looks great!
Title: Re: An ambitious project: Hawaii
Post by: WAS on October 30, 2015, 04:06:17 PM
Populations look great! If I'm not mistaken, there is a way to use a projection camera for populations?
Title: Re: An ambitious project: Hawaii
Post by: Ariel DK on November 05, 2015, 01:06:35 PM
just updated this thread, because I don't want somebody to think he's dead. I still working on this...

Quote from: Hannes on October 30, 2015, 02:51:34 AM
Quote from: Ariel DKMultimedia on October 29, 2015, 10:37:02 PM
hey Hannes, you can imagine to Vader vacationing over here? :D

;D ;D ;D Yes, Vader AND me!
Looks great!

;D ;) thanks

Quote from: WASasquatch on October 30, 2015, 04:06:17 PM
Populations look great! If I'm not mistaken, there is a way to use a projection camera for populations?

hi WASasquatch, i don't know what do you refer with "projection camera for populations", the most similar thing that I know is "Clip Camera", a little orientation here is welcome (sorry for my ignorance) ;)

Meanwhile, I leave this render to enjoy...
Title: Re: An ambitious project: Hawaii
Post by: Dune on November 06, 2015, 04:05:43 AM
Very moody. But I think the vegetation is too dark. Perhaps lighten up the parts, or increase GI a bit...
Title: Re: An ambitious project: Hawaii
Post by: AP on November 06, 2015, 05:18:40 AM
Also maybe check the translucency on the leaves.