Feature Request: Heightfield generate Use shader Improvements

[D.A. Bentley (SuddenPlanet)]

I couldn't find a specific thread for Feature Requests, so hope this is the right place.

I do a lot of terrain extractions using the Heightfield generate "Use shader" tab.  After doing the Generate I save the node contents as a TER/EXR.

One thing that would make it a little easier to use is a visualize box showing the bounds of the heightfield boundary.  Now I just make a simple shape shader the exact size of what I want to extract to get the dimensions I need in meters.  Typically I am extracting large terrains measuring 228000m x 334000m (228km x 334km) into 32k (32768 x 32768) heightmaps.  It would just be nice to have that right in the Heightfield generate node so I could toggle it on and off to easily and quickly verify my terrain is being entirely captured.

The other thing that would be even a greater help would be to have the Heightfield generate node be Multithreaded.  Generating 32k heightmaps can take many hours and often times it takes so long I have to do them overnight to be efficient.  

I think this would be a great feature to improve as I think a lot of Gaea and World Machine users would find Terragen even more useful if it was possible to quickly and easily get heightmaps / heightfields exported from Terragen.  With the 32-bit TER/EXR save options the quality is very good.

Derek

[digitalguru]

Have you seen this?
https://planetside.co.uk/forums/index.php/topic,21311.0.html
https://www.youtube.com/watch?v=bTKfAeEPDhY&feature=youtu.be

This is for vector displacement, but in the Youtube example, you should just be able to replace the Displacement shader to vector with Displacement shader to scalar to get a heightfield.
Then you treat this as a render and throw as many threads as you have at it
And, as it is based on an orthographic camera, you can define your capture area with precision.

[D.A. Bentley (SuddenPlanet)]

That sounds promising.  I'll check it out and see if I can make it work.

Thanks!

Derek

[Kadri]

Have you seen this?
https://planetside.co.uk/forums/index.php/topic,21311.0.html
https://www.youtube.com/watch?v=bTKfAeEPDhY&feature=youtu.be

This is for vector displacement, but in the Youtube example, you should just be able to replace the Displacement shader to vector with Displacement shader to scalar to get a heightfield.
Then you treat this as a render and throw as many threads as you have at it
And, as it is based on an orthographic camera, you can define your capture area with precision.

I have forgotten your post. Thanks for the reminder.

[D.A. Bentley (SuddenPlanet)]

Have you seen this?
https://planetside.co.uk/forums/index.php/topic,21311.0.html
https://www.youtube.com/watch?v=bTKfAeEPDhY&feature=youtu.be

This is for vector displacement, but in the Youtube example, you should just be able to replace the Displacement shader to vector with Displacement shader to scalar to get a heightfield.
Then you treat this as a render and throw as many threads as you have at it
And, as it is based on an orthographic camera, you can define your capture area with precision.

It works!  The "Displacement shader to scalar" replacement produced a nice 32bit EXR Heightfield.  My test was at 4096, and it looked great.
Of course, I had to convert it to 16bit to look at the detail in Photoshop.
I'm rendering at 32k now to see how long it takes.  I'll let you know when it's done. 

Now I'm curious how big I can render out a VDM for import into Modo & ZBrush.  So many things to try!

[D.A. Bentley (SuddenPlanet)]

OK here are the render speed results.  It takes just under 2 hours to render a 32k VDM, and about the same for the 32k Heightfield.
That was with Micropoly detail at 1, AA at 4, and using Robust Adaptive Sampler.

The VDM renders appear to look good but after further inspection the heightfield renders have some weirdness.
From a distance the heightmap looks ok, but when looking up close there are some black edges appearing in many places that shouldn't be there.

Also factoring in that it takes 2 hours per 32k heightmap, it doesn't seem worth it to use this technique in my case, since I can open 8 instances of Terragen easily on my rig with 128GB of RAM and do 8 simultaneous 32k Heightfield Generates from Shader in 9 hours (and I don't have to babysit my PC for those 9 hours).  It's great for VDMs though.  Just not out performing my Batch Method of using Heightfield Generate from Shader.  Also the quality is dependent on knowing the correct render settings, so it's a bit more tricky.  I still haven't figured out what's causing the strange artifacts.

Next I will experiment more with the VDMs in ZBrush and Modo.
If anyone knows what is causing these black artifacts that appear kind of like edge creases let me know please (see last three attached images)

Derek

[Dune]

They almost look like shadows.

[digitalguru]

They almost look like shadows.

I suspect diffuse shading has snuck in there.
EDIT: It has, see posts below:
Tip: If you're in need of a decent program to look at and manipulate 32bit exrs try Affinity Photo - very like Photoshop and has a far, far better handling of 32bit float files and is ridiculously cheap (about $33)

Also the quality is dependent on knowing the correct render settings,

Render settings are fairly simple, turn everything off! - GI, shadows, atmo should be off. Aliasing at 3 - most important is Microply detail at 1.0 (which is one micro polygon per pixel)

I'm curious about your multiple instances of Terragen, is it to export 8 different terrains? Or tiling one terrain?

Tiling terrains is my preferred method, the files become more manageable and with the render method, you can animate your camera to render a sequence of tiles. I don't have Modo or Zbrush though so not sure if it can deal with tiled input images.

Send me your project file if you can, and I'll take a look.

That original Youtube tutorial was done a few years ago and  Terragen has changed since then, so I'll make a updated version and post it here (for Terragen Professional)

p.s Just noticed you must have the Professional version as one of your renders is the Surface Position render element - you don't need this

[digitalguru]

I've just tried a quick test and it does seem the Displacement to Scalar node adds in diffuse shading to the render (it shouldn't) - will investigate further this evening.

In the meantime using Displacement to vector seems to output the correct results and you can just isolate the green channel in the .exr to get the heightfield.

Will let you know how I get on

Edit: Found it :-) Will post an example scene this evening.

[D.A. Bentley (SuddenPlanet)]

I attached the scene file I used if you still want it.  All I did was use the tutorial file "terragen_vector_tutorial_end.tgd" and removed all the fake stones, and then replaced the Displacement shader to vector with Displacement shader to scalar.  I was also using the latest version of Terragen 4.5.  I'll try on 4.4.67 next.  I did notice a couple settings in my file that might have affected the render though.  On all my loaded DEMs I had Fractal Detail turned on (On by Default).  Also after I removed the Fake Stones the Default shader still had a Displacement multiplier of .01.  Not sure if those things would matter, but I'll see in my next render.

Also Affinity Photo is on sale right now for only $24.99 so I got that to try out.  Thanks for the tip.

On another note, I still don't understand how a VDM works.  I keep thinking about how it first raises the surface (green channel right?) and then uses the Red & Blue channels to offset in X & Z, but what if your terrain has many mountains, or pillars sticking up?  How can the map do the offset for more than one surface (like surfaces behind other surfaces)?  Am I making sense?

Derek

[D.A. Bentley (SuddenPlanet)]

I'm curious about your multiple instances of Terragen, is it to export 8 different terrains? Or tiling one terrain?

I am exporting different terrains when I run multiple instances of Terragen.  The limiting factor for me is amount of RAM because each Terragen scene I load has between 9 and 35 DEMs loaded.  Also another limiting factor is screen space.  It gets too hard to manage more than 8 instances when I only have two 2k monitors.  I was going to try Windows 10 Virtual Desktop feature, but even if that works I still would be able to add only 2 or maybe 4 more instances.  Basically what I am doing is using DEM data to create a bunch of base heightmaps to bring into World machine and Gaea to manipulate, add detail, colorize etc.  Also since I am rendering out huge 32k terrains, I have to use World Machine to split them into tiles afterwards (4096 8X8 GRID).  World Machine names them nicely so that's why I use that method.


I considered rendering tiles out of Terragen, but then I would have to do some manual file naming to get the tile set recognizable by World Machine.

[D.A. Bentley (SuddenPlanet)]

p.s Just noticed you must have the Professional version as one of your renders is the Surface Position render element - you don't need this

Which render looks like Surface Position?  I didn't render that element, but if your referring to the green heightmap image that's just the VDM converted to 8-bit.


Also I loaded the VDM into Photoshop and just copied out the Green channel into a new Grayscale image, and that works perfectly as a heightmap.  So basically there is no need to use the Displacement to scaler node, as you can just render out the VDM and get both a VDM and heightmap in one render. 

[WAS]

You should be able to swap files with frames, and render frames, each frame could ve tile-x_0001.exr through whatever max is.

[D.A. Bentley (SuddenPlanet)]

World Machine uses a specific x/y naming so not sure that's possible in Terragen.  How would you tell each frame to be a specific xy location in the file name?

[WAS]

I'm not sure. From the looks of it you'd need to setup batches.

X being a category, Y being it's content. So you could hard set _x0_ and then render out frames 0-7 for Y, then the same for x1, x2, x3, etc.

Just a thought.