Planetside Software Forums

General => Terragen Discussion => Topic started by: vinyvince on July 21, 2018, 07:18:03 PM

Title: Export Terragen terrains and all displacements and color details to Arnold?
Post by: vinyvince on July 21, 2018, 07:18:03 PM
If i creat my landscape in World Creator 2 or other software, import it to Terragen and do fine details displacement adjustement like strata and so do better structure the rock, is there a good reliable option after to export my terrain and final displacement to Arnold and Maya? 

Your Experience with Terragen and helps will be greatly appreciated, thanks!


--
________________________________________________________________

Vincent Thomas   (VFX and Art since 1998)
Senior Concept and Env DigiMatte Artist all in one  with a deep background in Lighting Lookdev  And Creative Supervisor VFX supervisor
 http://fr.linkedin.com/in/vincentthomas
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: WASasquatch on July 21, 2018, 11:55:28 PM
I believe geometry exporting via the micro exporter is reserved for professional copies of the software. You can also save heightfields, but doesn't come with some of the bells and whistles most need.

https://planetside.co.uk/wiki/index.php?title=Micro_Exporter
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: Oshyan on July 22, 2018, 01:16:37 AM
The Micro Exporter is good for geometry+UV but does not export color/texture at this time.

Heightfield export works for planar-only data (i.e. no overhangs).

Here is a vector displacement export tutorial:


- Oshyan
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on July 23, 2018, 10:49:28 AM
Thanks for referencing my tutorial Oyshan!

vinyvince:

Quote
is there a good reliable option after to export my terrain and final displacement to Arnold and Maya? 

Yes there is  :-) - check out my tutorial for my Maya Mel script:

&t=2s

Should be exactly what you need, though as WASasquatch mentions you will need the Professional version for this to work.

Quote
f I create my landscape in World Creator 2 or other software, import it to Terragen and do fine details displacement adjustment like strata and so do better structure the rock,

That's my workflow, though I've been using World Machine to generate my heightfields.

The workflow using this script can export terrain from Terragen either using the Micro exporter or as vector displacement maps. Terrain can be easily tiled to up to 64 tiles to make the management of large data sets manageable.

Color textures can be tiled and exported in the same way, and the script also manages the textures for Arnold in Maya.

I do have to say though, that while it's possible to get all this data into Maya, it's much better handled in Terragen. Terragen uses micro-polygon rendering and render-time displacement to be able to create vast detailed terrain with minimal memory overhead. To get this data into Maya requires much more memory - some of my experiments pushed 50gb loading multiple tiles of micro poly terrain.

It is possible though, have a go and let me know what you think.

Link to script:

https://www.highend3d.com/maya/script/terraman-for-maya

p.s love your reel!

Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 03, 2018, 03:59:30 PM
Quick question. Can you tell me what causes the triangulation in the image below, and is there any way to avoid this?
[attachimg=1]
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: WASasquatch on October 03, 2018, 04:52:06 PM
Quick question. Can you tell me what causes the triangulation in the image below, and is there any way to avoid this?
(Attachment Link)

Looks like low MPD detail in combination with low geometric detail.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 03, 2018, 06:54:21 PM
Thanks for the quick response. The MPD is set at 1, so should be pretty decent? would setting smooth surface on the "Compute Terrain" node make any difference?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 03, 2018, 09:57:32 PM
Is this an Arnold render of a Micro exporter output from Terragen?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: WASasquatch on October 03, 2018, 10:03:10 PM
Thanks for the quick response. The MPD is set at 1, so should be pretty decent? would setting smooth surface on the "Compute Terrain" node make any difference?

Smoothing Surface may produce looks like this as it will smooth finer detail in the geometry, evening fine geometry out. Are you using a heightfield?

I am not familiar with the Micro Exporter as it's a Professional feature and I work off the freeware version, however, according to the Wiki the Micro Exporter can export in TGO, FBX, OBJ and LWO2, so I'd imagine Maya and Arnold can handle OBJ/FBX.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: Oshyan on October 04, 2018, 12:28:22 AM
My guess is you are using the new Path Tracing render method. This currently renders terrain with the MPD limited (actually multiplied) by the "ray detail multiplier" in the Subdiv Settings of the render node. In the default setup it is 0.25 or 1/4 of the actual main MPD setting. Set it to 1 and set MPD back to 0.5 or 0.65 or so and you'll get better results (with longer render time).

- Oshyan
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: Matt on October 04, 2018, 08:33:12 AM
I assume this is the Arnold render. If you export the geometry from an overhead view, the size of the triangles will be fairly uniform in world space, but of course when you view this in perspective you'll have less apparent detail in the foreground than in the background. One way to tackle this would be to do the micropoly export from a perspective render that is similar to the view you will render in Arnold. But that might be impractical. You can try increasing the micropoly detail when you export, but that could result in some enormous files. You probably need to smooth the surface in Maya (or whatever 3D program you're using Arnold in), but it's still going to lack detail in the foreground.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: Tangled-Universe on October 04, 2018, 12:05:13 PM
What if one disables "fully adaptive" in the subdiv settings, would that make the micropolygons more uniformly sized?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: Matt on October 05, 2018, 12:28:12 AM
That is not the problem. They're already uniform enough in world space.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: Oshyan on October 05, 2018, 03:02:00 AM
Oh, it appears I misunderstood - that's an Arnold render? Riiight. :D

- Oshyan
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: ajcgi on October 05, 2018, 09:16:30 AM
If you're exporting terrain that is mostly displaced in Y only, then just use the heightfield method.
Make it large, like 8k large or higher.
A method that doesn't use the built in heightfield nodes... [do try Terraman too btw]

If you get a Get Altitude node, plug that into the luminosity of a default shader and put that on your landscape, with your last displacement node going into Child, with diffuse, reflection etc all at zero, render it top down with an orthographic camera, then that results in a black and white altitude map which when loaded as a push map in your 3D software should result in correct altitudes, providing you make your grid the same width as the orthographic width of your camera. Don't have lights on and set the ortho camera to have exposure 1.

It might seem like legwork but it works perfectly every time and avoids all this terrain export, triangulated stuff which doesn't guarantee nice shading in your renderer. Then also you can project your colour texture the same way, rendered from the same camera. Things will line up nicely.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 05, 2018, 11:32:03 AM
This is all assuming  falcooat is trying to render a micro polygon export in Arnold.

If it is, then this is what is probably going on:

Looks like an import of an object into Arnold that has the normals turned off, hence the faceted appearance. This is probably compounded by the camera position in the render that is looking at very small section of the terrain and there is just not enough resolution in the mesh.

Turning on normals in the micro polygon exporter in Terragen can smooth the triangulation look / faceted appearance but the render can then look a bit "soft" . Also in some instances the file size can double with normals turned on (more on file sizes later) so I usually leave it off and smooth the mesh in Maya if I need to.

Main problem is, if you export your entire terrain as a mesh and then zoom into it,. you're going to see those individual polygons. You then come to appreciate how Terragen handles it's geometry - always rendering a proceedural version no matter how close you are to the terrain.

Ajcgi has a good approach,  and it's something I've tried during my vector displacement experiments, but there is a limit to how much detail you can get.

I tried rendering 8k and even 16k displacement maps from Terragen and using them on a simple plane in Maya/Arnold, but in order to realise the detail in the map in the render a very high subdivision level is needed. This can take quite a while to calculate and uses lots of memory. I have 64gb in my machine and it ran out of memory before it could render a surface subdivision high enough for a 16k map!

There are ways around it, and my approach is to use tiling. Many smaller files (meshes or displacement maps) can be handled much better by Maya / Arnold than one gargantuan file :-)

This is how the texture UDIMs work, rather than create a huge 32k texture for a character, it is split into tens or even hundreds of smaller 4k tiles that are easier to deal with.

Matt mentioned you can render the micro from the camera perspective - for a still shot this will work fine, but if the camera moves then the still mesh capture will be redundant and a top down projection will be needed.

In my experiments, I've found the best approach is to export the entire terrain to the micro exporter, previs to find where the camera is going to be animated, then replace just the close up section with a much higher resolution mesh (or vector displacement map)

I'm going to publish a new version of my Terraman script soon. In it, you'll be able to use an orthographic camera in Maya to Boolean out a chunk of terrain close to a render camera, then export that same orthographic camera to Terragen to create a micro polygon mesh of the area in high resolution (this can even be tiled further for even more detail)

You will then be able to import the  higher resolution terrain (or terrain tiles),  and it will slot right into the hole created by the Boolean .The Maya scene will only have detail where it needs it and low poly detail in the background.

I'm working on an animation using this at the moment and will post when completed!

Anyway, I hope this was what falcooat was asking :-)

If not, hopefully someone else might find it useful!
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 15, 2018, 03:30:07 PM
Thank you so much for your lengthy reply! I have not been at the computer to respond this past week, but I appreciate everyone who had suggestions!

I must admit, I am new to Terragen, so it is taking me a bit of time to digest all the recommendations. @digitalguru, you are correct in your assumptions. I was working through your tutorial for TerraMan, and the image I posted was the result rendered within Maya/Arnold. My ultimate goal is to render an original terrain that will serve as the base for an animation. It sounds like your latest tutorial is exactly what I am looking for, so I will eagerly await for your guidance on this.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 15, 2018, 04:42:17 PM
Did you try to tile the terrain in that render?

Quote
My ultimate goal is to render an original terrain that will serve as the base for an animation.

If you let me know more on what you want to do, will be glad to help.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 15, 2018, 04:56:09 PM
Yes, the terrain in the image is following the steps exactly as you laid them out in your tutorial. Right now my problem is calculating my terrain coordinates properly. My terrain is smaller (and much less detailed) than the one in the tutorial and am having some trouble getting the correct result. Again, this is probably more an issue of me being new to Terragen, as well as a different issue at this point from the triangulation. Below is rendered in Maya/Arnold and clearly has something set incorrectly in the coordinate setup.

Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 15, 2018, 07:37:16 PM
How many tiles has the terrain been split into?

p.s - send me the Terragen file if you can - I'll have a look
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 15, 2018, 08:42:34 PM
Attached is the result I get (currently with my terrain, but also had a similar result with your tutorial terrain.) I am also attaching the Terragen file, but little is changed from your tutorial file, with the exception of using a tif height-map instead of a .ter file for height data.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 15, 2018, 08:51:09 PM
can you upload the tiff for the heightfield?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 15, 2018, 09:23:17 PM
if you go to the link you can download the tif I am using...

https://drive.google.com/drive/folders/15YqHG2TM6zIHcAm7Bhp3wfKQTBAQ5CCf?usp=sharing
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 15, 2018, 09:26:44 PM
thanks - having a look now...
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 15, 2018, 09:40:07 PM
quick question - are you using the Professional version of Terragen?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 15, 2018, 09:48:08 PM
Yes I have the professional edition, via an educational application. It looks like I need to up the polycount on the obj?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 15, 2018, 10:53:03 PM
Quote
Yes I have the professional edition

That's great, really helps working with heightfields (mainly for exporting high-quality displacements as 32bit exr)

Your heightfield is an 8 bit tiff - that's not optimum to use as a heightfield as it only has 256 levels of grey and your map is quite smooth - you'll need a 32bit map to capture those smooth gradients

The tiff is quite noisy too, and it's just random noise that's not going to look good on the terrain:
[attachimg=1]
You can fix that a bit by adding a heightfield smooth filter:
[attachimg=2]

But you can still see some square artifacts on the ridges of the (river/canyon ? sides) - if you painted this map in something like Photoshop - try creating a file in 32 bit and painting 32bit from scratch - you can also then paint at a smaller resolution and get the smooth broad strokes of the terrain and add detail on top with Terragens fractals.

In the Terragen scene you uploaded, there is one Bake Micro node to export the terrain to obj. This is good as a start as it enables you to get a mesh into Maya and start layout and adding animation etc. I chose 1024 x 1024 as an output size as it produces a mesh that is a workable size in megabytes - any larger and the mesh gets a bit unwieldy to work with.

So when a higher resolution mesh is needed in Maya, that's when tiling comes in - the same area can be split into a number of tiles and rendered out in 1024 x 1024 chunks and loaded into Maya.

It's getting late, so I'll have a proper look tomorrow and make a quick file to illustrate.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 16, 2018, 03:39:04 PM
Wow, you are awesome! Thank you for taking so much time to look at this. Admittedly, the terrain map I have is less than desirable, Unfortunately the real-world map is even worse. (The Great Plains aren't nearly as interesting as Mount Everest. ;) ) I will work on making a better heightmap and  see how that goes. Would I connect a smoothing filter to the heightfield shader in Terragen?

Truly, thank you for your help, I have already learned so much!
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 16, 2018, 10:52:05 PM
Quote
Would I connect a smoothing filter to the heightfield shader in Terragen?

Yes - connect a Heightfield smooth node after the heightfield load - this works well in the context of your heightfield, removing the hi frequency noise, but if the heightfield was more detailed this would smooth the detail along with the noise - so it's important to get a good noise free map to start with (32bit map as a source)

I loaded your heightfield into Terragen and added some fractal detail and some fake stones in the foreground:
[attachimg=1]

Then exported the whole terrain to an .obj via the Micro exporter and loaded into Maya via Terraman:
[attachimg=2]
The whole terrain looks ok from a distance, but very lo-rez at this close angle.
WIth Terraman (an update coming soon) I used an orthographic projection camera to boolean out a section of the terrain close to the camera. Then exported the same camera to Terragen and used that to Micro Export the smaller section of terrain to import back into Maya:
[attachimg=3]
The terrain looks more detailed, and the low rez terrain in the background works from this camera angle also.
The foreground render in Maya is still not as detailed as the Terragen render, but this small patch of terrain can be further subdivided into more hi-rez tiles - it's possible to get pixel to pixel accuracy between the Terragen and Arnold renders.

I'll upload the scene files tomorrow - my upload speed is pretty slow :-)
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 16, 2018, 11:01:47 PM
screengrab of the medium rez tile in the boolean section of the lo-rez terrain:
[attachimg=1]
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 17, 2018, 09:21:21 PM
Hi Falcooat,

Just uploaded a Maya and Terragen scene based on your heightmap. Terragen is a gathered project so if you work out of the downloaded directory all the paths should be relative. If it doesn't work for any reason, just re-point the paths (for the heightfield for example) to the correct directory on your disk.

https://www.dropbox.com/s/ocjaiaig80ktw7q/project_gather.zip?dl=0

There is also an update to Terraman which includes the Boolean function - just replace all the files in the installation.

I made a micro export of your entire terrain and loaded into Maya with Terraman.

Then created a Projection camera - close_Projection_cam, selected the imported terrain and the projection camera and used Boolean Mesh from Projection camera to cut a hole out of the terrain that was close to the render camera.

I could then export that projection camera to Terragen and used it to render that smaller patch of terrain using the Bake Micro Close render node.

Re-import that .obj back into Maya to give a much higher-rez terrain that fills the hole made by the boolean.

Didn't include this mesh to keep the upload file size down, but try it on your machine.

To get even higher resolution I then created a 3 x 3 tile of the close patch of terrain - again you can try baking these 9 tiles out using the Bake Micro Tiled x 9 render node -

Just use Render Sequence on the render node to bake the sequence of obj files.

Hope that helps, any questions, give me a shout.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on October 19, 2018, 04:08:09 PM
Great, thank you! I will take a look at all of this and see what I can accomplish. I really do appreciate all your help on this. I will report back once I play around a bit. :-)
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on October 19, 2018, 08:25:44 PM
Quote
I will report back once I play around a bit. :-)

Please do, especially if it relates to the Terraman script. It can get a bit complex, especially if you're new to Terragen :-)
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on November 08, 2018, 04:18:21 PM
Quick question. I am trying to batch clean my newly made obj files in TerraMan and I keep getting a failed to load error (specified module could not be found.) This is the first time running into this specific problem and I am wondering if it is related to your update?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on November 08, 2018, 08:18:49 PM
sorry to hear that,  I'll have a look:

- what version of Maya?
- where is that error message? Maya or the cmd window for Meshlab that pops up when running Process Mesh?
- copy paste the full error message and post it here

Quote
This is the first time running into this specific problem and I am wondering if it is related to your update?

does that mean the same operation is ok with the previous version of the script?

p.s
Quote
specified module could not be found

that sounds like a Maya error message - could be Maya not finding the objExport.mll?
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on November 13, 2018, 03:50:05 PM
okay, I reinstalled the previous version of Terraman and it is working, so I think the problem is your new version.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on November 13, 2018, 07:04:09 PM
ok, if you can, let me know what you were trying to do with the new version so I can narrow it down a bit
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: falcooat on November 13, 2018, 07:40:59 PM
I was at the batch step in TerraMan to clean up (terrain_clean.mlx) the obj files that were created in Terragen and the create the Udims. Once it is finished processing I will reload the new version again and try to run it.
Title: Re: Export Terragen terrains and all displacements and color details to Arnold?
Post by: digitalguru on November 13, 2018, 08:51:46 PM
thanks

sound like you were doing something like this?
[attachimg=1]

if you run the "terrain_clean" process on its own it will invoke Meshlabserver outside of Maya.

Selecting "Create Udim/UV" will then take the list of .obj files after the Meshlab filter has finished, loads them into Maya to apply the UVs/Udims and write them out to disk again.

I'm guessing that since you got the "specified module could not be found" that would be a Maya error and suggests it might be that Maya hasn't got the objExport plugin loaded to export the UV'd mesh.

When Terraman is loaded it automatically checks if the obj is loaded and loads it if it isn't - so just restarting the script could fix (if that's the problem)

I'll have a proper look at the weekend.