Terragen, Maya, Arnold, Terrain tiling, vector displacement and color textures

Started by digitalguru, May 13, 2018, 03:27:54 PM

Previous topic - Next topic

digitalguru

[attachimg=1]

I've just added V3 of my Terraman Mel Script for Maya to Highend 3d:
https://www.highend3d.com/maya/script/terraman-for-maya

Terraman now features terrain tiling. Terrains can be tiled on export from Terragen, and up to 64 tiles can seamlessly combine in Maya.

Tiled terrains can be loaded as regular .obj meshes, Arnold Stand-Ins or Vector Displacement maps.

Terraman automates many of the steps needed to tile terrain.

Object and camera animations can also be exported from Maya to Terragen.

There's a full tutorial on Youtube:
https://www.youtube.com/watch?v=ALbWYn_YHMI&feature=youtu.be

And supporting scene files:
https://www.dropbox.com/s/qpd2uio6uuy8hjf/terraMan_v3_tutorials.zip?dl=0


Matt

Great work, Graham.

What changes/additions would you suggest for Terragen to improve the process? Feel free to start a different thread if you don't want to divert this one, of course.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

digitalguru

Thanks Matt!

It is a bit of a process at the moment, and requires a bit of back and forth between Maya and Terragen.

it would be ideal if the tiling could happen in Terragen and all the required micro-exports, color textures, VDM maps could be a one way export to Maya.

Using a camera to animate the tiling positions was the only way to transfer that tiling data over to Terragen - don't know how that could work procedurally in Terragen? If there were some kind of scripting in Terragen to set up an animated camera that would be the homegrown way I'd approach it - but I guess integrating scripting is not a trivial task. How about a bounding box to set up the area to tile with a control for the number of tiles that could pipe into a render node? Anyhow, it was good to find out the micro exporter could output a sequence of meshes!

Just installed the new Frontier update and it's great to have render projected UVs now, though each tile would have to increment by a UDIM tile so color and VDM textures could be assigned outside of Terragen.

Mesh processing takes a bit of time to clean up the .objs- I see where you can get duplicate faces on the borders of the render buckets. Just wondering what produces the extra verts? They can make the mesh very heavy - is that just a by-product of the micropolygon export? Is there something deep in the subdiv settings that could help?

If it's just the nature of the beast though - maype a simple delete duplicate verts/faces filter as part of the Micro Exporter?

Matt

Quote from: digitalguru on May 13, 2018, 05:52:59 PM
Just installed the new Frontier update and it's great to have render projected UVs now, though each tile would have to increment by a UDIM tile so color and VDM textures could be assigned outside of Terragen.

How big of a help would it be if you could assign a separate camera in the micro exporter to define the UVs? If the UVs were calculated using the first tile's camera for all tiles, each tile would have a different UV space (which I think is what you want).

Matt
Just because milk is white doesn't mean that clouds are made of milk.

Matt

Quote from: digitalguru on May 13, 2018, 05:52:59 PM
Mesh processing takes a bit of time to clean up the .objs- I see where you can get duplicate faces on the borders of the render buckets. Just wondering what produces the extra verts? They can make the mesh very heavy - is that just a by-product of the micropolygon export? Is there something deep in the subdiv settings that could help?

If it's just the nature of the beast though - maype a simple delete duplicate verts/faces filter as part of the Micro Exporter?

Each bucket renders independently, so some duplication naturally occurs. I usually recommend reducing threads to 1 and and increasing the bucket size to cover the whole image, but of course that slows down your render. It would be great to have the Micro Exporter find and merge these duplicates, and we'll get to that some day.

By "extra verts" do you mean that each triangle has its own 3 vertices instead of sharing with its neighbours? Yeah that's a by-product of how the renderer sends data to the micro exporter, but that is also something I'd like to auto-merge some day.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

digitalguru

QuoteHow big of a help would it be if you could assign a separate camera in the micro exporter to define the UVs?

I'm not quite sure that would work, each tile would have to move a unit of 1.0 in UV space, so the first tiles UVs would exist in 0 to 1 UV (in U and V) then the next tile would have its UVs in U (1.0-2.0)  and V stays in 0 to 1.0) Standard UDIMs would populate a row in the U direction for 10 tiles and then move up in V to V1.0-2.0 for another 10  (though I've deviated from that slightly and based my row length on the number of tiles used). is that what you meant?

QuoteBy "extra verts" do you mean that each triangle has its own 3 vertices instead of sharing with its neighbours?

Yes, I think so, and I guess the number of neighbouring faces determines how many duplicates you can get at any vertex position - I've seen as many as twelve in a mesh.

In my tests those extra verts have a cost, especially if they have a UV coordinate baked in. The mesh size can increase dramatically - example - a micro exported mesh with normals and uvs turned on is about 1.3 gb as opposed to 0.42 gb with normals and uvs turned off on export - and reduced to a further 0.16 gb if duplicate vertices and faces cleaned)  - it takes much longer to write out the .obj to disk at the end of a micro export render for instance, and I was always mindful of not creating potentially unwieldy geometry in the process.

It's a trade off, and exporting the mesh with all available threads turned on and uvs off is a lot quicker to render, even factoring in cleaning the mesh and adding UVs at a later stage.

ajcgi

Great to see this getting some updates. I used it a couple of years ago for something. It was pretty handy.

digitalguru


Matt

Quote from: digitalguru on May 14, 2018, 06:48:19 AM
Quote from: MattHow big of a help would it be if you could assign a separate camera in the micro exporter to define the UVs?

I'm not quite sure that would work, each tile would have to move a unit of 1.0 in UV space, so the first tiles UVs would exist in 0 to 1 UV (in U and V) then the next tile would have its UVs in U (1.0-2.0)  and V stays in 0 to 1.0) Standard UDIMs would populate a row in the U direction for 10 tiles and then move up in V to V1.0-2.0 for another 10  (though I've deviated from that slightly and based my row length on the number of tiles used). is that what you meant?

Yes, the generated UVs would look like this if the UV-definition camera was locked to the first tile. When a different part of the terrain is rendered it would have UVs outside of the 0..1 range. It's not very intuitive, but it would get the job done. Possibly a simpler version of this would be a checkbox to tell it to generate the UVs relative to a particular frame, which would default to 1001, and then you wouldn't have to create a separate camera.

Even though it's not very intuitive (until you understand why it works), it would be useful to have this option for other reasons too. I might want to output multiple chunks of terrain and have a UV space that covers all of them when they are eventually brought together in another app. That UV space would need to be defined with a separate camera or some other bounding box. Defining UVs with a camera is a simple way get this working with minimal UI changes, and is also tremendously flexible.

But, of course, a dedicated tile output feature in Terragen would be easier to understand and use.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

Matt

Quote from: digitalguru on May 14, 2018, 06:48:19 AM
Yes, I think so, and I guess the number of neighbouring faces determines how many duplicates you can get at any vertex position - I've seen as many as twelve in a mesh.

My guess is that it could go up to 12 near the corners of buckets (where 4 buckets overlap), unfortunately. But if that happens in places where buckets don't overlap then something is definitely wrong.

Matt
Just because milk is white doesn't mean that clouds are made of milk.

Matt

Regarding the UVs again, if TG had the necessary features to allow you to write out the UVs with the correct values that you need for UDIMs (e.g. the bottom left tile is 0,0 to 1,1 and the tile to the right of it is 1,0 to 2,1 etc.), combined with your animating camera trick to render tiles, would this remove some steps from your process or be valuable in some meaningful way? If so, I'll add that option. I guess what I'm not sure about is whether you would still need to process the UVs or do some other step so that I'm not really helping you at all with this option.

Another question.. I wasn't sure why you needed a step to rename the textures. Can they not be formatted how you want using TG's output filename?

Matt
Just because milk is white doesn't mean that clouds are made of milk.

digitalguru

QuoteMy guess is that it could go up to 12 near the corners of buckets (where 4 buckets overlap), unfortunately. But if that happens in places where buckets don't overlap then something is definitely wrong.
That makes complete sense now - the vertex joins to 8 faces, each with their own 3 verts - I probably just selected a bucket border vertex when I was testing.

QuoteEven though it's not very intuitive (until you understand why it works)

It's a bit hard to visualize how that would work, but sounds like a good idea, would grasp it better seeing it in practice:-)[/b]

digitalguru

Apologies for the partially duplicated post - tried all day to post this and the forum website crashed on me...

QuoteEven though it's not very intuitive (until you understand why it works)
It's a bit hard to visualize how that would work, but sounds like a good idea, would grasp it better seeing it in practice:-)

Quotewould this remove some steps from your process or be valuable in some meaningful way?
Yes it would be valuable - at present, the meshes need to be loaded in Maya, have uvs assigned and written back out to disk - with high-density meshes that takes some time, and of course, since this stage requires Maya, it would enable users who don't have Maya to utilize tiled UVs. (so long as they can make a tiling camera)

QuoteAnother question. I wasn't sure why you needed a step to rename the textures. Can they not be formatted how you want using TG's output filename?
Yes, that would work, I just decided to start the file sequence from frame 0001 rather than 1001, seemed less prone to errors.

Also, I thought it might be easier to visualize if the UDIM number reflected the position of the tile on a grid, so if you had a 3 x 3 grid of tiles the UDIMs would be 1001,1002,1003 for the first row, then move up to next V udim for the next row, 1011,1012,1013 etc - easily changed though if it's not intuitive.



digitalguru

Just uploaded an update: v3.3.10

if using non-standard world scales (other than 1:100) and terrain multipliers, projection planes will export at a correct scale for Terragen and sculpting in programs such as Mudbox.

See there has been quite a few downloads since this release - would be great to get some feedback - is the workflow easy to follow? any suggestions? any nice renders?