export and render imported object (to speed up rendering)

[Dune]

A new thread, which is hopefully easier to find for new users. I hope I covered it all and it's clear (and useful).

- I used the render camera to make an object from a surface (in this case a sphere with 2 water shaders to make a 'sea object').
- A new micro exporter should be set with no normals or UV's, or the outcome will be a disaster (setup for obj 2, tgout 002).
- All other nodes can be disabled; sun, enviro light, atmosphere, underwater textures, terrain, etc, so it's just the uncolored, but displaced sphere.
- MPD should be experimented with, as the obj can easily become huge. I set it at 0.25 with the render size intact. Hit render, and an obj will be made from the (in my case) cropped part. It only took 2mins in my case, resulting in a 31MB object.
- Import the obj, and attach the colors you need, but not the displacements to the object. Better also use a final transform shader set to final position to get the colors in world space. If it's water, don't forget to uncheck 'cast shadows' in the object rendering tab.
- Enable all disabled nodes for terrain, lights, atmo, and such.

Rendering will be much faster, especially with PT.

[Kadri]

Good summary Ulco.

If you want, you could add that some objects will need "Force displacement" selected.

The holes in the object are interesting. I have not got so big problematic parts in my objects so far i remember.
The objects polygons are always not what you could say ideal, but this is quite different from what i got to this day.

If your example scene works in my old version the same way, i am curious what the reason is for those bad parts.

[Kadri]

Ok! Nothing strange going on. Less detail, small object, more holes

[Dune]

So with higher detail a better object may be obtained, perhaps. I have to test whether the area with the holes renders fine.

[Kadri]

So with higher detail a better object may be obtained, perhaps. I have to test whether the area with the holes renders fine.

Yes. You get much better objects that are closer to the native Terragen look as you can see from the Lightwave obj look below.

This test  (Terragen v4.125) is with a 670 Mb obj file rendered in HD.

[Kadri]

Ulco too tired now to test it. Haven't slept tonight. But that strange water crop render...
Just guessing but as it looks like you exported the water with normals checked, maybe the normals are all looking at the same angle.
So Terragen is calculating the object kinda like glass without any displacements.

It kinda looks interesting too.

[Dune]

Good to know more detail gives better objects. So the point is to find a sweet spot, not too big or it won't load, and not too small.

Yes, it looks interesting, and the only difference is in the way I exported (with normals and UV's checked). I guess it's the UV's that mess it up, not the normals. But you may also be right. I might have another go, but I'm getting tired about it too.

[sboerner]

Thanks, Ulco, for summarizing all of this. I've been following the other thread with great interest. Thanks to Kadri and Was for your contributions as well.

One thing I haven't seen mentioned. When I used exported meshes for a project a couple years ago I found that Terragen exports many, many duplicate vertices. These can easily be eliminated using other software. I've generally used MeshLab (which is designed to clean up scanned meshes). It has a function to remove duplicates -- literally vertices at the same locations.

When I exported Ulco's watersphere (using his settings), TG gave me an obj file 237,533 KB in size consisting of 2,286,504 vertices. MeshLab removed 2,038,310 of these (no that is not a typo), reducing the total to 284,194 and exporting a new obj file 66,698 KB in size but identical to the original as far as I can tell. (View attached of the mesh in Blender.)

Blender's vertex merge (using the default distance of 0.0001 m) removed a few more, which makes sense because of the additional tolerance, yielding a result of 253,917 vertices. Blender's obj export function seems to be more efficient than MeshLab's, and saved out a new obj 35,700 KB in size.

MeshLab has a merge-by-distance function, too, but it's *very* slow and finicky so I didn't test it. Not sure if PoseRay has a similar function. Couldn't find it.

Anyway, this means that those huge obj exports we're getting from Terragen can be easily crunched down into files that are much more manageable.

Finally a question and a couple thoughts on this.

- Ulco, would you recommend this as a new workflow for water and other reflective surfaces?

- This might be less practical for large land surfaces because the resulting object isn't a planet, which means distribution and surface shaders should be set to "Use Y," correct? So if the original surface was generated from a large area, where the earth's curvature is noticeable, the resulting Y-keyed distributions on the object surface would be out of whack. (There may also be issues placing populations?)

- But for smaller foreground areas, or nonanimated water surfaces, this could be a lifesaver. When I disabled the planet (along with the atmosphere) in Ulco's scene and disabled all of the reflection/refraction settings in the water shader (leaving only displacements), TG exported the obj mesh very quickly, in just a few seconds. (I guess I could have disabled the sunlight as well.) Seems like this would be a very practical workflow for still images.

[Kadri]

...

- This might be less practical for large land surfaces because the resulting object isn't a planet, which means distribution and surface shaders should be set to "Use Y," correct? So if the original surface was generated from a large area, where the earth's curvature is noticeable, the resulting Y-keyed distributions on the object surface would be out of whack. (There may also be issues placing populations?)
...

I have that problem now actually in my animation. It is a big landscape.
The clouds are displaced over mountains. It changes from one side to the other side.
I wasn't aware at first. If you are aware of that you can act accordingly.

 As you can populate objects in any direction you want, that shouldn't be a problem in theory.
I wish that Matt fixed that rotated object bug for populations.
But if you don't rotate your main object (where the population will be sitting on) you should be fine.
As a workaround you could change the position, rotation the way you want in another software and load that obj.

[Kadri]

...
Anyway, this means that those huge obj exports we're getting from Terragen can be easily crunched down into files that are much more manageable.
...

Be careful. As you are a very good modeler you know this,
but when you automatically merge duplicated-bad vertices and polygons
you can get holes, alter the look more then you intended.
I begun to use direct Terragen exported objects because of this and because of editing time.
When you want to merge vertices, polygons in Lightwave a 3 Gb object can take a really awful long time.
Interestingly all those bad-duplicate vertices, polygons weren't much a problem from a render point of view for Terragen.
But can't say for sure that this will be so in all scenes.
I doubt that you will accept such garbage objects but just saying
In the past i decimated the objects to get smaller files but that produced here and there bad polygons too.
This might depend on the software you use too of course.
If the Terragen export was clean this would work very nicely
A Micro exporter optimised for better file size, direct decimating while exporting with more options would be great.
If it is relatively small like 1 Gb i clean the file. When it gets bigger i just use that as it is.

I don't care much but you can try rendering-exporting with only 1 thread (maximum thread setting not Windows).
From old posts here around it looks like this exports a cleaner file.
Takes longer of course.
I think there were some things related to the "Subdiv settings" part...
Adaptive etc...but don't remember which one did what if at all.
Have a look at those and see if it changes the export.

[WAS]

When I weld, and remove orphan vertices the object doesn't seem to change look. I havent tried directly as a render/import though. But indeed if disp is too fine you will get holes even with MPD2 i found with rocks. And unfortunately, even with welding, if there are holes you cant really subdivide as it will make those holes bigger. But if you need to subdivide you gotta weld TG objects as all the faces will shrink giving a weird patchwork mesh of holes.

[Kadri]

With small files manual editing can be done.
But after a certain threshold it gets too hard. For me at least.

[WAS]

With small files manual editing can be done.
But after a certain threshold it gets too hard. For me at least.

Yeah like my geonosis stuff. Not sure where to from here right now. I think it may need to wait for better exporter. Cause if I could export the needed textures then a higher MPD may look fine just imported. Would need filesize issue fixed though too.

Also I think the water issue is the UV issue. Maybe dont use default from UVs and use procedural at all directions of normal. If its still there then must be normals.

[sboerner]

Playing around with a recent scene to see how this might work with a real project. Exporting, cleaning, importing are not insignificant tasks and can take some time. If done once that's OK. If you make any edits to the water surface (for example) then you have to do it again. Takes care and planning.

Exporting at the original MPD of my test file (which is high, 0.8  ) creates a very large obj file with corresponding waits for cleaning, etc. So I'm experimenting to see how low that can be reduced and maintain quality. This could be a benefit, which would allow you to use a low MPD for low-frequency surfaces like water, while keeping a high MPD for high-frequency ground details in the final rendering.

I'm finding that MeshLab indeed does the best job fixing the duplicate vertex issue. I understand your warnings, Kadri. Meshlab counts vertices and faces. As an example, one very large file was fixed and the vertex count was reduced by 64,161,475 while only 76 faces were lost. So I think it's a pretty safe step to take, but I'll keep an eye on it. (I'm not doing any optimizing; the goal is to maintain the face count.)

Will have a test file to post soon.

[WAS]

So I played with MeshLab real quick on a rock object. It was 1.31gb, after running it through MeshLab I got a file that was 561mb.

I tried to make the mesh have really tight folds, and pocked it with rough high octave and high noise variation holes to try and break the mesh.

MeshLab seemed to do a great job, and no visible holes, and geometry looks decent enough. Textures don't align on the object anymore but oh well.

Obj - 42s
Cube - 2:21s

Quick note: However, there was an issue. The MeshLab object was no longer viewable in Poseray. It would load, but take a very long time, way longer than raw TG file. Then, it would be a blank preview, but when you select a material, the whole viewport goes red. So I tried zooming out... into infinity... and couldn't get the object to ever show. Lol