https://docs.google.com/presentation/d/101COAUXgKo_ouWcgoZhraDmDV9urZZu1gnOtxX1vB3g/pub?slide=id.gdcbab5350_2_75
Yes, we were pleased to see this at SIGGRAPH this year. And Martin (Tangled-Universe) gets a little time in the spotlight. :D
We do intend to make these workflows easier in the future too. Terragen is a great asset generator and the demand is only growing.
- Oshyan
Oh man, I wish I had the audio to go with this!
QuoteMaybe it's worth exploring using vector displacement instead of height in the future
- slide 75
HA !! Who around here has been saying that?
If it works all of those problems he was talking about would be gone. And moving the terrain around would be easy and fully editable via sculpting.
One thing I want to know more about from the slides, is, where he talked about displacing an a "tessellated" object. He said he applied a TG height map to displace the object. I only tried displacing a plane. Some one please explain what he meant, so I can understand the difference between what he did and what I tried.
Quote from: TheBadger on September 03, 2015, 04:09:28 PM
One thing I want to know more about from the slides, is, where he talked about displacing an a "tessellated" object. He said he applied a TG height map to displace the object. I only tried displacing a plane. Some one please explain what he meant, so I can understand the difference between what he did and what I tried.
When you say you only displace a plane, do you mean in Terragen or some other app? Generally speaking, you can use height maps to displace whatever you want. On slide 69 Veselin shows some basic modeled cliff shapes and then on slide 70 he shows them with displacement applied. He's talking about displacing the geometry in Unity or 3DS Max (it's not clear which, but the end goal is the same, some high res geometry in Unity).
Matt
QuoteWhen you say you only displace a plane, do you mean in Terragen or some other app
I was displacing a mudbox plane with a height map I made in Maya, of a TG terrain that I exported as a object ::) lol The way I went about it was a little strange, but there were reasons for it at the time.
It worked. But there were problems. I thought I should try doing like in the link. But I think I will just run into the same problems or other problems that have to do with mudbox once I was back there. Yeah pretty sure as I think about it... Vector is the way to go, assuming it outputs a map from TG that does not contain stretching, and gives a very clean, even, displaced mesh. Then I am still sure that would be best.
Not to push you about it, Matt. But you posted about vector output from TG before. You only said that you believed it was possible for you. Can you talk any more about it?
I haven't started working working on it yet. But there are two ideas I would like to work on. 1) We could export a vector displacement map of an area, perhaps similar to the way we can already generate a heightfield from shaders and then save that heightfield to an image, but we should be able to do it with vector information (RGB) instead of height information (greyscale). 2) The micro exporter could export a lower resolution mesh along with a vector displacement map to account for the missing resolution (and optionally just a regular greyscale displacement map, since this is often sufficient for this kind of situation where you're just using displacement to add the finest details).
Matt
Please shoot if you have any questions.
I can write a short'ish how to on the work I did for Unity.
Matt,
Interesting ideas for export.
It would definitely be nice if we could somehow export VDM's from TG.
Import those to ZB/MB and export either as new VDM or an "additive" one.
I don't know which of the 2 options you propose is the fastest in terms of performance in TG?
Perhaps also to develop?
Remember the strange raster/grid 'secondary' mesh underneath the main mesh, due to render bucket tiles?
For exporting a seamless (no raster/grid) and gap free mesh I needed to render the micro exporter without "fully adaptive" and as a single thread.
Because of disabling "fully adaptive" there was much less detail compared to rendering with "fully adaptive" enabled.
So I needed to render it single threaded at detail 4 or 5 @ 1-1.5k resolution. That took usually around 12-20 hours.
Not desired.
So if the problems with micro exporter are not easy to fix and thus still require these adjustments in render settings etc., then performance wise the first option would be the easiest and fastest way?
I like it though that with option 2 you get more different types of output at once.
Glad to ear about possible future improvement. I know Matt was already talking about it, but I didn't want to push
to much pressure.
Hi Martin,
QuoteFor exporting a seamless (no raster/grid) and gap free mesh I needed to render the micro exporter without "fully adaptive" and as a single thread.
That's indeed a problem. I was hoping Meshlab could clean this overlapping geometry problem when using mutli-threading rendre in Terragen, but without success.
(There are so many functions in this software).
But I'm not really sure why you render the 'hires' version without mutli-threading ? Having a 'clean' mesh is only important for the ingame lowres mesh, so why not generate a low detail version from Terragen ... even with mutli-threading off it shouldn't take that long ?
To me, it would be great to have some sort of terrain exporter node, working a little bit like the heightfield exporter, but with the option to export the result as vector displacement, or as full geometry ... and if the geometry option can store color vertex, this would help a lot to bake textures too.
... (any hint about the famous 3D erosion plugin ? 8))
If you export with the micro exporter while rendering multi-threaded then the mesh will have artefacts at the bucket tile boundaries.
Rendering with 1 thread and setting bucket size to the size of your rendered frame + unchecking "auto reduction" will force the rendering of the entire frame as 1 bucket.
Hi Martin,
I understand that, but again the artefacts introduce by the multi threading are not really visible for the rendering, or for normal baking.
So my suggestion was :
1) generate the hires terrain mesh with multi-threaded on, to speed up the process (so you don't end up with 12-20 hours of processing)
2) generate a relative low version (detail 0.15 or less) with multi-threaded off, auto reduction off, etc.
All you have to do with this lowres version is to merge vertex (Meshlab), and maybe reduce the mesh a little bit further using a decimation tool (zbrush, Atangeo balancer, etc).
Then you bake normal from the hires to the low geometry as describe in the slide.
Well, if you say so? :)
I think I don't understand your question, but most of all this is anything but my cup of tea.
I did the exports for TG and performed tons of tests on how to export geometry Veselin deemed suitable for use.
As far as I know he attempted the procedure you proposed, but it didn't work with multi-threaded hi-res outputs.
The tile artefacts and "shadow mesh" messed everything up, according to him.
You seem to differ in opinion about that, but I can't discuss it as it's beyond my skills and knowledge as well as not the part of the job I did.
I'm interested to see an example of your workflow. Can you show?
Cheers,
Martin
All in all from a personal point of view I'm not satisfied by the result I managed to get, both in quality and quantity.
In the end when we finally figured out how to get it properly from TG into Unity, perhaps in flawed workflows as Paq suggests, leaving very little time left for the creative process.
In the end an experienced ZBrusher could have done similar things much faster and with more artistic control.
If you export nice looking terrain from TG then it looks horrendous from other angles.
Modelling terrain in TG which looks good at more than 1 or 2 angles is insanely difficult.
It's interesting because TG shines at rendering displaced surfaces, but modelling tools in TG are virtually non-existent if you don't count plugging/chaining fractals.
So all in all I blame mostly myself for not having developed a skillset which allows for shortcuts and smarter solutions.
However I think having access to better export tools will make a TG<->ZBrush/Mudbox > App X workflow much easier and artist/art-director friendly.
Having ZBrush/Mudbox like tools in TG would be a dream, allowing to model basic shapes with brushes and then add PF's, but I'm afraid that's not realistic to ask to develop.
Matt,
QuoteWe could export a vector displacement map of an area, perhaps similar to the way we can already generate a heightfield from shaders and then save that heightfield to an image, but we should be able to do it with vector information (RGB) instead of height information (greyscale).
^^This is the way to go!
Mudbox and Zbrush are not the same as you may know. I have Chris' TG to Z-brush workflow, it does not work in Mudbox. The things I mentioned in my other post were about trying to find a way to adapt Chris' workflow for Mud. As I said I found a way. But the problem was that the result was not exact. That was a big problem for me, because in my workflow I had spent a lot of time in TG building a scene that was exactly like my source location as possible. What I found was that I could not keep flat areas flat, and I could not make my heights the right height in mudbox using height maps... So when I got to Mudbox, I lost everything that TG made easy and accurate, things that are not easy or exact in any sculptor.
TG, in a few very important ways, allows a user to be very exact with a scenes in some important ways very quickly. For example, I was recreating a location from Iceland and I was able to make my scene the exact size (2000 meters long), I was able to make all the major heights within the scene as they were in the real world (no more than 100 meters high). I was also able to quickly get important visual detail and landmarks set up in TG (like how the entrance to a canyon looked, and how the height of the canyon was tallest in a certain location). Then I added a bunch of detail that helped to visually define scale in TG relative to the over all scale of the scene.
My hope was to get all of this to Mudbox as an
Exact representation. But you cannot do that using brushes on a height map... It is different in Z.
*About your original response to me: I was looking for a way to displace a mudbox object much like the example in the OP link, that did not require me to use brushes to apply displacement. This would leave me with the same problems as the example in the link (stretching) But would be better than what I got using the normal Mudbox way.
Your option 2 may have use for some. But I cannot see a more direct time saving way that also reduces workflow and pipeline steps. ZBrush and mudbox both output every kind of map anyone can need today. If we can get an EXACT representation of a TG scene directly from TG to Mudbox/zbrush, this removes steps from both the mudbox and Zbrush workflows. Option1, does all the most efficiently.
I have Chris' TG to Z workflow in his own words. I want to publish it here because I think it would really shed some light for people. But He gave it to me in private so I don't know if it would be good of me to post it. I should PM him and ask.
...
T-U,
QuotePlease shoot if you have any questions.
I can write a short'ish how to on the work I did for Unity.
Did you mean Matt or anyone (especially me! ;D) If you meant including me, than I would want this!
QuoteI don't know which of the 2 options you propose is the fastest in terms of performance in TG?
As for me, this is not a good question. It can take a long time for Mudbox to save out a high res Vector map. So I don't even care how long it takes from TG. It is better to have than not have! I just say this because your post made me think maybe people would think it would take to long in TG, and thus not be implemented... Yes, better to have than not have, at any price.
QuoteSo if the problems with micro exporter are not easy to fix and thus still require these adjustments in render settings etc., then performance wise the first option would be the easiest and fastest way?
YES! I agree with you 100%! Option 2 only sounds like a more complex and time consuming way of doing option one. When option 1 is already ideal for how people already work in Z and Mud.
...
Paq,
Quote(any hint about the famous 3D erosion plugin ? 8))
Look in announcements! 8)
...
QuoteIn the end an experienced ZBrusher could have done similar things much faster and with more artistic control.
Again, Chris's workflow. But you end up with the same problems of stretching. Though you get to where you end up much faster than what has been described here.
QuoteHaving ZBrush/Mudbox like tools in TG would be a dream, allowing to model basic shapes with brushes and then add PF's, but I'm afraid that's not realistic to ask to develop.
Even if it did have them, you would still want a fast easy accurate way to get to other industry standard soft. So you are still right.
Hi Martin, and TheBadger.
So yes basically it's the same workflow that describe in the case study.
1) I pick a Hero Mountain from Chris (really cool examples)
2) I did 2 renders/export : one high res (using multi threading), and a low res version without m.t. to prevent geometry overlapping. For both I still use 'Fully adaptive' because I want to keep vertical details as much as possible. Render time was under 15 min for a 5M mesh (hires), and 10 min for the low res.
[attach=1]
[attach=2]
3) So here's how the low-res looks like. Fully adaptive indeed introduce nasty topology (unconnected triangle) over the unconnected triangle soap. I use Meshlab to merge overlapping vertices, but it doesn't fix everything.
[attach=3]
4) So here comes Zbrush, and the nice dynamesh feature. It basically re-voxel the mesh, and create a nice evently topology. The cool think is that it also help to remove/merge overlapping displacement that can often happens with Terragen.
[attach=4]
5) After the dynamesh I apply a decimation to reduce polycount a little bit more, and create relaxed uv's
[attach=5]
6) Then I bake occlusion and normal using xnormal.
[attach=6]
[attach=7]
7) Here's a quick result, with both normal and occlusion on the low-res final mesh
[attachimg=8][attachimg=9][attachimg=10]
QuoteLook in announcements!
Thanks, don't know how I miss that :)
That is a very nice show of your workflow, paq. Thank you.
I don't have Z, but the workflow chris showed me is much less work, with what I think is the same end results. I will send him a PM after this post, I think that you will be interested to read his instructions. It would be good to hear from you about them in the greater context of this thread.
My short comment is just me repeating my self, Vector solves all of this. But in the mean time, it may be of use to continue as we are. Perhaps if you and another Z user see the workflow from chris than You can broaden the discussion, and perhaps another Mud user will be able to better adapt it for Mud than I was able to. Please check back on this topic in a day or two, assuming he answers my message. Would be good to hear from him in this thread anyway, since he works with TG and Z all of the time. Maybe you guys can solve problems a little more rather than just waiting for some added power in TG? Would be cool.
Quote from: Tangled-Universe on September 04, 2015, 07:43:10 AM
I don't know which of the 2 options you propose is the fastest in terms of performance in TG?
Perhaps also to develop?
In terms of development, a vector map equivalent of the heightfield export will be easier than the other approach. It really only requires a 3-channel version of a 1-channel workflow that already exists in TG, i.e. bake to a vector field instead of a heightfield, then right click on the heightfield/vectorfield node to save to an image.
In terms of render/generate performance it's probably quicker too, because there's less overhead dealing with micropolygons and the render engine, and maybe easier for the user to set up. But you would be limited to selecting a rectangular region just like you are with a heightfield.
At the moment heightfield generation is only single-threaded, so I will need to make it multi-threaded to make sure it's faster than a micropoly render export, but that won't be difficult.
Quote
Remember the strange raster/grid 'secondary' mesh underneath the main mesh, due to render bucket tiles?
For exporting a seamless (no raster/grid) and gap free mesh I needed to render the micro exporter without "fully adaptive" and as a single thread.
Because of disabling "fully adaptive" there was much less detail compared to rendering with "fully adaptive" enabled.
So I needed to render it single threaded at detail 4 or 5 @ 1-1.5k resolution. That took usually around 12-20 hours.
Not desired.
So if the problems with micro exporter are not easy to fix and thus still require these adjustments in render settings etc., then performance wise the first option would be the easiest and fastest way?
Well, the main advantage of "fully adaptive" is that it increases resolution where the surface stretches due to displacement. If we're talking about a fixed-resolution vector displacement map, it won't be able to put extra detail in these areas either. So there won't be any advantage here. You'll need to increase the overall resolution until you get enough detail in these stretched regions. This is where micropolygon export can theoretically be better at producing uniform detail (barring these issues/faults with the subdivision algorithm which you ran into).
Quote
I like it though that with option 2 you get more different types of output at once.
And it would aim to eliminate a few steps in the pipeline, if the desired product is low-res or mid-res geometry + displacement map. But whenever using any kind of fixed resolution 2d texture map (displacement map, colour map), stretching issues are going to occur depending on the UV mapping, and in this case the UV map will be based on the undisplaced plane/planet, just like with the "vector field" baking and export in option 1.
Matt
That is neat to see different results from the hero mountains. There are some interesting image examples and workflows going on here. I am still working on additional styles and some more wild looks for the hero mountains.
VOID!
chris_x422 ;) ;D I can't remember half the stuff I have already learned or done either :o
Alright, that name is familiar.
QuoteVOID!
No, paq did use your file for his example, and he is right, those are cool files you shared :) And he made cool mesh from them! This stuff is just too damn much fun :)... planetside people must always be happy people ;)
The VOID! was for the name confusion.
Hi,
chris_x422 said he had no problem with sharing!
Quote1.. Import you're terragen mesh into maya.
2. Create a poly plane.
3. From a top view, scale and position the plane so that it encompasses the whole area of the terragen mesh.
4. Scale the uv's of the poly plane down by a very small percentage
5. Export the poly plane.
These are the only operations we have to perform in maya, the flat poly plane will be later conformed to the shape of the terragen mesh inside of Zbrush.
You seem to be getting hung up on the idea of quads vs tris, there is no need. We are conforming the base poly plane (that is already quads) into the same shape as the terragen mesh.
I don't believe that Mudbox will however allow you to do this as it doesn't have the same functionality as Zbrush, but as it's been a while sine I've used it, I could be wrong.
6. import the poly plane we exported from maya into Zbrush
7. import the terragen mesh also into zbrush.
8. Append the terragen mesh into the same subtool pallete as the poly plane.
9. Subdivide the poly plane a couple of times before deleting the lower levels, then store a morph target.
10. divide the poly plane to around 1.5 -4 million polys.
11. click on project (below the subtools) and change the distance to 1.
12. Making sure that the poly plane is the selected mesh, and that the terragen mesh is the only other mesh that is visible, then press 'project all'.
That will force our poly plane from maya to take on the shape of our terragen mesh, as if it had been a sheet of cloth draped over the terragen mesh.
So we now have a clean mesh for sculpting that also has the shape we started with in terragen.
13. hide the terragen mesh in Zbrush.
14. sculpt
15. when you are ready to create a vector displacement map, take the geometry level back down to 1.
16. Go to the morph target panel and press switch (this will restore the plane to it's original flat shape)
17. Go to the UV map tab and set the size of the map you want to export (I usually export at 8k)
18. Open the Vector displacement tab, turn off vd Snormals, then click export and give your map a name.
19. click save, and Zbrush will start calculating a vector displacement map based upon the difference between it's flat state and higher subdivided sculpted state.
This map (when project back correctly in terragen) will be a near 1-1 match for what you exported from terragen earlier.
Thanks to Zbrush it's possible to take a mesh with any shape and project a flat poly plane onto it, forcing it to conform to it's shape.
And because Zbrush is capable of storing a meshes original shape (morph target) we are able to calculate the difference and store those values into a displacement map.
Hope that makes sense, but please, just try to replicate those steps a few times, and it should al become clear.
Chris
He also said when he has some time he will make some more video tuts on the broader subjects of this thread for all of us, which will be really AWESOME, as far as I am concerned!
Ok, so this goes back to that big thread on sculpting workflows and TG. But the steps were never set out like this. Some people may be able to add to it by now. Also, mud users should see the difference now with Z, a bit better.
Again if TG outputs a vector, then this all goes away. But in the meantime, maybe something can still be simplified in these workflows and somehow get better results?
Hi guys, been a while :o
Nice discussion going on here, and great to see more workflows being developed that extend Terragen's usage!
Congrats on the Siggraph demo Martin, very nice work! I know from experience that setting these techniques up takes time and patience, and a lot more testing.
Zbrush has always been critical in terms of extending TG for me, it's mesh projection and mesh matching brushes enable me to take out any mesh from TG and match it to low res meshes set up in maya. Meaning, no need for low res exports, and clean uv's from the get go. It also makes thing very easy in terms of cleaning up any stretching or mesh artefacts at the edges. We can also use the camera projection (by fbx export from TG) to bake textures or combined AOV's into the maya mesh, and further extend or clean up in Mari etc.
Been, constantly on the move for the past few year's, with little time for anything but deadlines. I'm making a big changes right now, and setting up my own studio, so hopefully I can get back to some of my own passions for a while. Like Michael said, once the dust settles, I'll try to put together a tutorial or two on the techniques I've been using.
Chris
Really looking forward to it! Thank you.
congratulations on your successes too 8)
Matt,
If you understand and envision uses for both options you are thinking on, than why not do both? Not really sure how I would need option2 but I am sure someone else here will inspire me. As for me, if you add vector (and rectangular noises ;D), than I will be in heaven and have no need to harass you anymore. Thats gotta be worth something? Or at least it will take a while for me to think of something.
Hello,
So here's an other way to create a 'usable' low res model from Terragen.
Again I really want to use the 'fully adaptive' subdivision option.
So to resume :
1) Render (micro export) the terrain with low details (0.2 - 0.5 ... depending of the resolution of course), Multi-threading off, Fully adaptive ON
2) Load the mesh in Atangeo Balancer
3) To fix the mesh, just use these 2 options : Tools>Merge vetices and Tools>Eleminate T-Junctions
4) You can then decimate the geometry a little bit further if needed.
[attachimg=1]
Works pretty well so far.
(I didn't manage to clean the terrain so well with Meshlab. The only close result was to use Cleaning and Repair > Snap mismatched borders ... but ... it didn't fixed everything).
Quote from: TheBadger on September 09, 2015, 03:10:36 AM
Matt,
If you understand and envision uses for both options you are thinking on, than why not do both? Not really sure how I would need option2 but I am sure someone else here will inspire me. As for me, if you add vector (and rectangular noises ;D), than I will be in heaven and have no need to harass you anymore. Thats gotta be worth something? Or at least it will take a while for me to think of something.
We will do everything we can make time to do, in time :)
Matt
Interesting software, paq. Thanks for your explanation. Didn't know of it but I had issues with meshlab too. Some testing to do...
I never heard of Atangeo Balancer but it sounds well worth looking into.
Nice study and information.
Hello there,
Just a title update about Atangeo Balancer.
http://www.atangeo.com/ (http://www.atangeo.com/)
This little baby manage to load a 281.000.000 vertices mesh (Terragen terrain), using 'only' 20 gigs of ram. (the .obj size was about 32 Gb).
Process time (Minutes)
Loading time : 9.04
Vertex merge : 4.26
T-junction : 2.02
After this cleaning you end up with a 93.000.000 triangles mesh.
Structure simplification pre-process : 18.00
% reduction takes then few seconds.
The key here is to hide the 3d viewport, otherwise the display complexity will freeze your computer
Sorry for the little shout out, but this is the kind of developer I like to support.
Balancer is a top quality piece of software, that works like advertised.
And the support is top notch ! (only get one problem since 2 or 3 years)
I wasn't able to use it on Terragen terrain until I understand that multi-threading introduce
overlapping mesh problem in the Micro exporter.
This looks helpful! I looked at the forum dis not see if this works on quad based models. Or if it will translate to quads to tris. Do you happen to know the answer?
Works on OS X too, so really interested in this. $144 for pro (not cheap) regular is 52.00. Not sure what the differences are, no chart on the price page.
Hi Badger,
No quad output, I don't remember any 'poly cruncher' that can output quad mesh.
I know Meshlab has an quad remesher, but I'm affraid it will not give any good result.
The only solultion I see is to use zbrush remeshing.
I'm using the 'cheap' version, more than enough for my needs.
There is a chart comparaison here :
http://www.atangeo.com/products (http://www.atangeo.com/products)
:-[
So here's a little practical test, using the micro exporter + balancer for vertex merge/T-junction fix and a little poly reduction.
The only limit is the memory needed by Balancer to process the poly reduction.
Original Terragen model was about 270.000.000 points, reduced into 160.000.000 triangles, and rendered in Clarisse.
That is very impressive! I just bought Atangeo Balancer, thanks for the information and examples.
Have been a bit quiet lately, also in this topic, but I'm reading along with all these messages :)
Very interesting stuff and I'm keen on seeing more!
In the end I'd be interested to see your workflow layed out step by step as well Paq, if possible :)
Quote from: paq on October 01, 2015, 10:27:37 PM
So here's a little practical test, using the micro exporter + balancer for vertex merge/T-junction fix and a little poly reduction.
The only limit is the memory needed by Balancer to process the poly reduction.
Original Terragen model was about 270.000.000 points, reduced into 160.000.000 triangles, and rendered in Clarisse.
Looks nice.Just curious how big was the file size of the object?
Hi there,
Hi Martin,
Well basically it will be a ripoff from Veselin, except that I replace the Meshlab part with Balancer.
But I will prepare a doc this weekend, there are few tricks here and there to speed up the process I haven't discuss yet. Also at this point I'm more interested to import/render hi quality mesh from Terragen in Clarisse, very different from a Terra to Unity workflow.
Hope someone wouldn't mind to correct the grammar/typo :-X
Hi Kadri,
Original Terra mesh was about 32gig ... but it require 'only' 24Gb of memory to process in Balancer.
(dont need any normal or texture coord info).
After the merge and small optimization, the mountain was about 4Gb.
From 32GB to 4GB...nice. Balancer looks very useful.
It would be much harder to do in another software probably.
Thanks for the info :)