Planetside Software Forums

General => Terragen Discussion => Topic started by: reck on August 24, 2012, 04:03:19 AM

Title: Populator v4
Post by: reck on August 24, 2012, 04:03:19 AM
A few suggestions for the populator. Not sure how feasible these are.

1.   Multi-threaded. A combination of a high number of dense populations results in very slow population calculations. It's not nice hitting render the first time you open such a project only to have to wait 10 or so minutes before it actually starts rendering. Also I've noticed that Terragen sometimes recalculates the populations when it doesn't need to resulting  in more waits each time it thinks it needs to recalculate. For "quick" renders it's possible for the the population calcs to take longer than the actual render. Please multi-thread the populator.

2.   Support for multiple objects. It would be nice when adding a population if you could select a number of different objects to be in the same population and then weight each one to tell Terragen how much of each object there should be in the population. I have a number of tree objects for instance that come with a variety of trees with different looks and heights. At the moment I have to create multiple populations in the same spot to load each object.

3.   Colour tone variety. When creating a population it would be great if the populator could  make slight adjustments to the colour of the leaves, for instance, to give a more realistic look to the trees. This make some of the leaves slightly brighter and some slightly darker to give a more varied look. I've seen attempts on the forum but I think an option like this should be added if possible.

Do you think these could see the light of day?
Title: Re: Populator v4
Post by: Upon Infinity on August 24, 2012, 04:16:56 AM
You might get away with number 1 and 2, but I think number 3 would be nice, but not probable unless they implement a new, separate kind of populator because the current one does not assume you are populating trees, nor should it. 
Title: Re: Populator v4
Post by: reck on August 24, 2012, 04:47:14 AM
Well number 3 would be an option, you probably wouldn't want it for manmade objects but you could use it for all types of plantlife, grass, bushes etc, trees was just an example.

I suppose the way it would have to work is for the user to tell the populator which part\shader in the object that this effect would be turned on for. So staying with the tree example you would go into the tree object where you'd see all the different parts, trunk, branches, leaves etc and then select the part that needs the colour variety. So in this case you'd select the leaves so that the bark on the trunk and branches would not get affected. Then you could select the strength of the effect to set how much colour variation there should be.
Title: Re: Populator v4
Post by: Hetzen on August 24, 2012, 07:34:16 AM
I would agree with all of your suggestions, 3/ can be achieved with some blue perlin functions btw.

I would also add another...

4/ Object spacing works in 3 dimensions rather than just two. ie, trees don't thin out on steep gradients. The distance is calculated with surface relief in mind.
Title: Re: Populator v4
Post by: reck on August 24, 2012, 08:03:59 AM
Hetzen that's a good number 4 :)

When I tested out adding colour variation from files on here it never worked too well for me. All I got was a much darker set of trees.

Do you have a clip file that demonstrates the perlin function you mention? It would help until populator v4 is released   ;D
Title: Re: Populator v4
Post by: Tangled-Universe on August 24, 2012, 10:23:26 AM
#5 set up multiple populations' coordinates, wireframe modes, sit on surface, planet, density shader, etc. etc. with a masternode or by drag-select and right-click or...
Basically anything which saves me copy pasting tons of similar settings from one population to the other...

This is actually #1 by my preference
Title: Re: Populator v4
Post by: dandelO on August 24, 2012, 12:00:10 PM
I usually do my populations, if I need lots in the same area, with the object maker in the main internal network view and reassigned to the populator there. Then you can duplicate the populator node without duplicating the internal object, waiting on an object reload etc. only to then delete the object again to replace with the new object you're populating.
A bit quicker than copying every pop' parameter but you still need to reassign terrain tab inputs, seed and density shaders etc.
Title: Re: Populator v4
Post by: yossam on August 24, 2012, 12:13:22 PM
Reck,

When using the color variation clip files, you usually have to increase the value of the color shader to 1 or above. This actually depends on the individual models, but a lot of the color values are set at .5.........
Title: Re: Populator v4
Post by: reck on August 24, 2012, 02:35:07 PM
Thanks Yossam i'll have another look. I thought I had increased the colour value to greater than 1 in an attempt to get the now black tress some colour again and didn't have much success.
Title: Re: Populator v4
Post by: Tangled-Universe on August 24, 2012, 02:40:11 PM
Quote from: dandelO on August 24, 2012, 12:00:10 PM
I usually do my populations, if I need lots in the same area, with the object maker in the main internal network view and reassigned to the populator there. Then you can duplicate the populator node without duplicating the internal object, waiting on an object reload etc. only to then delete the object again to replace with the new object you're populating.
A bit quicker than copying every pop' parameter but you still need to reassign terrain tab inputs, seed and density shaders etc.

Hmmm...if I duplicate the populator, but need to re-assign an object then that's equally slow/fast as making a fresh population and copying all the stuff from the other without having to re-assign any object. If it has any difference in speed then it is very minimal if you'd ask me.
Title: Re: Populator v4
Post by: reck on August 24, 2012, 05:32:33 PM
Quote from: reck on August 24, 2012, 04:03:19 AM


1.   Multi-threaded. A combination of a high number of dense populations results in very slow population calculations. It's not nice hitting render the first time you open such a project only to have to wait 10 or so minutes before it actually starts rendering.

It seems I was being a bit optimistic with the 10 minute time. In my current project i'm having to wait nearly half an hour each time Terragen decides to run through the population calculations.

The 7 populations are taking 27 minutes to calculate.
The actual test render took 22 minutes.

I'm using a 4ghz core i7-2600 with 8GB of ram.

With these times it really stifles the creation process.
Title: Re: Populator v4
Post by: Oshyan on August 24, 2012, 06:10:42 PM
Sounds like some truly massive populations! I'm actually surprised they fit in 8GB of RAM for how many instances there must be. Is there a lot of heavy masking/blending going on with very large/dense population areas, where it ends up to actually not have a huge number of instances, but still has a large area to calculate? If so you may be able to significantly speed it up by using smaller population areas in more specific places.

As to the broader issue, I think making population calculations parallel (as in calculating *separate* populations at the same time) is probably pretty easy to do, and it's something I'd like to see implemented soon. Actually parallelizing single population calculations to use multiple threads is more challenging (I don't know how much more). Population caching is another possible future option which would help with some aspects of this.

- Oshyan
Title: Re: Populator v4
Post by: TheBadger on August 24, 2012, 08:03:03 PM
Yeah this time for a populater sounds crazy to me too. I haven't run into any issues like your describing. Not that I need to see it to believe it. But you must be doing something massive in terms of population number?
I was going to say I did not care much about "#1", but now Im very curious!

I do like the sound of #2 & 3! The way you described it, I think they sound like great ideas. You put into words perfectly what I think when I'm trying to design a scene. Have no idea at all how hard it is for staff to do it though

QuoteBasically anything which saves me copy pasting tons of similar settings from one population to the other...
I also suport this statement^^
But I would simplify it to, "Basically anything that saves me time "."" Because anything that is repetitive I would like to see go faster. But hey, this is a wish list thread.

@Oshyan
Sounds like positive statements from you! Can you please give a tiny hint at the next update. Just something to hold an addict over untill the next fix/update.
Title: Re: Populator v4
Post by: RArcher on August 24, 2012, 11:38:05 PM
I've had scenes that take a couple of hours or more to populate.  Not really a matter of too many instances, more an issue of wanting small clumps of trees spread out over a very large area.  Think 20k x 20k area - 1 meter or less spacing blended by a powerfractal to get small clumped areas of trees.  If I recall correctly this scene: Mission Peak Preserve (http://www.archer-designs.com/adwp/wp-content/gallery/terragen-2/mission-peak-preserve-v7.jpg) took around 3 and a half hours to populate.
Title: Re: Populator v4
Post by: Oshyan on August 25, 2012, 12:08:46 AM
Yes, that's what I figured Ryan. I know it's not as convenient, but I wonder if you couldn't instead use a number of smaller, specifically placed populations, blended by the same input as the single large one, but simply confined in area to where the trees actually show up. Obviously not ideal, but for a scene you're working on for many hours/days, it could save quite some time and could be worth setting up.

- Oshyan
Title: Re: Populator v4
Post by: reck on August 25, 2012, 08:30:03 AM
lol here's me moaning about 30 minute population times when people are having to cope with hours. But for me even having to wait half an hour is too long.

Oshayn, I suppose more precise placement of smaller populations would cut down some time, but in Ryans case even if it knocked the time down by 2/3rd's your still talking over an hour for the calculations. I hope that we can get the populations calculating at the same time soon, it seems such a waste of the cpu resource most of us have.

@Oshayn, TheBadger - With regard to my project i'm not sure what's considered "massive" for a populations. In my case the 7 populations range from 3k x 3k area up to 5k x 5k area. Is this really such a large area when you're looking out over a landscape? Most of them are distributed with a fractal breakup and then a mask is applied, maybe this adds a lot to the time?
Title: Re: Populator v4
Post by: Oshyan on August 27, 2012, 01:44:41 PM
It's not the total area alone that determines population time, it's really the total number of instances which is area times density. In your case I suspect the populations are pretty dense, even though the area is not particularly large. The calculation time is dependent on the total number of *potential* instances, in other words the number of instances there would be with no masking whatsoever, so even if you're masking your dense populations down to just a few areas in the larger population square, it's still having to test for each potential instance to see whether it should be populated. So a dense population area with a million *potential* instances but only 1 *actual* instance will, I believe, take just as long to populate as all 1 million instances would. That's why I suggest making smaller areas in separate populations, especially for dense things like grass. The differences can be much, much larger than knocking the time down by 2/3s, it could be a factor of 10 or more, depending on how heavily masked your populations are.

- Oshyan
Title: Re: Populator v4
Post by: RArcher on August 27, 2012, 03:45:44 PM
Certainly a possible solution to do it that way, but I find I am not nearly random enough to make things look realistic if I place too many specific populations myself.  I like to be surprised sometimes by where things end up because they aren't always just where I would put them.

It would be nice if it was possible that the software would first take into account the density shader/masks and then only try to populate the areas that where the trees can possibly go.
Title: Re: Populator v4
Post by: Oshyan on August 27, 2012, 04:31:16 PM
Quote from: RArcher on August 27, 2012, 03:45:44 PM
It would be nice if it was possible that the software would first take into account the density shader/masks and then only try to populate the areas that where the trees can possibly go.

I believe that's exactly what it does. But it needs to test whether an instance exists in a particular part of the population at some point, so the entire population needs to be evaluated at that time. That's my understanding anyway.

- Oshyan
Title: Re: Populator v4
Post by: RArcher on August 27, 2012, 04:46:11 PM
Right, now what I would like it to do (though I am sure there are good reasons it doesn't) would be to first take a look at the masking to determine where the populations can and cannot go and then only do the calculations for those areas it can go instead of calculating every possible area in the population bounding box regardless of whether it has been masked out or not.
Title: Re: Populator v4
Post by: Oshyan on August 27, 2012, 04:51:22 PM
Without the ability to define arbitrary population shapes, I don't see  how that's possible. It's similar to the way clouds work. Localize Clouds speeds things up because the renderer knows the clouds will exist in a finite area, it doesn't need to check the entire sky for possible clouds to render. But you have to define the shape very specifically, and that can't be done with a procedural noise function because the function itself needs to be calculated to generate the bounds. I imagine there may be ways to do this more cleverly, at present the issue exists not just with populations, but with clouds and really anything that is blended by a procedural function - you have to calculate the function to determine the coverage, and that is apparently the expensive part.

- Oshyan
Title: Re: Populator v4
Post by: otakar on August 28, 2012, 01:21:00 PM
Quote from: Tangled-Universe on August 24, 2012, 10:23:26 AM
#5 set up multiple populations' coordinates, wireframe modes, sit on surface, planet, density shader, etc. etc. with a masternode or by drag-select and right-click or...
Basically anything which saves me copy pasting tons of similar settings from one population to the other...

Population groups is what I've suggested elsewhere. Set the parameters (as above) for the group and they would propagate to the children (individual populations). Would make it so much easier to modify a busy scene (and organize its elements).

Title: Re: Populator v4
Post by: reck on August 29, 2012, 07:26:57 AM
@Oshyan I guess lots of medium to small populations instead of a few big populations are the way to go then until population calculation speed improves. One of the downsides to this is that I already have a lot of populations to manage and splitting them into even more smaller groups mean that they get harder to manage and there's more copying\pasting between populations etc.

I think the answer to this is population groups as Otakar mentioned below. Then you can group related populations together and enter settings that affect the whole group.

Title: Re: Populator v4
Post by: paq on May 04, 2013, 10:19:15 PM
Hello,

The limitation of the populator to scatter only one object at the time is really concerning when you are comming from Modo,Vray multiscatter or Vue. Maybe I miss something, but how do you avoid overlaping instances if you scatter for example 5 or 6 trees model ? Having to sample the same terrain area 6 times is also a waste of computing ressource ... but again maybe I dont really understand the process.

(that having said, it doesnt seems to prevent some guys here to create amazing image :))
Title: Re: Populator v4
Post by: Dune on May 05, 2013, 03:01:56 AM
If you add a high contrast power fractal and with some merge shaders to make a set of non-overlapping masks, you can avoid overlap. There are other methods, which have been posted. The thing I am mostly concerned about is overlapping of instances within a population. I wish (I'll mention it as a feature request) there would be a minimum distance between instances to be set.
I occasionally have a flock of sheep where some Siamese twins occur  >:(
Title: Re: Populator v4
Post by: paq on May 06, 2013, 10:50:04 PM
Hi Dune, after 2 days I only understand the method you describe, seems to be a really smart idea, I ll give a try.
Title: Re: Populator v4
Post by: Tangled-Universe on May 08, 2013, 06:00:31 AM
Quote from: Dune on May 05, 2013, 03:01:56 AM
If you add a high contrast power fractal and with some merge shaders to make a set of non-overlapping masks, you can avoid overlap. There are other methods, which have been posted. The thing I am mostly concerned about is overlapping of instances within a population. I wish (I'll mention it as a feature request) there would be a minimum distance between instances to be set.
I occasionally have a flock of sheep where some Siamese twins occur  >:(

Then your object spacing is simply too small. If you have models with say 2m circumference and the spacing is 1m then yes...they will intersect.
I see where you're coming from though. If you increase object spacing you end up with not enough instances to your tast and also some regularity may occur.

In the end I think the populator needs an overhaul.
It desperately needs a kind of masternode to control multiple populations at once and having a kind of collission detection coming along with that would not only be ideal, but also almost kind of necessary to have it work properly.
Title: Re: Populator v4
Post by: Dune on May 08, 2013, 07:15:05 AM
QuoteThen your object spacing is simply too small
Even if it's not too small you may get overlapping instances.
Title: Re: Populator v4
Post by: Tangled-Universe on May 08, 2013, 07:29:14 AM
Quote from: Dune on May 08, 2013, 07:15:05 AM
QuoteThen your object spacing is simply too small
Even if it's not too small you may get overlapping instances.

That's because of the spacing variation, especially if you have it set to default 1 or even higher.
With spacing variation @ 1 you can have an instance being positioned at nearly the exact same position as its adjacent instance.
With spacing variation @ >1 you can have instances moved outside their "object space/box" into the "object space/box" from its adjacent instance.

So you can avoid this with playing with these settings, but like I said you may end up with pattern like positioning of instances, which is unwanted.
Title: Re: Populator v4
Post by: Dune on May 08, 2013, 07:37:04 AM
I know, thanks, Martin. Though I never actually used >1 variation, I might do some testing....

EDIT: Strange, with a pop of spheres the variation @0.1 has no influence, have to do some more testing.
Title: Re: Populator v4
Post by: paq on May 09, 2013, 01:08:59 AM
Hello guys,

I have done lots of reading lately, but I'm still not sure if it's possible to have color variation by instances.
All I see are noise color solutions over the whole populator group (which is cool too), but nothing that can create a unique random color by populator elements.

Title: Re: Populator v4
Post by: Dune on May 09, 2013, 02:36:27 AM
I don't think that's possible.
Title: Re: Populator v4
Post by: hughbee on May 16, 2013, 01:03:52 PM
I love Skittles.    ;D