I am sorry that the scene you are working on is causing so many problems. Without knowing how much memory Terragen is using when it renders your scene (memory use + VM size), I do not know whether the problem is simply that Terragen has run out of address space (2Gb). There is little I can do to reduce the amount of memory used by populations, imported objects and textures. (OK, Terragen could handle textures more efficiently, but I think vegetation textures usually are not the main problem.)
With populations, the main thing that affects memory use is the number of instances. You should be able render 2 or 3 million instances in total (perhaps more if your scene uses few resources), but if you have multiple populations then each population would need to be reduced.
If you have heavy objects in the scene (no matter whether they are single objects or populated), they will use memory, leaving less available to the populators, so for a scene with multiple complex objects you're unlikely to be able to render 2 million instances. When thinking about choosing simple objects for your populations remember that memory use works like this:
Memory needed to load object + memory needed to store positions/sizes/rotations of all the instances in the population.
Therefore, when deciding what objects should be made simpler, you don't need to choose the one that has the most instances. Choose the object that uses the most memory when it is loaded.
Matt
