What exactly is it you're having difficulties texturing? Terrain? Plants? Imported objects? The issues and solutions are different depending on what you're trying to do.
Same can be said for the use of masks to control populations (or surface shaders in general). Basically, though, the idea there is to use grayscale images to control the distribution, right? A typical scenario (for me, anyway) would involve limiting the area where grass might be applied. I'd create a camera with an orthographic view, place it high enough above the scene so as to encompass the desired area, point it straight down and render the scene. (No atmo, no shadows). Then using a good image processor (I'm partial to
Paint.NET), I'd create overlays, painting them black where I don't want a particular feature, white where I do. Blur it as desired to smooth the transitions.
Then, use an image shader and import the layer you saved out as the mask. Select 'Position Center' or whatever it is and give it the same X and Z coordinates as used for the ortho view. Select 'Planar Y' for the projection and set the size to the size of the region depicted in the mask (the size of the region shot with the ortho camera).
You can now hook it up to the blending input of a shader. You'll have to mess with settings to get the desired effect - it can enforce a really firm boundary or have no effect at all, depending on the settings in the shader.
Oh, and a really, really good resource for textures is
http://lemog.fr/.
