When the populator is deciding whether to place an individual tree at some location, the probability that the tree is placed is determined by the shader, where white pixels mean the tree is always placed and black means it is never placed. In grey areas you will get fewer trees, randomly placed. To see what the distribution shader's settings are doing, you could attach the shader to the landscape and render an image. Don't forget to disconnect it again when you use it for your population! Coverage is going to affect the number of trees overall. Fractal breakup tends to provide some variation in the density, but with populations it's often quite difficult to notice its effects with the default settings. It might be more controllable if you change its 'feature size' and 'smallest scale' to something large enough to see when spread over lots of trees, such as 100 or higher.
If your coverage is at 1, then fractal breakup won't have any effect unless you also have altitude or slope constraints. In the fuzzy (grey) regions where the altitude or slope constraints are coming into effect, or where coverage is less than 1, that's where you'll see the fractal breakup.
Matt
