Masking is easy. I often use a final surface shader as a tester (purple color). Take a PF, make a hard noise that makes patches you like on the ground (feed into masking slot of that final test surface shader), and if satisfied plug it in the masking input of the tree population. And turn testing surface off of course.
I made you a little method I often use. It's made in version 3.7, so there are other shaders and functions now, but the idea is there. There are lots of ways to mask, but this is one.
Btw. multiplying the main mask by a distribution shader for angle and altitudes is needed if you want to use the inverse of a main fractal mask. You can't just mask the fractal by the distribution shader, as you need to complement the fractal and the distribution shader will then also be complemented. This is a workaround for that.