Another issue with using billows, is the voronoi folds can end up on your buildup, and no way to remove them. Similar issues when creating lateral displacement with voronoi, I'm sure you're aware.
Idea
It would be cool if there was a function for the world light source position (based on whatever input sun), that could be used in conjunction with the dot product of the terrain and normals to do what we want via sun position. Now shadows needed, just position of a sun.