Interesting issue, if you scale down perlin too small, it breaks displacement. Either disp disappears or launches down or up exponentially.
I was using a single perlin scalar, and scaling it and translating it for different mask, and I went to scale it down to use as noise for surface microdisplacement, but it just broke the terrain. I had to use a PF at small scales.