So I just realized you can't invert a planes normals (to have it above the camera) and have "Normal" displacement that's positive, it will act like your positive displacement is negative. If you swap edge AX to -1 and BZ to -1 the displacement will be correct, but the planes normals won't.