From looking at the image, I see the grain appear mainly where there are godrays. The fact that, as you mentioned, the clouds look fine in the top left corner is because the sun doesn't shine through the atmosphere between those clouds and the camera (the clouds in the distance are shadowing the clouds in the foreground).
Have you used Ray Tracing in your atmosphere? Either way, it am certain that it is an atmosphere samples issue. Try 64 samples in a cropped render and see if that cleans your image right up.