In my experience this all depends on the content of the render but I would say downsampling is probably not needed if you get the original balance of render settings just right. However, I do have a render going at the moment which I have had to give very high detail and AA settings because it has lots of very narrow, long and contrasting but slightly modulating details, meaning that if I don't use these high setting I will get a lot of pixelated looking lines running across the render. Smaller details in the distance such as stones or grass can also cause problems because they are details at pixel level that will can noisy results unless the render settings are appropriate to smooth it.
By the way, in the situations I describe above, what is best to tweak higher - detail or anti-aliasing. Ot what kind of relationship between the two. Is the answer to concentrate mostly on anti-aliasing to smooth these pixelations?