An important result that Kees Veenenbos achieved here was long shadows and a high percentage of the foreground being in shadow. The sunlight hitting the ground and rocks appears to be quite a saturated orange which contrasts nicely with the neutral greyish frost. Also it seems like the frost is picking up light from different directions from the other surfaces. I don't know how he got that effect, but if I had to guess I'd say that the frost uses a reflective surface with a blueish reflectivity. He may have added another light (low intensity) which the frost could reflect if its reflectivity was set high enough. The frosty surfaces seem to pick up light from different directions from the main sunlight, suggesting some combination of reflectivity and maybe a couple of different light sources, but it's hard to tell. Maybe he achieved a lot of this in post.
Kees's image works very well, but I personally don't think it is photo-realistic because there is no way the frost could be that bright and neutral coloured in the shadows in the face of the environment it's in. The only things the frost has to reflect are a reddish environment, so the frost should take on the same hue. The sky that the frost is reflecting is red, orange or black, and Kees's frost looks like trying to reflect everything except the sun which is the only thing that might be neutral-coloured. However, what Kees has done has sold the impression of frost as we are used to seeing it, giving the impression that it's slightly reflective by pickup up light from different directions, and shown a strong colour contrast between the red rocks catching the sun and the frost reflecting the rest of the environment, which is what we're used to seeing on Earth. I think that's one of the reasons why it's a successful piece.
I've taken a stab at doing something similar. Normally I try to stay away from 'trick lighting' because I love absolute photo-realism, simulating realistic lighting and knowing that I've got a good result by simulating reality, but it's still interesting to try a different approach and do something that wouldn't be possible with absolute photorealism. Here are my steps:
Made the sun lower (5 degrees elevation), to elongate the shadows and put much of the foreground in shadow. This is quite an important component to the effect that Kees achieved in the picture that you referenced. I also found a part of the landscape where the foreground was sloping away from the light a little bit, which helped to elongate the shadows even more. If you don't want to lower your sun then choosing the right piece of terrain might be an important tool for you.
Enabled do soft shadows on the sun. Increased the soft shadow diameter to 3, which is quite an exaggeration, but it helps to soften the edges of the shadows. This is believable because when the sun is so low in the sky most of the light could be coming from the light scattered in the atmosphere, which produces much softer light. Normally I'd prefer to use a second sunlight to simulate this effect, but to keep things simple I haven't done that here.
Made the sun quite a saturated orange colour. Maybe Kees didn't do this, maybe it was just his rocks that were strongly saturated, but I've opted for something simple. I also gave it a strength of 10. Maybe you just want to adjust your rock colours to give them a stronger, brighter red colour so that you don't affect the sky you already have.
I added a Surface Layer which I called 'Frost' and gave it a white colour. Later on I'll add some reflectivity to it, but I'll come back to that.
Added a second sun that I want my frost to reflect (called it 'Frost Light'). Its colour is white. I want my frost to reflect this light but I don't want it to affect the rest of the scene too much. So I've given it low strength (I chose 0.1). I disabled glow in atmosphere, kept specular highlights enabled so that my frost can reflect it. I put the Frost Light at the same heading as the main sunlight but gave it a much higher elevation (45 degrees), thinking that this would be a good angle to reflect off the surfaces. I disabled shadows on this light because I don't want shadows to break the illusion and remind us that there's really a light source up the sky that shouldn't be there.
I decided to disabled the Enviro Light (and set GI relative detail to 0 so that the pre-pass isn't rendered). I think the Frost Light will give me quite a lot of control over the look of the shadows, and we want some of the backfacing rocks to be very dark. You could keep the Enviro Light but reduce its intensity maybe. The Frost Light is adding additional fake light that wouldn't be there in reality, so I'm not really tied to the "correct" lighting that the Enviro Light gives us.
I rendered what I got up to this point. It's called MarsPolarFrostImpression_v01.jpg and I've also attached the associated project file.
Now I want to get the reflectivity working.
Select the 'Frost' layer in the shaders node list, Add Child Layer -> Other Surface Shader -> Reflective Shader. Go to the Reflective Shader and turn ray traced reflections OFF. They would dramatically increase the render time. Turning them off means that the Reflective Shader only reflects specular highlights from light sources and the Enviro Light. EDIT: I forgot to do this in the project files I attached!
Immediately I see it reflecting the sunlight in the 3D Preview, but I want to stop that from happening. (In Kees's image, the ground has some weak specular reflection of the sunlight as you look to the horizon, so maybe I'm on completely the wrong track here, but I'll see keep going anyway). On the sun, I disable specular highlights. However, I still want my 'Frost Light' to have specular highlights because that's why we're making the Frost reflective.
Now I want to pump up the reflections so I can really see the Frost reflecting the Frost Light. I increased the highlight intensity to 10. Alternatively I could have pumped up the reflection tint to 10. If I was doing that, I'd chang the reflection tint, not the reflectivity, because the latter really should never go above 1 because of how it affects reflections of dark objects. None of this is an issue if I just crank up the highlight intensity.
A quick render shows that it's working but the highlights are too 'sparkly'. Bumping the specular roughness up to 1 fixes that.
The frost is brighter now but I'm still not getting the reflective effect I want. My next step is to go back to the frost surface and make it black. I'm trying to make all of the light on the frost to come from the specular highlight of the Frost List. Then I turn up the reflective shader's highlight intensity to 30, and I tweak the elevation of the Frost Light until it picks up parts of the surface that I like. I settled on 60 degrees elevation.
The reflective effect didn't work how hoped it would. I've rendered those steps as v02 and attached the JPG and TGD.
Looking again at Kees's image, I realise we need to be much brighter, but not in an overall way. He's really brought out detail in the shadows somehow. Maybe some of this was done in post. He also has a lot more detailed displacements in his foreground than I do. Anyway, my next steps were:
I doubled the sunlight strength to 20 and reduce the atmosphere colours to compensate. Increase the Reflective Shader's index of refraction to 10 to try to pick up more reflections from angles facing towards the camera. This made the frost much too bright to I brought the highlight intensity down to 4 (from 30).
I rendered this as v03.
It's still not right. I want the shaded side of the rocks to be darker. My Frost Light is lighting up the rocks too much, so I've brought its strength down to 0.05 and increase the Reflective Shader's specular highlight intensity to 5. I've also decided that the frost seems a little too blue, so I've given the Frost Light an off-white subtle peach colour (RGB 255,225,200). The rocks might work better if my Frost Light were at a lower elevation but then I can't seem to get other aspect right. Maybe another light is needed.
Unfortunately I've lost the bright whites in the foreground. It doesn't really compare with Kees's image, but he probably went about this in a completely different way. He's doing something to get the foreground bright without losing contrast overall. It looks a bit like it's been processes with some HDR tone mapping tools. Some things are much easier to do in post, especially when breaking away from photorealism.
My final version is v04. I didn't really succeed in what I set out to do, but I hope you find some of these ideas useful. You might get better results than I did because you have more detailed displacements in your foreground which might catch the light better.
Matt