I didn't work on The Martian, but did work on the series called Mars, using Terragen. Additionally I'm currently working on a BBC show about the solar system, oddly enough including Mars again.
Get yourself over to the NASA JPL site (
https://mars.jpl.nasa.gov/msl/) and dig around for references, especially images shot by the Curiosity rover.
Try to concentrate on one thing at a time, whether that's undulating sand dunes, rocks of a certain type and size or huge mountains and cliff formations. I've spent days in the past just making interesting atmospheres or working on the huge rock structures like you see in your Martian refs there. Imagine you're working on a painting - do the large brushstrokes first and work them up. If you're new to Terragen, don't test everything in one scene at once - have a scene for cracked ground, one for a mountainscape etc, then worry about bringing them together later.
One more thing - TG works in metres. Don't forget that or your mountains will be pebbles in other scenes.