I would make a mask for the shape first, using simple shape shaders.
Just make a terrain using a powerfractal or heightfield and decide where you want your flat area.
On that position place a simple shape shader and set the size of your flat area in that shader.
Also, in the shape shader, set the border type to smooth bevel and then define how large the soft border needs to be.
You need this to make soft transitions from the flat area to the mountaineous area.
Once set up, connect the simple shape shader as a blendshader to your terrain shader(s) and choose "invert blendshader".
The simple shape shader tells the terrainshader to exist on where the mask is, but since you invert it you will create the terrain everywhere except for where the mask is, thus leaving a flat area.
To get an exact U shaped mountain range you can add an extra step using the same principle. However, it's best to follow above steps first to get an understanding of generating a mask and using those as blendshader.
Create 2 squaresimple shape shaders, say 5000x4000 size and 3000x3000 size and move the 3000x3000 inside the 5000x4000, leaving an edge with the width of 1000 surrounding the 3000x3000 square.
Using a subtract colour function (function -> subtract -> subtract colour) you can subtract the 3000x3000 mask from the 5000x5000 mask.
This leaves a squared U shape.
If you make this big and soft enough and use it as a blendshader for your terrain you won't notice the square shape that much anymore.
