Welcome to the forums
To create extensive connected canyons I'd suggest to start with this shader:
http://forums.planetside.co.uk/index.php?topic=6432.0This shader creates cracks which also don't look too straight.
"Simply" increase the scales to your taste (I'd make them around 2-3 kilometer-scale at least If I understand your idea correctly). Don't forget to scale the displacements, set in the 2 surface layers, accordingly.
You now have huge cracks which you can use as a base for the canyon.
To add detail to the walls you can use a redirect shader (under displacement shaders) to displace the vertical walls.
Attach a powerfractal to the X and Z port of the redirect shader and play with the scales. In general a nice start is to have the displacement factor half the size of your feature scale.
This should get you started I think
I'd like to see how this works out for you. If you need any help then let me know. I've made a couple of canyons myself too.
Cheers,
Martin