The rainbow won't exactly move when the rain moves, but it will become bigger as the rain moves away from you.
To try it out, water your garden with a hose while standing with your back towards the sun. Then the rainbow is quite small because the water droplets are close by.
Since the rainbow is a projection of the sun through little lenses, the important parts are the observer (you) and the projectee (the sun). When you stand still and so does the sun, the only thing that will change the projection is the lens. If you move a magnifying glass toward the ground the projection gets smaller, moving it away makes it bigger.
But when the rain moves side to side, the only thing that changes is that you will see the same object (the sun) projected trough other lenses, roughly at the same position.
So the projection of the rainbow will not move, but it can change size...