Imagine that your body is made up of cylinders representing the torso, legs and arms. As the wind blows around you it effects the breathability of your clothing. Where it increases pressure, into wind, breathability is reduced. Where the pressure is reduced, breathability is increased.
This image shows stable air flow around a cylinder (white). The red and yellow region represents increased pressure and the blue area represents reduced pressure. As there is more blue than red, this indicates that, on average, the breathability is improved by the action of wind. (Reynolds number = 10. Click here for other Re.)
Note that this does not take account of the fabric flapping, which should improve breathability.