People have mentioned grabbing and holding the outside of your pants legs to keep your arms at your sides.
I'd also recommend getting someone to video you riding. Watching yourself, while cringe-inducing, can be very educational. If you don't have someone to do that, get a Contour and make a frankenpole out of a hockey stick.
I did this one of myself last year:
and it helped me to see that I was using very little edge.