I'd say the opposite and get the larges. I'd rather have a binding that works at it's smallest settings versus one that you'd have to max out all the settings. I bet with larges you won't even be at the smallest settings, definitely not if you have a non-burton boot, or a burton boot without shrinkage tech.
With mediums, your straps will be at their longest along with the footbed, and you'll have to use the disks in the one way that adjusts toe/heel drag, meaning you can't use the micro-adjustments of the burton disks when in the tail/nose orientation and are stuck with less options when adjusting your stance. You may also have to deal with getting your foot in and out perfectly when strapping in/out if the binding is TOO SNUG with a medium binding.