Yep, boot fit is a lot more important than just the size listed on the boot. Some boots run wide, some boots run narrower. I have a narrow heel and finding boots that fit well is always a PITA. I've found that adding addition j-bars and the SOLE Ed Viesturs footbeds really helps my boots.
Sometimes though, you have to choose between comfort and response. I've found that I'm willing to sacrifice a little bit of response to gain a lot of comfort.