Ahh righto I never wear my inner gloves. Mainly because its usually too warm and I hate how much it restricts being able to move my hands.
I have the exact same problem with mine when the zipper is all the way up. It hurts my chin and annoys the crap out of me. I just usually have a neck warmer on which stops it digging in but it's still tight. No dramas with the zip yet but I don't use it all that much.
I'm not much of a fan of how big the hood on it is either but that's just being picky.