I recently moved from MD where 20°F was FRIGID, to Vermont where 20°F is a warm day. It's actually the coldest winter in a decade according to a local on the lift.
I've had two days now where the windchill was down to something ridiculous like -30°F, but I sure as shit wasn't going to take my glove off and check it out on my phone. My water bottle has frozen in my backpack after an hour or so of riding, and my phone's battery drains super fast on cold days.
As everyone has already said, it's the lifts that get me. At the top of the quad at Stowe my face was literally in pain around the little bits of skin that were exposed. I wear two base layers and a mid-warmth jacket and it's otherwise manageable.
All that being said, I've never actually had to do triple layers or anything this year. Yeah, it's been cold in Vermont. But I've had some great days.
No bad weather, only bad clothes.
Yes, I've been chilly on the lifts, but heat up on the runs. It's been a weird season. Rarely a "perfect" day. It's either too cold or too windy or too full of New Yorkers...yet, I've had some great times once the season got started.