Yeah it's supposed to be -6 Fahrenheit without wind. That sucks to ride in. Once it drops under 10 I usually take a day off. It makes everything so much more difficult to do and enjoy. I also ride 75+ days a year.
The coldest I rode in was -16 °C, which surprisingly didn't feel cold at all, as long as you keep moving. Never really had a chance to try anything colder, but if it's a trip somewhere really nice and you will not get a chance to go otherwise, I'd go for it. Layer up well though, good thermal gear and nothing cotton.