If you don't have a huge tolerance for cold it's gonna depend on how active you stay. If you ride hard all day I think you'll be fine. If you have any concerns just pack an extra layer or two and stick them in a small pack to carry with you.
The only thing I would really suggest would be something to cover your ears. I like wearing a helmet myself (helps with goggle fit and they just feel good to me) and most of them have earflaps. Oh, and you'll want to wear some kind of thermals on your legs too.