...Yet another vote ( from a NooB )
for "Shoulders & Alignment"!
I had this same problem last year about a month after I first started riding. It began happening soon after I started getting good & comfortable linking my turns.
My issue was actually worse than yours as I was getting turned ALL
the way around & winding up Switch!! (I was also weighting my back foot more, so that's probably why I was getting turned All the way around and not just "Drifting" some.)
A quick chat with one of the instructors @ Boyne, and he immediately suggested that I was leaving my shoulders "Open" & rotated as well as weighting my rear foot too much, and if I paid closer attention to getting my shoulders straight & pointed where I intended to go, that would probably fix it.
He was right! The good news was that as a result of having this issue,.. Getting switch doesn't freak me out!!!