The board will chatter and feel really unstable when you're pushing it too hard. I'm talking probably 35+ mph is where you'd really start to notice it.
If you're not hard carving runs at significantly higher speed than most people on the hill, I wouldn't worry about it, you could probably downsize a bit.
I recently picked up a 4 cm shorter board than what I'm used to riding, and at low speeds it feels about the same, at high speeds it's noticeably more "sketchy" feeling underfoot.
I guess the other question is, are you having trouble with short quick turns now? Trees, moguls, spins and technical riding? If not, I'd stick with what you've got.
Yeah, short quick turns are kinda sketchy sometimes. It's not usually a big problem, but I have to really put the effort in, and the odd time I'll still miss my intended line and have to re-correct. I haven't tested this in situations where it really matters, and figured that I need more practice before trying to dodge trees in narrow trails.
I used to ride casually only a few times per year, and then quit for about 7 years, so it's my first season back actually being serious about it, and there was a bit of re-learning...but I don't *think* I had that problem riding shorter boards back in the day. So maybe the board is a factor there.
I think I'll dust off my old board and get it cleaned up then take it out, just to compare. It's an old Option (RIP) with traditional camber, so it won't be apples to apples vs a C2 BTX with magnetraction, but I'll see how it goes.
From what it sounds like though, it doesn't seem like there is any reason for me to look at longer boards in the future until I'm basically doubling my current speeds. I kinda bought it knowing it was a bit longer than ideal, thinking that I'd just figure it out with practice and be a better rider in the end because of it.