Three things: flex, lb/edge cm, lb/surface area. You flex the board more, could possibly overflex or break it IF you'd do butters, land on the nose. You also put more pressure on the edge, if you carve at higher speeds, the edge wouldn't hold cos you have too much force on it. And if in pow the board lacks surface to float with your weight. IF.
But for simply having fun cruising on the groomers, it's no harm to have a short board, especially as beginner/intermediate.
"Yeah... no, that's not clumsiness, that's just being topographically challenged" ~ Noreaster