The board doesn't know if you have fat or muscle weight. It just knows you have weight.
Now your strength is a different matter. But just because you're lean and big doesn't mean a fatty isn't just as strong.
Back to the point, your strength only matters when you forcibly manipulate a board which you really shouldn't be doing most of the time. Riding is about manipulating your weight more than it is manipulating your strength.
In other words, don't size down just because you're muscle weight rather than fat weight. If anything, like someone already pointed out, it's worse to size down if you're super strong.
There's a rainbow at the end of this scene though, the Proto CT is blunted. This means it's actually a longer board than it is because the tips were cut shorter. So that 154 is actually more like a 156. I weigh 180 and ride the 157 and have no desire to size down. I'm the same way as you in the park; smaller features. However, I do hit whatever I find on the slopes. I also jump more than I jib. If you're into hitting jumps, a 154 won't be as good as a 157.
By the way, I would warn against centering your stance on a board with a setback. Unless of course the sidecut isn't centered around the setback stance (it normally is). I'm not sure if the SL has a centered sidecut with a setback stance. By all accounts though, the SL only has a slight setback so you really don't even need to center it.