I am an instructor and the mtn where I work just started offering snowboard lessons for 4-6 year olds. In my experiences with these lessons, there are a few things to take into consideration. First, is overall coordination. Once you strap these kids in there balance is usually pretty good allowing them to do fine while going straight. However, the more complex muscle movements needed to twist/tilt the board to make it turn are very difficult for most children at this age.
The other things to take into consideration are attention span and energy levels. Both influence their ability to learn and determine how much fun they are having.
So, I think its a great idea to get your kids out there at an early age just dont expect them to be ripping down the mountain at the age of 3 or 4.
However, you do get the occasional 3/4/5 year olds that can pick up skiing and can advance to green and maybe even blue runs.
I've been working on getting my two younger boys (ages 4 and 6) riding, and my experience has been right in line with what you're saying. My 4-year-old seems more enthusiastic than his brother, even though he's less coordinated. He did lessons a few times last season at our local hill (and a couple the year before) and loved it. He can't wait to go more this season. The 6-year-old did a couple lessons, but said he prefers to have me teach him (really, I think he just likes the attention and trying things himself). They can both go straight and and sideslip heelside to stop. The 6-year old can do shaky heelside turns and understands the concept of toeside turns, but has trouble executing them while in motion. Also, they seem to have little leg dominance at their ages, so switch is nearly effortless. I will say their progress has been much slower than my oldest son, who started at age 8 (he's now 11). He and I both took lessons together, and he progressed as quickly as I did (much quicker when it came to jumps and grabs). Then again, my younger ones have really only gone about 5 or 6 times total so far, and you can undoubtedly get them progressing sooner if they go often, assuming they have the interest.
On the other hand, this kid didn't have much trouble at age 3, though he's no doubt the exception, and I'm guessing he got plenty of days on the snow: