Old dog this is a familiar story. I was sparring with a high school girl that caught me off balance with a round house to the head. Needless to say I was on the floor seeing stars. I loved fighting guys 1/2 my age, but it does take it's toll on your body.
I started riding 3 yrs ago at 42.
You sound solid but don't over look the Azzpadz recommendation. A little pain is fine but when/if it ends your day when it could have been prevented and you'd still be riding. Why not use them and enjoy your new sport a little more.