Rights = freedom of choice. Anything that removes that freedom of choice is impacting rights.
I think you greatly underestimate just how highly the American founding fathers and early Americans in general valued freedoms and rights. They were willing to die for them and they were willing to kill for them. Today we all but beg for them to be taken away in the nane of safety because by and large this country now consists of a bunch of pussies.
"People say that marijuana smoking is going to get in the way of my career. I say to them that on the contrary, my fighting career is getting in the way of my marijuana smoking." -Nick Diaz