My Factions are held in with velcro tabs, and i could totally see the tabs ripping loose if one wasn't cautious when removing them. That said, that velcro isn't doing anything when my foot is in the boot. I think as long as you open the shell all the way and grab the liner down low instead of yanking at the top, it shouldn't cause any damage at all. And the moisture left in there is going to do damage if not allowed to dry.
"my only interest in statistics is in not becoming one"