Whether or not I take the liners out depends on the boots, my pair before the ones I use now were so tight I had no choice or I'd struggle forever to get my foot out of the boot no matter how loose I could possibly make them. My current boots which are just as comfortable, I can pull my feet out with ease without removing the liner.
To let my liners dry out, I keep my boots open in a well ventilated area and forget about them until the next time I went riding - which is typically a day or two later. Never had a bad odor come from my feet or boots.. except maybe freshmen year of high school during football season, but that's another story. That involves hookers, cocaine and lots of donkeys.
Oh, and there was never any noticeable wear on any of my liners. No matter how much use I got out of a pair of boots, my liners have usually looked fairly mint aside from them being molded to my feet. For what it's worth, I've mainly stuck with K2 for boots.