WALove, for sure you'll destroy the pins in the Voile brackets. I'm pretty sure that it's not advised to do it the other way around either, going off of memory here though. So maybe my advice was adviceless...
the bushings in the LT brackets are roughly 1/4 inch on each side and fully support the pin, and it rides on plastic bushings = no wear. The thin steel voile bracket only supports the pin on a small area, the high forces caused, and the steel on aluminum contact creates the wear problem. Touring in Bozeman means im around a lot of sparks.
side note: voile steel pins are too short to use with spark binders