It's quite easy to remove the lens, not so to put it back on.
The difficulty lies in ensuring all the c-shaped hooks/cut-outs around the lens clip onto all the corresponding tabs in the frame.
I've always had extreme difficulty locking the final one in place, all the rest leading up to the final one are reasonably manageable.
Also, you should be very careful with where your fingers are placed and make sure you've gotten yourself a secure grip. In e midst of tangling with the goggle and forcing the tab into the cut-outs along the lens, one of my mates had his fingers slip and tore the foam on the frame.