Any strong glue will work for the drivers. Polyurethane is what is commonly used as it is very strong and stable but with a long cure time. Martin Logan use tape but they also use it as spacers.
You can use poly to attach the driver to the baffle but it is very strong so odds of removing them ever again, are slim. Better use double sided tape as that's what Stax use. Now theirs is crap so any strong glue that doesn't leave a lot of residue works. I bought once everything I could find locally and some Tesa carpet glue worked very well. Two different kinds though, the transparent one which is reinforced with some twine is the best one. Ditto for the earpads, double sided tape there too.
Now with the dust cover compromised you can expect some squealing. How much is impossible to tell.