Figured out that the guys who did my sway bar install 1) installed it wrong 2) forgot to properly tighten the bolts on one side. I managed to get the bolts out, install them correctly and tighten them. This was the source of the clunking that has been driving me nuts for months. Super happy!
I wish people would do stuff right the first time.