So as I mentioned before I was having some fit issues with the overall build. The result of that was that the bottom of the front panel looked like this.
That was unacceptable in my book and I wanted to see if I could at least make it better if not go away entirely. The solution, countersink the back of the front panel to allow the body of the pot to nest into it. So I mounted the panel in the vise on the mill, measured the pot body (just over 1/2" in diameter for the part that was binding on the front panel) and selected a 5/8" diameter end mill.
Unfortunately I wasn't able to completely cure the problem, the countersink would have been uncomfortably deep, so I decided to go with roughly half the panel depth as my max and see how it worked. The gap on the bottom is now less than half of what it was and is something I can live with. Moreover it was a proof of concept of something that I'd be wanting to try ever since I got the mill, namely that even after drilling and unmounting a panel I can remount, recenter and counter bore with good accuracy. This will be a nice bullet to have in the gun for future use.
[edit] Decided that I was being too much of a kitty cat and counter sunk the pot into the panel a bit more, fit is now really good. I also found two 1/4" male/female standoffs in the parts bin which work perfectly for supporting the pcb around the tubes so those went in too. Now I just have to solder in the two 221ohm resistors to lower the CCS resistor and I'll be done.