That makes sense.
TL;DR version:
With a 10K pot you have a cut-off frequency of 80kHz-160kHz, depending on pot rotation and time constant of 2ms-1ms. I find 0.5ms-1.5ms to be inaudible, and 1-2ms is acceptable.
With a 50K pot and 10K, you get -3dB at 26kHz worst case, and 6ms group delay. In my opinion, this is suboptimal, and I'd consider reworking the values if I had a source wimpy enough to not use a 10K pot.