Yes, that's why I never liked the HE 60 - I have now heard many people say it sound's much better on other amps.
Actually someone saying "nothing special" normally says to me that the headphone is actually excellent - because if it's doing everything accurately and not adding or subtracting anything from the sound. A superb unit will actually sound "nothing special" and will probably be the best.
When people start using words like "amazing" etc. I tend to feel that the headphones are coloured and are changing the sound. Nothing against coloured headphones, of course, for pleasurable listening if you like such things - but my own personal choice is always to tell me what is there without adding and without taking away.