There are a couple of bottom lines here:
1. If you don't know and don't know how to check how your speaker amp's outputs are configured you shouldn't be messing with it. The Grado adapter picture clearly shows 4 conductors so that it doesn't have to worry whether or not the output of the amp has a common ground connection.
2. It's pretty likely that the Grado adapter had its resistor values chosen for a Grado headphones (all of which are 32ohm). Assuming that it'd be either appropriate or useful with headphones of varying impedances is a "at your own risk/peril" type of deal. It may work great, it may load your amp in such a way as to cause it to go into thermonuclear meltdown.
Me, I'm content to have a separate speaker and headphone amplifier.