maybe I'm so used to it now, that I have forgotten about 32 bit speeds
The "bit'dness" of a processor has nothing to do with its speed.

The fact that the majority of 64bit processors are "faster" than their 32bit counterparts is a simple matter of technological evolution.

Take a 64bit processor and down clock it to the same MHz/GHz speed as a comparable 32bit processor, test the functions they have in common (ignoring SSE/MMX or whatever might be added between generations of CPU) and there won't be any performance difference (assuming they have similar pipeline depths etc).