No, it shouldn't be getting hot with the temperature control switched off.
I would expect the problem to be one or more of the control relays to the heater to be stuck permanently 'on'. The relay contacts should close and open at a suitable temperature but wear and tear is likely to be the cause of the contacts welding themselves together overriding any control.
Turn the heater off at the mains isolator switch as it could eventually boil the water.
Dave.
------------- Give a man a fish and he will eat for a day,
Teach a man to fish and you can get rid of him for a whole weekend.
It will be just like the thermostat sticking on in an immersion heater. And probably would boil if its left long enough without draw off. Needs sorting ASAP.