Like from psychiatry to emergency medicine?

I was told that it is hard to change specialties in medicine.

Does one need to start from scratch again? Or do some courses transfer to the new field?