Yes, it is reminiscent to them of when their mother licked them as puppies.
Additionally, you can tell by the quiet sounds they make when you pet them over a long period of time, that it just plain feels good and is relaxing.
Cats are similar, I think, but they mutually like rubbing against things, even us, I assume to mark their territory with pheramones. It implies bonding and ownership.