It’s all a function of surface area and how well you can hold onto that sucker as it shrinks. If you can’t hold it securely, then a lot of times you won’t be able to rub it against your skin, washcloth, or whatever. In that case, no lather is produced, even though you’re expending the same (or more) effort.
I use Ivory soap bars, and when they start to get that size I press them onto a new bar. I have months and months worth of soap bars layered together in this way. My soap could be a geology experiment someday.