Yes, there is always benefit to even small exercise bits. They do add up.
It just happens to be that when one does hard, intense exercise, more energy is burned as your muscles move more, and it may also rev your body up such that while you are calming down and showering, your body is still burning a few more calories. There’s probably some different body chemicals released when engaging in longer periods of exercise, which may have an overall impact on the efficiency.
The thought that exercise equals high heart rate is not true, as evidenced by pilates or yoga which will leave you feeling tired and well exercised despite not keeping a high heart rate all the time.
It is true though that you lose weight the fastest while engaging in more cardio versus strength training. So it really depends on your goal. If your goal is to be more active, then certainly going for a brisk walk over lunch or taking stairs over the elevator will help, and as @GAMBIT says it’s helpful for getting your body used to more activity. However, if your goal is pure weight loss, then you will not see immediate nor drastic benefits from small contributions, and I have a hunch (but no data) that you would not lose weight at the same rate if you substituted one 20-flight walk with 5 4-flight ones.