They’re probably “People Watching.” That’s one of my favorite pastimes, and I wouldn’t be offended if I were you. It’s just interesting to watch what people do when they’re alone, how people interact, and things like that.
Also, at school I look at everyone I pass in the halls, sidewalks, scan the cafeteria when I walk in, etc. My university only has 1.3k students, so if I don’t look it could appear like I’m ignoring someone that I actually know. I’d rather look and smile than be rude and look down. :]