Whether they can do anything about it, and whether it is illegal, are two different questions.
The answer to the question on this post is that yes, it is considered illegal. Since they are paying for the internet service, and you are not, to use their internet connection without their permission is, in effect, theft of something they have purchased.
As to whether or not they can do anything about it, for one thing they can put a password on it. I have heard stories of people being caught and fined for using someone else’s wireless connection (one particularly absurd case was a man who sat in his truck outside of a cafe which offered free wireless anyway; it was not the cafe owner who pressed charges, either), but I do not know how they go about tracking someone down.