Hostas grow well in shady areas, and spread in clumps, but are not invasive. They come back year after year. Ferns also grow well in shaded areas, as does vinca. I have a small, shady yard (30 ft x 60 ft) with a privacy fence on two sides. My neighbors on either side have very old elm, walnut and chokecherry trees. I have hostas along the fence and in a center bed that I planted 12 years ago, and they are quite manageable, and bloom in late summer.
It would be helpful to know what kind of trees you have, and how old they are. You may need to invest in limb trimming if they are quite old. That can make a big difference in the amount of sun that gets to the lawn. Like @worriedguy said, certain trees produce fruit that are toxic to grass, such as black walnuts. Hard leaves, like magnolia and oak, are hard to compost, and if they’re left to sit on the grass, will kill it off.