It’s about the same as my opinion on alcohol; when done responsibly in moderation it’s fine, but it does impair you, and it does have long-term health effects, though less severe than alcohol.
One other thing is that they both affect those who are still growing far more adversely than those who have reached physical maturity and stopped growing. If you’re a teen, then you will mess yourself up.
As for separating fact from fiction, most claims that it lowers IQ are from biased sources. There are some reputable studied that suggest it may lead to short-term memory issues over time, and others saying that it can slow/halt the progress of Alzheimer’s, so it’s hard to say. However, one thing that is fairly widely accepted is that it does cause cognitive impairment in children and adolescents, but not in adults. While I’m normally against age limits, I think that marijuana is best left to grown-ups.
That said, Carl Sagan didn’t seem to lose many brain cells to marijuana.