I’m somewhat like @dr34m3r, I haven’t watched enough people grow up for this to be pertinent to me personally. However, I can both understand the logical argument for it, and the emotional taboo against it.
Logically, if you had never known that individual as a child, it would simply be an issue of dealing with the effects of an age gap between partners. What, morally, makes this taboo, simply because you knew the individual longer? Let me clarify.
You have person A and B. A and B are identical twins separated at birth. You knew Twin A since she was 5 and you were 15. When you are 32, you meet 22 year old Twin B, completely unaware of the twins relation. You fall in love, get married, and have kids. You later find out these two are twins. What makes being with twin B morally straight, but being with Twin A not? Is simply knowing one person longer than the other a legitimate reason to categorically rule them out as a potential partner?
However, you also have the emotional side. While this individual may no longer be a child, when you first met, the relationship created was of the sort found between adult and child. Changing this relationship, and learned attitude, toward this person is what strikes someone as taboo. Our society ingrains it so strongly to discourage people from taking advantage of children. And the feeling that one is using a child will be a difficult taboo to get over.
However, does this societal taboo have a place after the individual is legally, mentally, emotionally, and maturity-wise, an adult? In all honestly, probably not.
Will most be able to accept this and start relationships with adults they once knew as children? Again, probably not.
-Dan