I believe jealousy has allot to do with it.. Having a child with someone is the ultimate display of connection and most people can't handle not being 'the one' who makes that connection with you.. There is this 'second best' thing hanging over their heads.
And then their is the people that don't want 'the hassle' or the responsibility.
Meself.. I would gladly date someone with kids.. I married someone with one.