In some cases, most with your situation that I've run across, there isn't much you can do. You haven't done anything wrong- it's how he is and he'll be that way in the future if you allow it. If he starts acting like a douche, cut time with him out until he figures it out. If he doesn't keep up the relationship with you, I recommend giving up on it.
Personally, I've been through wasted family relationships before, and I'm happy that I've cut out an entirely cancerous portion of my family from my life. They're all drug addicts and abusive people at this point. I'm grateful that I had the stregnth to do it then.