Simple: It's your body, it's your choice.
If you want sex, go for it.
You obviously know what you want to do... anything that happens after is just something you know you'll have to deal with. Make sure it's worth it to you, and not do what anyone else wants you to.