Find his real dad. That's how.

He gave you a pass cuz he loved the kid but now he doesn't love the kid more than he hates you.