How do I file for sole custody if the father is on the birth certificate?

My son is 12yrs old and Ive raised and have supported him financially 100% percent. His father & were never married but he is on the birth certificate...his father has 5 other children, 3 of which he lost complete custody of. He's been in and out of my son's life due to drugs and prison and is now rehabbed...I have allowed him back in my son's life because I felt it was in my son's best interest however, he wants to dictate the visitation etc or in other words, just come and go as he pleases which is not in the best interest of my son. He is either in 100% or not all...his ignorance and lack of ability or knowledge to be a real father is frustrating and creating a lot of problems which affect my son. I am now at a point where I feel like I need to declare sole custody and have everything in writing to clear up any confusion. I never filed for child support and he does NOT support my son at all...I make more $$ than him and he owes back child support from his other kids. Im afraid that if I take him to court, the court will decide that I have to pay HIM. where do I start?