I share the same belief with you, that a couple shouldn't live together until they are married. When this is something you believe, and something you want to follow I understand that going back on that can be hard. And that little voice in the back of your head starts to drive you insane!
I suggest that, for your sake and your relationships, that he moves out. This will most likely make your relationship even stronger, and you'll find out how much you truly love each other. He says he wants to marry you. Don't rush him. Don't rush into anything permanent that might not turn out to be the right decision in the end. I do think that him moving out till you guys get married would be the best choice for you guys.
Good luck!







