Why not just come out and tell him how you feel? Sometimes we see things the way we want to and not for what they are. I suggest sitting down with him telling him what you said in your post and seeing how he takes it. That should help guide your course of action counseling etc.
