Only you know what is best for you. Either your don't care about the financials and his difficulty obtained his GED. If you can't get over those things, then you have to end it. It is not mean or selfish of you to do that, but you have to be firm and it has to have finality. You would need to tell him that you cannot get past that, it will not work out, and that it is your final decision in the matter. If he begs you to come back or tries to make promises, you have to stand firm.
Before all that, though, you need to decide what is important to you.