Why shouldn't I be mad that my boyfriend has no job?

My boyfriend and I have been living together for the past year. I have two children that live at home and he has 1 child that lives at home also. In Sept. he was fired from his high paying job because of being wrote up to many times. So it is now June and still has no job. I have a lot of mixed emotions about this..He has made very little effort to find a job and he always got something planned in the future with someone who wants him to work...but guess what? He never follows through. I get up every morning and go to work while he is still laying in bed. It starts my day off all mad and I can't seem to get over it. His argument is: "Well I am getting unemployment." "There is just no jobs out there." "I don't want to work there." "That one doesn't pay enough." I am so frustrated. Being a single mom for so long and dealing with those struggles, I learned to work anywhere. You have to start somewhere. We downsized and moved into one of my fathers rent houses but I get upset and embarred when we can't pay rent on time. To help us out my father has put him to work doing a few odds and ends, but I don't think it's right for him to get paid by my dad when we owe him money anyways. Every time I bring all this up to him, and tell him how I feel. He says I am wrong for thinking this way and that I am being childish. Am I wrong for being upset about this?