People make me feel like I am doing something wrong, Is being a Mom only tough for ME. FTM to 5 month old....?

You sound like a great mom. It takes selflessness and that seems like what you have. I wouldn't listen to people who are trying to bring you down.