I found this on the web today and I think it says, in perfect words, what I am realizing lately.
"The principle of going your own way and doing your own thing is the principle of non-intervention. This principle is lived by those who have an enlightened consciousness. The principle is the principle of freedom. You are free from having to meddle with the affairs of others, and are free to live according to your own goals and agenda. Every man’s purpose is to follow his own path and only engage with the affairs of others where it intersects with his path. Always live in the center and never move off the center.
Seek to relate with those who can best fulfill your life purpose. You have no business to relate with everyone in the world. Successful people are the ones who concentrate on their goals and harness all their resources towards the accomplishment of their aims. This might be the selfish way to live, but it is selfishness with wisdom. Everything is done for the Higher Self, and the Higher Self leads you by desires within yourself. No matter how noble something seems, it never feels right when it is not your life path."