I used to struggle with these thoughts too when I would find myself studying master games and techniques to improve my game for a tournament or rating. I just came to the conclusion that If chess makes you happy and it doesn't interfere with your priorities than keep enjoying the game.
I personally try to keep my study down to about an hour a day visiting sites like chess.com, chesstempo.com, chessbounty.com, etc. Just my two cents.
True story