Chess.com analyzes your games exactly the way you want but only those games played online and through chess.com (and I think you have to be diamond member). If you find that software you are looking for, please, tell me.
Game Analysis Software

Chalk up another vote for Scid Vs PC.
What does an example analysis from chess.com look like?
You have to be Premium, I believe. Not positive but I think it's true!
You have to be Premium just to see an example?

Anyway, I use Scid (very similar to Scid vs PC, in fact the latter is a fork of the former). And Stockfish as engine.
Both are open source and free.

Chess King 4 has been great for me. I can save each game I play and have the computer analyze it afterwards move by move. If you want something cheaper, get Chess King 2. It is the same thing.
Free i reccomend SCID vs PC,Crafty is also free but not so pleasing in my eyes (imhO!)
Engines use Stockfish,it's free

BTW, you can read http://www.chess.com/article/view/the-best-chess-of-2014 to check that Stockfish is still the strongest chess engine, so it's not only a matter of money (Stockfish is open source and free), but it's also very powerful software.

Chess.com analyzes your games exactly the way you want but only those games played online and through chess.com (and I think you have to be diamond member). If you find that software you are looking for, please, tell me.
Nonpaying members can get 1 game a week analysed on chess.com, with an analysis rating of about 2000

Non-paying users can get one analysis per week from chess.com, so everyone posting in this forum can get a sample for one of his/her games free of charge.
As for the OP question I can recommend ChessX http://chessx.sourceforge.net. A stockfish engine comes with it in the base installation. ChessX is derived from SCID, another free chess analysis program which is widely used.
I'm looking to start analysing my games with software to improve my chess. I'm wondering if there are any recommendations for appropriate free software. I have tried arena but it didn't seem to work all that great.
What I'm looking for is full analysis of the game for one side. E.g. When I played as white, the best 17th move for white, 18th, etc. I'd also like it to be easy to navigate through and see on the user interface.
Cheers chessmen, (and women)