I just went through that process not long ago after returning to chess after decades. Here's my novice opinion and what I did.
First, go download a free Universal Chess Interface (UCI) program just to see how they work. I like Arena 3.0 or SCID but there are others that are similar. Then download one of the better chess engines such as Houdini 1.5a, or Stockfish 2.3.1 and the opening book from their site. Play with it a bit and see what it does.
Finally, I bought a copy of Fritz 13 which comes with databases, an opening book and other goodies. It handles all common formats and is updated on a regular basis. Its engine is quite good, though not up to Houdini 3 level, but that's not an issue unless you're playing at the GM level or better in my opinion. There are lots of analysis and teaching options in Fritz which may not be attractive to upper level players, but for the rest of us, they work great.
Once you learn how to handle your UCI, forget it for game analysis until AFTER you go back over your games and analyze them without the chess engine. Only then, load the PGN of your game and turn the engine loose to check against your own analysis. You have to do the work in order to learn. Letting the engine work first is far less effective.
Is there any software I can buy where you can learn, play, and have analysis capabilities? I especially like the analysis on here as it is quite clear. I have heard about Chessmaster and Fritz and would just like to know what you guys think of these kinds of programs as I'd like to pick one up.