Good question, I actually had that question myself, as during some live chess games my opponents have played a very early Bg4 and was wondering why that is not the norm if Bg4 is a good play.
Ruy Lopez main line... Help me out here!
I believe that the reason 9. d4 Bg4 is generally considered OK for Black is not so much that it pins the f3 knight as that by doing so it puts indirect pressure on White's d4 pawn (since the f3 knight is one of the defenders of that pawn). This makes it difficult for White to maintain his ideal pawn centre. If Black plays 8...Bg4 before White has gone d4, however, then it has less point; White isn't obliged to push d4 but can instead play more quietly with 9.d3 (or 9.h3 Bh5 10.d3), aiming to play d4 only later. The purpose of Black's bishop move is perhaps less clear in this case.
This doesn't mean that 8...Bg4 is a terrible move, of course, and some strong players have employed it (Romanishin used it to get a draw against Anand in a candidates match in the 90s). All these variations are quite playable, I think (including 9.d4), especially at the level of we mortals who are more likely to lose a game by hanging something on move 34 than by misunderstanding some opening subtlety. Still, the differences are there.
I've not played the Ruy Lopez myself for a while and I'm sure there are other people out there who understand it far better than me, so don't take any of this as gospel.

Hi,
8..Bg4 is not mistake, not bad, but white can play 9.d4 and take better touch on center. Continue 9...0-0 10.Qd3 Bxf3 11.gxf3 or 9...Bxf3 10.Qxf3 exd4 11.Qg3
I recommended for you that play during a mainline 8...0-0 9.h3 and now 9...Bb7 and put pressure to the center that way. This is my opinion.
I believe that the reason 9. d4 Bg4 is generally considered OK for Black is not so much that it pins the f3 knight as that by doing so it puts indirect pressure on White's d4 pawn (since the f3 knight is one of the defenders of that pawn). This makes it difficult for White to maintain his ideal pawn centre. If Black plays 8...Bg4 before White has gone d4, however, then it has less point; White isn't obliged to push d4 but can instead play more quietly with 9.d3 (or 9.h3 Bh5 10.d3), aiming to play d4 only later. The purpose of Black's bishop move is perhaps less clear in this case.
Thank you, this makes a lot of sense. Playing 9. d4 immediately is rather forcing...white has to play either Be3 or d5 after 9...Bg4, it seems. Not bad, per se, but it cuts down white's options.
Something I've never understood about the Ruy Lopez:
Looking at the main line...
1. e4 e5 2. Nf3 Nf6 3. Bb5 a6 4. Ba4 Nf6 5. O-O Be7 6. Re1 b5 7. Bb3 d6 8. c3 O-O 9. h3
White plays h3 to stop the annoying Bg4, which pins the f3 knight. Personally, I have found that when I allow Bg4 as white it takes away my iniative and stifles my game. Now what I for the life of me have never understood is why black can't just play 8...Bg4 instead of O-O. If Bg4 is a desirable play, I see no reason why black can't play it one move earlier. If not, why does white even bother to waste a move with the passive h3 on move 9?
Please, someone who understands the RL way better than me, help me out!
Cheers,
Jay