The d4 pawn push


You could play 2.d4 in response to the French (1...e6) and the Caro-Kann (1...c6). This is in fact the main line.
You could even try to go for 2.d4 after 1.e4 e5. After 2...exd4 3.Qxd4 you get the obscure and maligned Center Game, which is still playable. However, Blackburne was the latest great player to use it regularly.
3.d4 is played as a matter of course in the Scotch Game. 4.d4 is the main line in the Four Knights Game.
In the Spanish and the Giuoco Piano you normally delay playing d4 until you have set the stage with c3.
These are just rules of thumb. You need to analyze a specific opening to get more info.

In the Spanish, Steinitz Defence is the line 1.e4 e5 2.Nf3 Nf6 3.Bb5 d6. You are correct to play 4.d4, this is a respected line. Black cannot be busted easily, but white will have long-lasting pressure. A book such as Greet's Play the Ruy Lopez could help you get better in this line.

Thanks very much. I have not yet invested in books that explain opening lines because I had a misconception that just reading the ECO would suffice. I was wrong. I will look in to these type of books. This will help me better understand the theories behind the opening moves will it not? I have read a great number of chess books and tried to take it to the board, but not with the greatest success rate. Hopefully this will be different.