You have to take the d pawn. It's the best line.
This is an example line. The Scandinavian is not a refuted opening so it's still a game and you have to play good chess to win. White should be better here though as black has wasted moves with the queen.
Alternatively you can play d4 and transpose into a Blackmar-Diemer Gambit (instead of exd5) but it's unsound.
On 1) e4 d5 2) exd5 Qxd5 3) Nc3 Qa5
a : I don't really know what's going on but In chess.com's lessons on the Scandinavian Defense from memory before there was a video of Niguel Short that advised the o-o-o strategy with x) d3
1) e4 d5 2) exd5 Qxd5 3) Nc3 Qa5 4) Bc4 Nf6 5) d3 then for example 5...c6 6) Bd2 Qc7 7) Qe2 Bf5 8) o-o-o
b : In forum a GM advises the king's fianchetto
1) e4 d5 2) exd5 Qxd5 3) Nc3 Qa5 4) g3 then for example 4)...Nf6 5) Bg2
c : the o-o-o strategy with x) d4
1) e4 d5 2) exd5 Qxd5 3) Nc3 Qa5 4) d4 Nf6 then for example 5) Bc4 c6 6) Bd2 Qc7 7) Qe2 Bf5 8) o-o-o
On 1) e4 d5 2) exd5 Nf6 :
a : I have no real advice just play 3) d4 Nxd5 and we fall into a position with a space advantage.
b : I don't like 3) c4 because I find the Icelandic gambit (3...e6) is strong and 3)..c6 4) dxc6 Nxc6 is probably even more strong.
c : Bots can plays 1) e4 d5 2) exd5 Nf6 3) Bb5+ Bd7 4) Bxd7+ Qxd7 5) c4 and with the exchange of the Bishops the gambit seems potentially less dangerous but after 5)...c6 6) dxc6 Nxc6 7) Nf3 e5 we have one developed Knight while our opponent has two developed Knight + developed Queen + control of the center and the Bf8 who has an activation + column c and d... in short I don't like the White position
Any Idea how to counter Scandinavian without taking Blacks D-Pawn? I can't win these kind of games!