Can anyone tell me what the goal is for the Elephant Gambit for Black? I've read that the main line is that Queen takes d5, but I've seent in the Scandinavian Defence, that that not really a good respons is. So... I try something else.... Instead of Queen takes pawn, I play e4.... Is it good?

According to game explorer, e4 is slightly more common than Qxd5.  The mainline runs Qe2 Nf6 Nc3 when Black has a choice of either recapturing on d5 or gambiting a pawn.  White often comes out of the opening with slightly more active pieces, but not much else.

In 'Gambit Lexicon', IM Alexander Bangiev recommends 3...Bd6 or 3...e4.  Of the two, I prefer the former.  Here's an example:

Just one question.

If the Elephant Gambit and/or the Latvian Gambit was/is so good.

Then WHY did no one ever play them against Robert J.Fischer, Mikhail Tal, Leonid Stein, Boris Spassky, Tigran Petrosian or any of the other REAL IGMs of the past.

And today NO ONE dares play these wonderful gambits against Vishy Anand.

Why hummmmm (CHICKEN ?)

Because when you have to win and place well in tournaments to feed your family there's a large incentive to play as safe as possible. 

The real question is, why do you care if an opening is played against Vishy when you/we play chess in a completely different universe?  Gambiteers play for fun.  It's really that simple.

There are grandmasters who do indeed play the elephant gambit.  GM Keith Arkell and FM Jonathan Rogers of England have the elephant gambit in their reperitores.

I have heard from more than one source that the elephant gambit is rather unsound and usually leads to a worse position for black with accurate play. Unfortunately I always seem to run into trouble against it (mainly in blitz because I rarely see it anywhere else) because I find that the position of my queen on e2(after black plays e4) gets in the way of everything especially my king's bishop. I have to do some preparation in this line forsure because I have been blown away too many times by it haha.