The change to 1.d4, should a 1550 do it ?

"... For players with very limited experience, I recommend using openings in which the play can be clarified at an early stage, often with a degree of simplification. To accomplish this safely will take a little study, because you will have to get used to playing wiith open lines for both sides' pieces, but you can't eliminate risk entirely in the opening anyway. ... teachers all over the world suggest that inexperienced players begin with 1 e4. ... You will undoubtedly see the reply 1 ... e5 most often when playing at or near a beginner's level, ... After 2 Nf3, 2 ... Nc6 will occur in the bulk of your games. ... I recommend taking up the classical and instructive move 3 Bc4 at an early stage. Then, against 3 ... Bc5, it's thematic to try to establish the ideal centre by 4 c3 and 5 d4; after that, things can get complicated enough that you need to take a look at some theory and learn the basics; ... Of course, you can also play 1 d4 ... A solid and more-or-less universal set-up is 2 Nf3 and 3 Bf4, followed in most cases by 4 e3, 5 Be2 and 6 0-0. I'd rather see my students fight their way through open positions instead; however, if you're not getting out of the opening alive after 1 e4, this method of playing 1 d4 deserves consideration. ... a commonly suggested 'easy' repertoire for White with 1 Nf3 and the King's indian Attack ... doesn't lead to an open game or one with a clear plan for White. Furthermore, it encourages mechanical play. Similarly, teachers sometimes recommend the Colle System ..., which can also be played too automatically, and usually doesn't lead to an open position. For true beginners, the King's Indian Attack and Colle System have the benefit of offering a safe position that nearly guarantees passage to some kind of playable middlegame; they may be a reasonable alternative if other openings are too intimidating. But having gained even a small amount of experience, you really should switch to more open and less automatic play." - IM John Watson in a section of his 2010 book, Mastering the Chess Openings, Volume 4

My First Chess Opening Repertoire for White
https://www.newinchess.com/media/wysiwyg/product_pdf/9033.pdf

First Steps: The Colle and London System
https://www.chess.com/article/view/how-to-understand-openings

I would suggest you move to 1.d4 if you feel it will be good for you. You will learn new kinds of positions. Then come back to e4 to see if your game has improved.

I encourage you to make the switch to 1.d4. It will increase your knowledge. Then you can always come back to e4

Also, against the Sicilian, maybe you can try the Botvinnik System against it. It isn't the best response against the Sicilian, but it doesn't involve crazy amounts of theory. Melbourne Chess Club has a decent video on it.

play the french exchange then you got a pawn on d4 anyway

I just want to say I am very similar. 8 years ago I was rated 1566 and then I switched to d4 in the middle of a tournament. The main reason I switched was because I found my games uninspiring and needed to switch something up (although my results with other openings were decent). This is not to say you can't find interesting positions with e4, but making the change without worrying about studying theory worked really well for me. I didn't even lose rating points while making the change which surprised me a bit. Now I am rated 1903 and still playing d4.

It really is true that after moves like 1.d4 Nf6 2.c4 e6 3.Nc3 Bb4 I was already out of theory and had to improvise. I did really well with just the basic opening principles though and learning from my mistakes when analyzing games afterwards (which of course included the opening).

Note that often-closed positions like those reached in the French and Caro-Kann resemble the kinds of strategic positions requiring a lot of maneuvering to reach dynamic positions that are more typical of 1 d4 than 1 e4 openings.

And the Caro-Kann's cousin, the Slav, has basically the same typical 1...c6, 2...d5, ...Bf5, ....e6, ...Nbd7 setup that typically closes things up. The Queen's Gambit Declined is even more closed.

I understand, but I think my disatisfaction with playing the white side of the french and karo is that you are on pressure to generate activitie and attack and it's black who seems to have the structural and strategical advantage. In the QGD and Slav you can attack with white whistle having the positional advantage, I guess.

Great stories, guys, I think this is the rating to make that change now, or it will be harder if I get stronger.

you're 1550. Play whatever you want. You don't even need to play a systematic opening. You can play Bird's Opening every game with White, if you want to. It won't change your results.

I like the other answers better, but my dumb answer is just that it's foolish to decide your first move based on what your rating is.

If you have seen a great game with 1. d4 and want to try it, then try it

If you have studied 1. d4 recently and want to practice it, then practice it

If you have a big tournament soon that you really want to do your best and you know 1. e4 better, then maybe work on that a little more and use it

It is all up to you.  There is no super-parent there to guide you so sometimes you have to be your own parent!  For example, some people are orphans, or their parents do not play chess very much, or they have never bothered to pay money regularly to a chess master for guidance; so at that point you get to just make it up on your own! The great thing about chess is that nobody dies, only chess pieces getting captured, but usually even the chess pieces are not destroyed.  So if you are worried about making decisions in your life, and want to start making more decisions on your own, then chess is a great place to start. 

go play d4. realize the grass isn't greener then come back to e4

I'm starting to play d4 and losing a lot but I think I get active positions. Let's see how it gets. 

I have never been able to get much of a positional advantage with white whistle, but it does seem to annoy my opponents.

To be honest, I don't think that the fact that you don't know how to play against certain openings justifies such switch. The choice of e4 over d4 should be based more on a playing style - i.e. what sort of positions you feel comfortable - whether you are more tactical/attacking player or technical/endgame one. I also switched to d4 because of exactly the same reasons as you are giving, although I was already around 1850 FIDE at that point, and now, at 2126 FIDE I'm switching back to e4, because I simply don't handle the positions after d4 well. So already for a long time I'm getting poorer results with white than with black (talking about OTB chess), despite the fact that I spend more time as white for opening preparation. Therefore before deciding to switch to d4, you should evaluate whether it fits your playing style. In the end, there is plenty of ways how to handle French and Caro-Kann without preventing them being played. I play French defense already for 15 years and still find it very hard to play against properly executed exchange variation with speedy c4. Same would go for many Caro-Kann players I think.

This is a decision that's very hard to make, because at my level we actually don't know what positions we handle better, I'm really in a deep sea right now.

David Gerrold once said that one should resolve a difficult decision by flipping a coin - if you don't like the coin's choice, then do the other thing. To me, both choices seem plausible, so it may be that it is just a matter of how you feel about it. For what it is worth, Queen's Gambit repertoire books seem to be longer than 1 e4 repertoire books.

For awhile, there seemed to be a 1 e4 series by Parimarjan Negi, but I wonder if the project has been abandoned. One of the books was 1.e4 vs. The French, Caro-Kann & Philidor.
http://www.jeremysilman.com/shop/pc/1-e4-vs-The-French-Caro-Kann-Philidor-76p3875.htm
http://www.qualitychess.co.uk/ebooks/GMRep-1e4-vol1-excerpt.pdf

Perhaps relatively short 1 e4 repertoires are more commonly published because, in part, beginners are commonly advised to use 1 e4.

My First Chess Opening Repertoire for White
https://www.newinchess.com/media/wysiwyg/product_pdf/9033.pdf

Every time one plays a game, one has to deal with the question of the first move. When considering a switch, it might make sense to consider whether or not there is a realisticc expectation of having less of an "uncomfortable" feeling.

Are you suggesting to prepare for a year before switching to d4? I would say that is a very cautious approach. There will always be holes in your opening knowledge, so if your too cautious and think switching requires a lot of preparation you might create hurdles for yourself that don't need to be there. I would recommend to maybe learn the first moves and names of the most common openings and maybe learn some key ideas, but that's it. That should take less than a day, maybe a couple of hours depending on how much time you want to spend, but even that might not be necessary.