Does Colle have the same idea with Semi Slav?

No, even the Colle has more aggressive plans than playing h3 to stop Bg4.

I suggest you look at a book or video on the Colle System. 

One can argue that certain lines of the Colle are similar to the Semi-Slav, particularly the Phoenix Attack (9.b4 instead of 9.e4 in the Koltanowski).

However, just because the London appears to look like a Slav with an extra move and a Colle looks like a Semi-Slav with an extra move, that doesn't mean the ideas are the same at all.  Just chucking in something stupid like h3 or a3 and saying you are playing Black with a bonus move is idiotic.  There are things that can be done in the Colle that can't be done in the Semi-Slav, or those that can be done in the London may not be doable in the Slav.

Think about how many other openings work for White but not Black.  The change in tempo changes the game completely.  White doesn't just mimic Black's normal play with an extra move:

1) King's Indian Attack - First off, White often plays h4 instead of f4.  In the King's Indian Defense, f5 almost always takes center stage and has a much higher priority than ...h5.  Also, if White plays the KIA via 1.Nf3, and Black plays a "King's Indian Reversed", he is ok playing the Reversed Fianchetto KID or the Reversed Saemisch KID, but he doesn't have the time to play the Reversed Classical.  Black will get smashed, despite the fact that the Classical is the most respected line for White in the King's Indian Defense

2) English Opening - After 1...e5, you have a reversed Sicilian.  Let's say White plays g3, Bg2, Nc3, Nf3, d3, O-O, etc, in some order, and Black plays Nf6, Nc6, d5, and N captures after a trade on d5, you may be looking at a Reversed Dragon.  Black can play a reversed Classical Dragon and be perfectly fine with full equality, but that extra tempo for White does kill all ideas at the Yugoslav Attack, and if Black tries to play that down a move, he will get smashed if White knows what he's doing.

So while the Colle might look like a Semi-Slav, only the Phoenix Attack even remotely truly resembles a Semi-Slav, and even that it's not Semi-Slav strategy in its purest form.