Yep, I think people that don't consider the timer relevant are trying to blame the tool rather than looking at their weaknesses. If you can't spot simple tactics quickly you just can't play a good game of chess because all your calculation of lines will be slow. Once you have these tactics down you will be given more difficult ones and thus given more time.
E.g. at the 1500 level there are a lot of smothered mates. They rarely take me more than 10 seconds. If you're taking two minutes you may technically be getting it correct but you probably wouldn't see it in a game, because you can't afford to spend that amount of time looking at a single line unless you're almost certain there is something there, which you won't be if you don't know the pattern well enough.
Active practice sharpens skills, but neglect dulls one's weapons. Earlier in 2012, I was solving 50-100 tactics problems every week. Some were in books, such as Lev Alburt, Chess Training Pocket Book II or Encyclopedia of Chess Combinations. Most problems were on web sites devoted to tactics training. There were a few sessions on Chess.com.
During one of these session, I was presented with http://www.chess.com/tactics/server?id=146084.
I required a mere twelve seconds to execute all four moves of the combination.
This morning, rusty from two months of neglecting my training, the problem took me 26 seconds.
Without the pressure of the timer, my potential gain from active use of TT would be much reduced. Moreover, I might judge myself to be improving because I easily solved a problem, when in fact I have become weaker.