As always: it depends on the position. All rules have exceptions. Against the Sicilian Defence 1 e4 c5 for example white will almost always move his f-pawn sooner or later, either to f4 or to f3. Also against the Pirc Defence 1 e4 d6 white will often play either f4 or f3 later. The Dutch Defence 1 d4 f5 is risky but playable and so is the Bird Opening 1 f4 for white. King's Gambit 1 e4 e5 2 f4 is risky but playable. So the advice not to play the f-pawn has many many exceptions.
Why do GMs push f4/f5 so unapologetically?
If you push your f-pawn with White or Black, your king will still have two pawns in front of him to shield him. So it's possible to tuck your king away in the corner on h1/h8, if need be, and use the f-pawn to attack, open files, activate pieces. As a "battering ram" as GM Naroditsky calls it sometimes.

I'm not aware of a rule for beginners that says don't move the f pawn when queens are on the board. The rule for beginners is usually not to move the f pawn in the opening.
Sure there are exceptions, like literally on the first move, 1.f4, is a perfectly ok opening. It has some practical issues so it's not popular, but that's starting to get off topic. Mostly I should say that playing f4 after your development is complete is not nearly as dangerous as playing it very early in the game.
After an opening ends, generally you seek play in the area of the board (queenside, center, or kingside) where you have a natural advantage... so in the area where you have more space, or more active non-pawns (active pieces). This space condition is usually taught to new players as "attack in the direction your pawns point." For example look at the pawn structure below:
-
-
The central pawn chain points to the kingside, so in many cases it would be reasonable to consider playing f4 in the middlegame.
Let's make it even better with the diagram below
-
-
Now f4 is a pawn break against a central pawn. This kind of move is always worth considering. If it hasn't already happened yet, then in many middlegames a big goal (for both players) is to play one of the pawn breaks against the opponent's central pawn. In the diagram above white's two pawn breaks against black's center pawn are d4 and f4 while black's are d5 and f5.
Remember the other condition besides where the pawn "point?" It was to seek play where you have more active pieces.
-
-
In the above diagram we see both white's knights are on the kingside. Both white's bishops and queen have diagonals to the kingside, and if f4 is played, even white's f1 rook can join. White really has no option but to seek play on the kingside, and f4 is a pawn break that will help.
Oh, and just a quick reminder about pawn breaks. Pawn breaks are useful because they open lines... this is a fancy way of saying they remove pawns from diagonals/ranks/files. Imagine a game where it was illegal to capture pawns... well it would be a boring draw because pawns block up everything. Pawn breaks remove the blockades, and allow the other pieces to gain an advantage in that area.

As always: it depends on the position. All rules have exceptions. Against the Sicilian Defence 1 e4 c5 for example white will almost always move his f-pawn sooner or later, either to f4 or to f3. Also against the Pirc Defence 1 e4 d6 white will often play either f4 or f3 later. The Dutch Defence 1 d4 f5 is risky but playable and so is the Bird Opening 1 f4 for white. King's Gambit 1 e4 e5 2 f4 is risky but playable. So the advice not to play the f-pawn has many many exceptions.
Let me use this for some quick examples.
What is the biggest weakness of f4 after 0-0? Well I guess it depends on the position, but to me the most obvious and glaring problem is the a7-g1 diagonal. I'll put a bishop on that diagonal to make it more obvious to the OP below
-
-
Now notice some of the example openings talked about in the post I quoted... the f pawn often moves in the Sicilian? Also the Pirc? Ok... where is black's bishop in those middlegames? The answer is it has often chosen a different diagonal or is stuck behind its own pawns.
-
-
-
Now let's look at the king's gambit which was listed as risky...
-
-
Boom, instantly on the diagonal. In fact 2...Bc5 is a good line, and the line I prefer playing when playing against the king's gambit.
I could go on, but I think I made my point, and my posts have been long enough as it is

What they teach beginners are just guidelines. They aren't strict rules that are meant to be followed at every situation. As one gets better at the game , one gets better at identifying situations where those guidelines can be violated.

Yes, just guidelines, the only thing big bad thing about moving the F pawn is a possibility for quick queen attacks, and that is something that beginners cannot defend easily against and stop. Obviously, masters can defend easily, (Edward Lasker double bishop sac) and use the F pawn to their advantage.

llama did a good job explaining the logic behind a common f-pawn push.
It's also worth noting that Naroditsky plays the King's Indian Defense (and the King's Indian Attack) a lot. It's a regular part of his repertoire. And in both the KID and the KIA, the f-pawn push is a thematic move.
It's true, though, that pushing the f-pawn can create some weaknesses in one's own camp. Pawns can't move backwards, so once the f-pawn moves, any weak spots created from its advance could become problems in the future.
I remember GM Simon Williams playing against GM Hikaru Nakamura on stream, and Williams was saying, in one game, that he wished he could move his f-pawn backward, because Nakamura was really attacking the holes that his f-pawn advance left behind.
So, yes, pushing the f-pawn can sometimes be more trouble than it's worth. Depends on a lot of factors.

As for KID / KIA, that gets more complicated because there are many setups, but trying to keep it simple, I'll say that in the early middlegame, direct defense is almost always a very poor idea. If your opponent is gearing up for action in one area (kingside, center, queenside) using the intrinsic advantages in the position (space and active pieces) then objectively speaking direct defense is doomed to fail, so you're required to counter in a different area. Sometimes this means pushing pawns on the kingside even if your king is castled there.
Aren't beginners told it's dangerous to open up your king? Well yes, but the deeper rule is that open lines favor the side with active pieces. Opening lines in front of your own king can be advantageous as long as that's the area of the board where your active pieces / space advantages are.
That doesn't mean it's not double edged, or that there are exceptions. Chess is tough, but those are the basic ideas.
As a beginner, I was always told not to move my f pawn while queens were still on the board, as it is too weakening. However, after watching Daniel Naroditsky's speedrun for the past few months, I've noticed that the f4 (or f5 if you're black) break is played very commonly after short castle, regardless of pieces on the board. Could a stronger player explain why f4 is safe to play in certain situations, even with the queens on the board?