That's a fair, but different, objection. The point of calling someone a "Mozart" in their field is not meant to be a disservice to anyone.
"Mozart" has become shorthand for, as you mention, child prodigies who remain prolific as adults. In the same way, "Einstein" is shorthand for someone who is a revolutionary genius.
Child prodigies as accomplished as Mozart? In mathemathics, how about Gauss? I'd say his accomplishments stand well against any prodigy in any other field. In chess, I wonder how many people forget that Kasparov was a prodigy and remained the #1 ranked player for over 20 years. Does anyone else in chess compare to that?
It's become a bad cliché to label anyone who is very talented in their field as "a Mozart". Mozart was truly a one-off. There is no Mozart of chess, football, painting etc etc.
Well, I disagree. Music, mathematics and chess are all 3 fields notable for child prodigies. I think it's arrogant to think that Mozart was so freakish of a prodigy that no other prodigy could match his accomplishments in other fields.