Hard to say, but Kasparov probably would have had the advantage.
Garry became much stronger in 1985, primarily by learning from Karpov in their matches.
Fischer once took a year off from competition, and was very "rusty".
In 1984, Bobby would have experienced serious "rust".
Whether or not Karpov would have beaten Fischer or not is an argument that will always stir emotions and facts alike. But a few have suggested that Fischer was so strong he would have even beaten Kasparov. Maybe if they were both in their prime – but that becomes even more hypothetical. What I wish to raise is: If Kasparov and Fischer had met in 1984, 85 (Fischer into his 40s and Kasparov coming into his prime) I recon Kasparov would have prevailed. Please advise.