I think it just depends on your personality. Personally I would like to hear them, but I have friends that would probably like to just watch it. Its up to you on how you want to do it!
How useful is it to watch a video of a classic game when there's no commentary?

Try it out! https://youtu.be/Kbf0kGQkikc
No commentary but with captions as you proposed. Question is why people want to watch in the first place. If you want to learn then I think commentary is important. Want to think for yourself and just see the game play out, then no. Or if there is an entertainment angle on it like the video above.
Being a person who makes YouTube videos, and a beginner at playing serious chess for about a year now, I'm always trying to figure out what would be needed in a video. I don't really think commentary is possible. I'd just be reading off the Wikipedia entry for a particular famous game. And ... I don't have a microphone.
Is there value in watching a game with just the move strength icons on this site providing the context? I've heard it said, "I don't want to watch other people play." Is that a universal feeling, or do some people like to watch chess games?
I was thinking one way of providing more context would be to add the moves in as closed captions.