How do I look at my win rates with/against certain openings?

Talking about this.

Is it a premium thing?

No. I think this is taken from the chess.com Opening Explorer. https://www.chess.com/explorer There are some general things under the "Stats" section, but chess.com's Insights feature offers way more statistics like this if you do have a diamond membership.

With Diamond membership I usually just check the Insights page, but it's good that all members can analyze their opening stats properly on the Explorer page. All players need to know which openings they are winning/losing with.

There's also a way to get basically the Insights page, but without chess.com premium. I mention it in this video with the link in the description. You just have to wait a while for it to run and collect your stats.