Well, you have to keep in mind that rook endings are particularly drawish in nature, especially with four rooks on the board, and an extra pawn is often insufficient to win.
This being said, I think you played fine until your opponent until 35...b6 — that's the moment when an alarm should've gone off in your head. Black has made a hole out of the c6 square, all you have to do now is invade it (or at least threaten to) with your king.
I've been struggling a lot lately with endgames. My opponent always seems to create a fortress which I can't break through, and I end up becoming desperate and trying to force something to happen, which leads to me blundering. Take this example:
How could I have played this better? Any advice for me in general?