If your pieces are centralized and active entering a middlegame then the only way you can improve them further is by... playing good chess. So this is a very difficult question to answer because it depends on the position.
Generally speaking though if you understand the middlegame themes that come out of the openings you play and even more generally if you're familiar with the common plans dictated by the pawn structure, then you'll be able to find reasonable candidate moves for your pieces. i.e. study master games / read middlegame or strategy books.
If it's a very closed position then both sides are usually working towards a pawn break. Maybe not right away but it will be on their mind for the future. When a pawn break happens and lines are opened, the side whose pieces stand better will have the advantage. Usually "better" means they're not in eachother's way, and they're ready to use the newly opened lines. So in these cases you'll usually prepare your pieces on the side (or center) of the board where your useful pawn break is.
I'm having some real problems playing out the midgame right after the opening. Sometimes it's just this extremely closed position where both sides have all pieces basically pointing at the middle and there's just no good move.
Are there any principles for good improving moves that I can follow? I watched a few videos on Youtube, but I can never pinpoint why the move is good, just that the IM/FM thinks that it makes the position easier to play.