Chessbase is excellent for manipulating large game collections. In Chessbase, right-click on a database and select Tools | Find Double Games. Alternatively there's the freeware program SCID. In that, select File | Maintenance | Delete Twin Games. In either case, be sure to make a backup before performing this operation.
Eliminating duplicate games in a Database


There is a YouTube video that demonstrates how to do this in SCID vs PC:
https://www.youtube.com/watch?v=Zwy-pE54V9w
Although this video is a bit slow and annoying it actually explains things extremely well. @lrsalokhe is correct and SCID opens PGN databases in read-only mode, what you do is create a new DB in SCID and then import your PGN database(s) into that - once there, you can delete duplicates, create indexes, modify annotations etc.
This is all explained in the video.
If I have a large collection of games in a database in PGN format, how can I eliminate duplicate games. I see that my db has plenty. I have Fritz 11 among other programs but none seem to be able to do this. Any help appreciated.