Maybe it's cause the rook is really old. Queens, kings, and bishops generally live around the same age, which is rarely past 100 years. Pawns tend to not live as long due to poverty, and knights less, cause they are horses. But the rook lives for 1000s of years!
Maybe gravity overtook the rook over the years, making it the size it is today.
Interesting observation!