I've gotten the impression that not all successful challenges are "accepted," but some are paired with other open seeks. ... ?
I once played a fellow, and having won, got a rematch request which I accepted. After the second game, a waited a short time and hit the rematch button. The same opponent came up, but I soon got a chat message which said "You again?". Evidently, he had posted another new challenge, and my "rematch" request paired us without waiting for his consent.
I have the min rating set to 1600, yet 1500 rated players accept my challenge.