The pairing system and the fair-play mechanism is supposed to cause those that disconnect to be paired with each other as a preference. Don't know if it completely prevents them from getting paired outside that pool.
Maybe he is manually picking up your seeks?
They need to add a Disconnectors list. If you disconnect against the same opponent twice in an hour, against different opponents 3 times in an hour, 4 times in a day, 5 times in a month, or 6 times in a year, you should get permanently added to the list, and if on the list, and you disconnect or abort, it's a forfeit loss, including loss of rating points.
I put in for a rated game, it pairs me against Oberon80 - he immediately aborts. I put in for another 5 minute game, get the same opponent, and the moron aborts again.
That's another thing that should be added. If you abort, you can't match for 10 minutes. Multiple aborts by the same opponent back to back is obvious abuse.