Proctor Eliminates Most Cheating in Prize Events

At the beginning of September, we made Proctor (our Fair Play monitoring program) mandatory for all players participating in prize events. Since its debut, we are excited to announce that we've had significantly fewer instances of engine cheating in prize events. It's an amazing development, and the result of a long journey to safeguard the integrity of our events. Let's take a moment to examine how we got here.

What Is Proctor?

Proctor is an app that wraps Chess.com in a secure browser experience. Through Proctor, we can observe players through front and back cameras, listen to their audio, monitor their desktops, and see all applications they are running on their computers. Proctor also establishes a private two-way communication channel between players and Chess.com staff who are monitoring the tournament.

How Does Fair Play Monitoring Actually Work?

Real-time Fair Play monitoring has always been integral to overseeing prize events. This monitoring entails identifying overperformance—seeing who is in the midst of an incredible event—and validating their physical/virtual setup to ensure they're not receiving assistance. For years, we proctored players using Zoom, which meant we operated under meaningful constraints.

An event like Titled Tuesday has hundreds of players, and while we ideally wanted to have every player in a call with a two-camera setup, that wasn't plausible using Zoom. In Zoom, breakout rooms are finite, so players would have to be in virtual rooms with their peers. Despite our best efforts, noise and disruptions from other players could intermittently distract someone locked in their own game. Any requests made by Chess.com staff in a call (to adjust a camera, conduct a room sweep, or screenshare) would ultimately be administered in front of a small audience, which was less than ideal. Last but not least, we'd have to ask players to open up the task manager to verify there weren't any nefarious programs running during the event.  

Given our Fair Play operating constraints using Zoom, we initially had a system in place where we had a percentage of the overall field playing on camera. These players would typically be pulled into calls at event breaks. Recognizing this could interrupt someone who was legitimately having the tournament of their life, we ultimately evolved toward requesting overperforming players to join Fair Play calls days before an event began. This system was smoother, and cheating in prize events had greatly reduced. Ultimately, players were being caught and closed for cheating or staying away from prize events altogether because of the more stringent Fair Play requirements. Still, it did not address the underlying issue that the majority of players were not playing prize events under the same conditions. 

We recognized that a more just system would be different, so we charted a course to build something every player could use. Proctor is different for several reasons. Firstly, each player is essentially in their own private breakout room, so there are no distractions from other players. This means players can play music and be as audible or casual as they want while they play, an added benefit for streamers/content creators. Secondly, we can see the systems a player runs in the background without having to explicitly request to see the task manager. This means that players can spend more time focusing on their games rather than on everything else. Thirdly, if a player's camera setup has an issue, troubleshooting is much easier because an unambiguous pop-up appears on screen, informing the player what to do to make corrections. Lastly, and most importantly, everyone is playing under the same conditions, so a player can rest assured that anyone they're playing against is also in Proctor. 

Proctor By The Numbers

Since Proctor became mandatory for prize events, we have closed a total of six accounts on the basis of Fair Play violations we identified while players were using the software, typically around account impersonation, and human assistance (e.g. from a nearby peer weighing in on a position during a tournament game). In every one of these six cases, it was an open-and-shut case that did not require statistical or expert review because we had video evidence of clear violations of our event rules. Most importantly, the type of cheating that requires extensive investigation—off-camera overperformances suggesting outside assistance—has nearly vanished from prize events. Our assessment is that Fair Play violations in prize events are now both exceedingly rare and (when they do occur) unmistakable.

Below is a short summary of the titled Fair Play closures made since June 1 of this year, before and after we required all players in prize events to use Proctor. We draw particular attention to the "closures in prize events," which are reduced by 50% due to Proctor. Seven of the closures in prize events were in events that did not use Proctor—either Bullet Brawl or the Comet Open. Note that we have made more titled closures in pool play. This doesn't indicate an increase in cheating in pool play; rather, it reflects increased bandwidth from our team to review pool play cases.

In summary, since June, we have observed one instance of engine cheating on Proctor, which occurred before the full rollout on September 2. We've also had a 50% decrease in prize event closures, and Proctor has massively reduced engine use in prize events.

Proctor is not yet a perfect product, but we are working to ensure that it works well for all players. We have received feedback and concerns about lag and VPNs, and in most cases have identified solutions that have improved the player experience. We remain readily available to actually meet with players individually and help them with these tech concerns because we believe in the trade-off—relative inconvenience has given way to a clean prize event experience. 

The real existence of online cheating, and separately, paranoia around the possibility of online cheating, has been an existential threat to online prize events in the last decade. We believe these results offer a light at the end of the tunnel, and we are excited about the next stage of online prize events.

IM Kassa Korley

Senior Fair Play & Communications Advisor