The cheating scandal of the 3 French players continues...

Sort:
Avatar of ChessMarkstheSpot

  Good morning everyone!

   In a recent posting on Chessbase.com, the saga of the 3 French players accused of cheating during the Olympiad continues. They have all been suspended, although one is playing in the current European Championship event in France (doesn't suspended mean NOT playing?) and the other two players have demanded additional security.

   A petition was sent around to professional players demanding that more steps be taken to make sure players do not cheat during tourneys. The article is long but very well written and it is a serious matter causing a black cloud over this tournament. The full article can be found here.

   -Mark

Avatar of Orvall

As to the cheater playing in the European Championship, I heard 2 theories:

1) His sentence (ban) only starts 10 day after it's official, so he could still participate.

2) He appealed against the decision.

Worst of all: the French chess federation (who started the law suit) is paying for his hotel expenses during the tournament. Ironic, not?

Avatar of Orvall

Well there's is ample evidence: "200 messages exchanged during the games, and only during the games". Those messages were coded moves - moves that were made on the board.

Losing against a 2300 at the Olympiad could be explained as Feller who trusted the signs of his team captain too much (or misinterpreting them).

Furthermore both Vachier-Lagrave and Edouard learned of this, not via the Federation, but through Arnaud Hauchard’s personal confession to them.

The fact that Feller's defence was based on technical stuff, like invoices that couldn't be used in court is not denying the facts.

I hope his good prestation at Aix is due to his qualities, but an elaborate cheating scheme like used during the Olympiad is very hard to spot. The cheating would never have been found when the man at home (International Master Cyril Marzolo) would have been more careful.

The only way to prevent this has already been proposed to FIDE: a 15 minute delay on live broadcasts. (like mentioned in Marks link)

Avatar of SteveCollyer
Here are the non-database engine match rate results for all of Sebastian Feller's games at the 2010 Chess Olympiad.

The analysis was done under the following conditions:

4 x AMD Phenom 2.30Ghz 4 GB RAM
Houdini 1.5 x64 Hash:640 MB Time:45s Fixed Depth:20 ply

{ Batch Summary }

http://www.chessgames.com/perl/chessgame?gid=1592256
{ Sebenik, Matej (Games: 1) }
{ Top 1 Match: 13/27 ( 48.1% ) Opponents: 21/27 ( 77.8% )
{ Top 2 Match: 19/27 ( 70.4% ) Opponents: 24/27 ( 88.9% )
{ Top 3 Match: 22/27 ( 81.5% ) Opponents: 25/27 ( 92.6% )
{ Top 4 Match: 24/27 ( 88.9% ) Opponents: 25/27 ( 92.6% )

http://www.chessgames.com/perl/chessgame?gid=1592804
{ Victor Mikhalevski (Games: 1) }
{ Top 1 Match: 20/45 ( 44.4% ) Opponents: 26/45 ( 57.8% )
{ Top 2 Match: 28/45 ( 62.2% ) Opponents: 34/45 ( 75.6% )
{ Top 3 Match: 33/45 ( 73.3% ) Opponents: 36/45 ( 80.0% )
{ Top 4 Match: 33/45 ( 73.3% ) Opponents: 39/45 ( 86.7% )

http://www.chessgames.com/perl/chessgame?gid=1593327
{ Markus, Robert (Games: 1) }
{ Top 1 Match: 13/30 ( 43.3% ) Opponents: 18/31 ( 58.1% )
{ Top 2 Match: 19/30 ( 63.3% ) Opponents: 24/31 ( 77.4% )
{ Top 3 Match: 25/30 ( 83.3% ) Opponents: 27/31 ( 87.1% )
{ Top 4 Match: 27/30 ( 90.0% ) Opponents: 29/31 ( 93.5% )

http://www.chessgames.com/perl/chessgame?gid=1593823
{ Howell, David W L (Games: 1) }
{ Top 1 Match: 20/34 ( 58.8% ) Opponents: 25/35 ( 71.4% )
{ Top 2 Match: 28/34 ( 82.4% ) Opponents: 33/35 ( 94.3% )
{ Top 3 Match: 32/34 ( 94.1% ) Opponents: 33/35 ( 94.3% )
{ Top 4 Match: 32/34 ( 94.1% ) Opponents: 35/35 ( 100.0% )

http://www.chessgames.com/perl/chessgame?gid=1594445
{ Kreisl, Robert (Games: 1) }
{ Top 1 Match: 31/48 ( 64.6% ) Opponents: 34/48 ( 70.8% )
{ Top 2 Match: 40/48 ( 83.3% ) Opponents: 39/48 ( 81.3% )
{ Top 3 Match: 44/48 ( 91.7% ) Opponents: 42/48 ( 87.5% )
{ Top 4 Match: 46/48 ( 95.8% ) Opponents: 44/48 ( 91.7% )

http://www.chessgames.com/perl/chessgame?gid=1594941
{ Alsina Leal, Daniel (Games: 1) }
{ Top 1 Match: 8/18 ( 44.4% ) Opponents: 14/19 ( 73.7% )
{ Top 2 Match: 10/18 ( 55.6% ) Opponents: 16/19 ( 84.2% )
{ Top 3 Match: 13/18 ( 72.2% ) Opponents: 18/19 ( 94.7% )
{ Top 4 Match: 16/18 ( 88.9% ) Opponents: 18/19 ( 94.7% )

http://www.chessgames.com/perl/chessgame?gid=1595532
{ Timofeev, Artyom (Games: 1) }
{ Top 1 Match: 18/29 ( 62.1% ) Opponents: 22/29 ( 75.9% )
{ Top 2 Match: 23/29 ( 79.3% ) Opponents: 25/29 ( 86.2% )
{ Top 3 Match: 23/29 ( 79.3% ) Opponents: 26/29 ( 89.7% )
{ Top 4 Match: 25/29 ( 86.2% ) Opponents: 27/29 ( 93.1% )

http://www.chessgames.com/perl/chessgame?gid=1596036
{ Gelashvili, Tamaz (Games: 1) }
{ Top 1 Match: 9/20 ( 45.0% ) Opponents: 18/20 ( 90.0% )
{ Top 2 Match: 13/20 ( 65.0% ) Opponents: 20/20 ( 100.0% )
{ Top 3 Match: 16/20 ( 80.0% ) Opponents: 20/20 ( 100.0% )
{ Top 4 Match: 16/20 ( 80.0% ) Opponents: 20/20 ( 100.0% )

http://www.chessgames.com/perl/chessgame?gid=1596544
{ Efimenko, Zahar (Games: 1) }
{ Top 1 Match: 6/10 ( 60.0% ) Opponents: 8/10 ( 80.0% )
{ Top 2 Match: 8/10 ( 80.0% ) Opponents: 9/10 ( 90.0% )
{ Top 3 Match: 9/10 ( 90.0% ) Opponents: 10/10 ( 100.0% )
{ Top 4 Match: 9/10 ( 90.0% ) Opponents: 10/10 ( 100.0% )

{ Feller, Sebastien (Games: 9) }
{ Top 1 Match: 186/264 ( 70.5% ) Opponents: 138/261 ( 52.9% )
{ Top 2 Match: 224/264 ( 84.8% ) Opponents: 188/261 ( 72.0% )
{ Top 3 Match: 237/264 ( 89.8% ) Opponents: 217/261 ( 83.1% )
{ Top 4 Match: 247/264 ( 93.6% ) Opponents: 228/261 ( 87.4% )

{ All Players }
{ Top 1 Match: 324/525 ( 61.7% )
{ Top 2 Match: 412/525 ( 78.5% )
{ Top 3 Match: 454/525 ( 86.5% )
{ Top 4 Match: 475/525 ( 90.5% )
 
It means that Feller played far more engine-like chess than his opponents, especially with regards to top 1 & 2 choice moves in this sample.

However, only 9 games were analysed & some of these had very few non-database moves so the data is hardly conclusive when used with this methodology.

If Feller had achieved these engine match rate %'s from 20 objectively chosen OTB games, all with 20+ non-database moves, then he would have massively out-performed any other OTB GM (and indeed pre-computer era CC World Championship finalist) who has been tested so far using this technique.
 
Avatar of goldendog

To provide a comparsion, here's what other current top players score using the same method of analysis.

{ Anand, Viswanathan (Games: 10) } Morelia 2007
{ Top 1 Match: 163/274 ( 59.5% ) 
{ Top 2 Match: 203/274 ( 74.1% ) 
{ Top 3 Match: 228/274 ( 83.2% ) 
{ Top 4 Match: 240/274 ( 87.6% ) 

{ Carlsen, Magnus (Games: 9) } Morelia 2007
{ Top 1 Match: 186/325 ( 57.2% ) 
{ Top 2 Match: 245/325 ( 75.4% ) 
{ Top 3 Match: 279/325 ( 85.9% ) 
{ Top 4 Match: 293/325 ( 90.2% ) 

{ Carlsen, Magnus (Games: 9) } Nanjing 2009
{ Top 1 Match: 200/335 ( 59.7% ) 
{ Top 2 Match: 256/335 ( 76.4% ) 
{ Top 3 Match: 283/335 ( 84.5% ) 
{ Top 4 Match: 298/335 ( 89.0% ) 

All these are small game batches. Take them and be entertained and have your curiosity piqued.

Avatar of TheGrobe

Small sample size notwithstanding, coupled with the evidence Orvall referenced it looks pretty damning to me.

Avatar of SteveCollyer

Noticed I did fixed depth =20 ply, sweetie? Kiss

By the way, I haven't analysed any other games from Feller.  The small sample size of non-db moves from the 9 games I did do took between 9 & 10 hours, and that was with the relatively fast Houdini engine!

I'm not sure what you don't understand about the cumulative non-db figures.

Opponents played 27+45+30+34+48+18+29+20+10 = 261 non db moves from the 9 games

Feller played 27+45+31+35+48+19+29+20+10 = 264 non db moves

You ask how many moves were counted & I thought that this was pretty straightforward.  Both the Feller non-db moves 264 + all moves from his opponents 261 appear in the { All Players } stats = total 525

Avatar of wilford-n
Fezzik wrote:I would bother to challenge Steve Collyer's data, but the last time I did, he accused me of cheating, of not understanding what I was talking about, and several other things that got deleted by moderators. Knowing that the f-bomb doesn't always get deleted in these threads, that's saying something.

I'm not sure what about the data you would "bother to challenge"; it seems he made his methodology pretty clear to me. He identified the processor, the engine, and the analysis depth, so unless you replicated his experiment and got different results, I fail to see any problem with the data... and if you have, I'd like to see your data. Now you might wish to challenge his interpretation of the data, but when compounded with other evidence, I'd say that the cheating has been virtually proven.

Avatar of wilford-n
Fezzik wrote:
If we were to find that his moves matched a computer engine (not just Houdini, but, say Firebird, which is apparently the most popular among GMs), with a +90% top 1 match-up rate over more than 200 moves, I would be more inclined to believe it. But again, the match-up method wasn't used by the FFE. Instead, they relied on testimony. The match-up method wouldn't stand up in a court of law in part because there is no consensus on what "massively outperforming an OTB GM" really means.

For me, the question is why is there not an even stronger correlation between computer and human moves?  If Feller was getting the moves, there should be close to a 100% match-up rate with the engine they were using. Since we have confessions, we can even find out which engine was purportedly used and find out how well they match up!


The first point you make is about the small sample size, and Collyer does mention that shortcoming in the data. So yes, a larger sample size would be nice, but we don't have that.

To say that you need to see 90% top-one matches to be "inclined" to believe it, or that "there should be close to a 100% match" is incredibly naive. Feller is still a GM, after all, and computers don't always make the best move in a given situation. They don't make long-range plans, whereas Feller can and does. But what computers are really, really good at is avoiding tactical errors, and taking advantage of your opponent's tactical errors. So the potential cheater is fed a list of computer-generated stong tactical moves, and then selects whichever best fits into his medium- and long-range strategies, which is not always the computer's first-choice. The result is that Feller is playing what Kasparov calls "advanced chess," against opponents playing a traditional OTB game.

Finally, you mention that we do indeed have confessions. How much more evidence do you need?

Avatar of TheOldReb

If there are confessions I certainly do not need anymore evidence. I really don't understand why anyone would ? 

Avatar of TheGrobe

Well, when it comes to evidence of cheating, Fezzik is a contrarian by nature, with the apparent desire to set the standard of proof so high as to be unattainable

Avatar of SteveCollyer

Fezzik simply cannot read a single one of my posts without taking my words out of context and/or attempting to twist them for his own strange needs. 

With your track record, I should imagine any results you produce would need to be thoroughly scrutinised for evidence of manipulation.

On top of that, you can't seem to understand a simplistic results output log.  Or add up.

Maybe engine detection analysis isn't for you, Dan?

Avatar of SteveCollyer
Fezzik wrote:

So in 9 game, Feller played a grand total of 17 moves that were more "computer-like" than his opponents. (89.8% of 261 games would be just over 234 moves.)

No, but as per my results

{ Feller, Sebastien (Games: 9) }
{ Top 1 Match: 186/264 ( 70.5% ) Opponents: 138/261 ( 52.9% )
{ Top 2 Match: 224/264 ( 84.8% ) Opponents: 188/261 ( 72.0% )
{ Top 3 Match: 237/264 ( 89.8% ) Opponents: 217/261 ( 83.1% )
{ Top 4 Match: 247/264 ( 93.6% ) Opponents: 228/261 ( 87.4% )

 

237/264 x 100 = 89.77 = 89.8%

Steve Collyer suggests that he must be cheating because of this huge difference.

I said

"However, only 9 games were analysed & some of these had very few non-database moves so the data is hardly conclusive when used with this methodology"
Did you see that..?
Jolly good!

 

In the Efimenko game, it was a difference of precisely one move.

In the Kreisl game, he was outperformed by a relatively unknown player.

As you know, match rates from individual games can be very high indeed - even OTB where no foul-play is suspected.  You know full well from your experience in the cheating forum that the more non-database moves in the sample the better with this methodology.

And all this amounts to what Steve Collyer says, Feller "massively outperformed any other OTB GM ...who has so far been tested using this technique." I'd like to see how Karjakin and others scored at the same event. And how Feller scored at the European Championship. 

...If there were 20 games each with 20+ non-database moves, which typically yields a sample of 600-800 non db moves, as I thought I made clear (see my quote above) but you're already fully aware of the methodology aren't you, devious Dan?

If we were to find that his moves matched a computer engine (not just Houdini, but, say Firebird, which is apparently the most popular among GMs),

Houdini is incredibly popular & gaining ground all the time with top players, just PM Robert Houdart & he'll tell you many use it, but whatever makes you happy...

with a +90% top 1 match-up rate over more than 200 moves, I would be more inclined to believe it. But again, the match-up method wasn't used by the FFE. Instead, they relied on testimony. The match-up method wouldn't stand up in a court of law in part because there is no consensus on what "massively outperforming an OTB GM" really means.

I wasn't about to send the results to court - I was merely curious to see how this small sample of games would match using a method I'm familiar with & that is all.  I thought others might be interested to see them, too.

For me, the question is why is there not an even stronger correlation between computer and human moves?  If Feller was getting the moves, there should be close to a 100% match-up rate with the engine they were using.

This just highlights your total lack of understanding.  You're way out of your depth - that much is clear.

Why should there be anything like a 100% correlation?

Different engine

Different system

Different hash table size

Different analysis time

Different analysis depth

Possible multi-tasking (or lack thereof) & unavoidable CPU or RAM fluctuations can alter results even on the same systems using the exact same criteria, giving marginally different results for the same batch of games.

Since we have confessions, we can even find out which engine was purportedly used and find out how well they match up!

One of the few parts of your post that makes sense.  Well done!


Avatar of wilford-n
Fezzik wrote:

Reb, the confessions are for two others, not Feller. That's part of why Feller was allowed to play in the European Championship, and part of why other GMs have come to his defense.


Fezzik, I must confess that I have not seen the contents of these confessions. Do you have a link? I'd like to read about that.

Avatar of Here_Is_Plenty

I thought I was in a chess forum.  Appears I am in some sort of advanced Statistics class.  How do I escape?

Avatar of bondocel

Steve: you could use Firebird for the games against Howell, Efimenko, Timofeev and Gelashvilli :)

Avatar of TheGrobe
El_Senior wrote:
TheGrobe wrote:

Well, when it comes to evidence of cheating, Fezzik is a contrarian by nature, with the apparent desire to set the standard of proof so high as to be unattainable


Guilty until proven innocent, then? I don't agree.

If 100% of the accused are guilty, there's no need for any debate about methodology. Problem solved.

IMHO you've got to have some safeguards in place to protect innocent/legit players, therefore...the detection standards MUST be high.

Accuracy is important. Otherwise it's just paranoia and witch hunting.


I'll be the first to agree, but your definition of "beyond a reasonable doubt" has to be, well, reasonable. 

Avatar of ozzie_c_cobblepot

Am I crazy, or do the numbers not add up?

For the opponents, I hand-counted in the top 4 matchup that 17 moves did not match. But the cumulative stats do not show this. What am I missing?

Because looking on a game-by-game basis, it looks like the opponents did much better than he did.

Avatar of VLaurenT

According to French GM Laurent Fressinet, the engine to use for testing is Firebird, not Houdini, nor Rybka.

Avatar of bondocel
ozzie_c_cobblepot wrote:

Am I crazy, or do the numbers not add up?

For the opponents, I hand-counted in the top 4 matchup that 17 moves did not match. But the cumulative stats do not show this. What am I missing?

Because looking on a game-by-game basis, it looks like the opponents did much better than he did.


Yes, that analysis looks totally screwed-up. Apparently in the games I mentioned above Mr. Feller matches Firebird quite consistently. In a game he follows Firebird only until he gets to a -5 advantage. I guess that the rest was too easy :)