Engine-Match Analysis of FVF Games

I intend to post a summary of the most recent engine-match analysis of our completed VC games here. Games that timed out, finished within book, or were of a different variant (Chess960) are excluded.

*

1) 'Total' moves is the number made by FVF in the game

2) 'Book' moves are those available from most opening databases such as Explorer & are normally  excluded from analysis

3) 'Other' are those such as forced moves or moves in which there's a large difference between the engine's first & nth. choice. Any moves in which any of the engine choice moves exceed 2 pawns in strength are excluded, because a competent player would be expected to be able to see this for themselves

4) 'Net' moves is the number of qualifying moves used to calculate the T1, T2 & T3 figures

5) 1st, 2nd & 3rd give the counts of those moves we made that corresponded to the engine's assessed first, second & third best moves in the position. Sometimes we made moves that weren't in any of those categories, so the three counts won't always add up to the Net figure

6) T1, T2 & T3 represent the proportion of moves we made that correspond to the engine's best choices, as a percentage & are calculated like this:-

• T1 = (1st / Net) * 100%
• T2 = ((1st + 2nd) / Net) * 100%
• T3 = ((1st + 2nd + 3rd) / Net) * 100%

These are the T3 figures calculated for all our qualifying games but on a 20-game rolling basis:-

.

*

.

.

.

I'm going to post some T3 results for one or more 'trusted VC teams' here as soon as I've obtained them. That will be useful I think as a comparison with our own figures.

I've been asking around & MGleason (always helpful in these matters) has suggested one of his own groups, The Ultimate Training Center. He wrote this about it...

"The Ultimate Training Center is mostly pretty clean; we have had the occasional cheater sneak through, but most of them aren't involved in vote chess. We've got NMs @Impractical and @CedrHask involved (and @canwedoit has joined our most recent game). We don't have any FMs or above involved in vote chess, though"

"I would assume that appropriate benchmarks for vote chess would be a bit above the best plausible correspondence stats for the strongest player. A well-organised team involving 1-2 FMs and additional players a bit below that level could likely be a match for an IM, but probably not for a GM"

It's been quite a headache getting that data into some form that I could post & was readable, so I'm hoping it looks o/k for both of you?

If so, I'll just point out something that struck me as soon as I'd produced the T3 figures: While many of our games over the last two years exceed the benchmark by a worrying margin, the final two games are just within. This corresponds with the closing of Dalephilly's account, although that may simply be coincidental.

To remind you both, the usual benchmarks used for this kind of analysis is:-

• T1 = 60%
• T2 = 75%
• T3 = 85%

In the short term we still have a difficult issue to deal with & that's the problem of d-d.

Joe, I received a message from the admin of Last Rites a while ago, drawing our attention to the high T1/T2 results for that game (T2 = 92%!). The benchmark is usually set at 75%. They also examined our discussion in that game & came to the conclusion that d-d was probably using engine assistance. I pointed out that we'd had a banned player in that game as well (Dalephilly) but all of his comments had been erased of course.

I didn't think they were convinced so I asked Stephen for his considered opinion & he's of the view that d-d most probably is using assistance. My next task is to write to him to ask if he can explain the strange gap between his Daily & shorter time control games. I don't intend to make accusations just yet.

There's some useful additional information here in posts #11 & #12....

https://www.chess.com/clubs/forum/view/engine-use-in-vote-chess