Databases - what is allowed?

I think that the line of reasoning represents a very slippery slope.  I think we've all agreed that dynamic assistance -- any tools or outside advice based on forward looking analysis on a specific position -- are offside and this is clear in the rules of the site.  When it comes to static tools (excepting endgame table-bases of course, as they are exhaustive), however, it is clear that it is not agreed where (if at all) the line should be drawn:

• Engine vs. engine databases
  This is the original topic of this thread so I won't rehash everything that's already been said.
  
• Master game databases
  By the line of reasoning espoused in the argument against Engine vs. Engine databases this is basically equivalent to getting outside assistance help from master level players.  Should we ban these?  This includes the game explorer.
  
• Opening ideas that have been developed with the assistance of engines
  Are these forever tainted by their association with engines?  I'm not an expert on the subject, but I understand that a number of opening lines have been discovered and previously accepted lines refuted since chess engines came into the scene and that many of these improvements have even been adopted as new main-lines.  What's worse, is that the engine versus engine database methodology that prompted this thread is one of the means by which they were discovered, and by master level players in many cases to boot. Do we have to discard them altogether?  Is there a statute of limitations after which an engine novelty is no longer a cheat?  If so what are the criteria -- introduction at master level play?  Elapsed time?  Inclusion in the chess.com games explorer?
  
• Opening books and other non-database resources
  Extending the line of reasoning that sees master versus master databases as being akin to master level player assistance in the same way that engine versus engine databases are akin to engine assistance, opening books and other non-database resources can also be construed as master (or at least expert) assistance, and if they include any engine novelties we'll of course have to take that into consideration as well.

Perhaps someone can help us parse this confusing landscape of outside influence in chess in order to determine the best course to either unwind it altogether, or find the appropriate place to draw the line between the appropriate and inappropriate influence of outside resources on gameplay here at chess.com.  I thought the static versus dynamic or exhaustive resource distinction that is in use here at chess.com was a very logical place to draw that line -- I'd like to better understand why it's not and to hear exactly where it should be drawn.

Very good summary Grobe!

I think that books/master databases are OK. But databases from ICCF/cyborgs/engine-vs-engine are not. For instance, look at the games from post 3 in this topic:

http://www.chess.com/forum/view/chess-openings/budapest-with-nc3

It is very likely that, in a gambit like that, a human might not stand a chance against such a database.

So, the question is: who is winning a chess game? The better player or the person who has more time to gather resources?

I have one remark though: we can't label opening ideas discovered by engines as bad altogether. What I label as bad are opening ideas with long engine lines, found only in cyborg games. Not even in OTB 2700+ games.

Ah, a funny thing. Shirov once said that, when playing a certain line in Marshall he forgot the theoretical recommendation. He said he was very worried, but luckily for him his opponent did not choose the best continuation. If a guy like Shirov is worried he does not remember the theoretical line, then what chance I would have to play against someone who has all these theoretical lines in his database??

Richie&Oprah:if things come down to who has the better database OR who is the better researcher, how is this a measure of who is better at playing chess?

I still fail to see how people are making this (Quantuum?) leap.

Sounds something more like how librarians would hold a competition.

It is a measure of who is better at playing CC!  Richie, Postal chess is library chess.  It's not OTB chess.  Moves are researched... in the old days people went to their libraries and periodicals to research moves...  and would go to their homecooked prepared lines when they could. So yes, it was in part about who was the better librarian... that's certainly part of the fun of cc. Sadly IMO ICCF rules allow engines, so modern CC is less librarian than ever before, more software engineer.  C'est la vie. I like chess.com's rules best despite their unenforceability.

If you don't enjoy being a librarian, digging into dbs and books, for goodness sake there's Live chess online, and there's good old fashioned OTB chess -- go to the local club's chess night and play OTB chess! Why try to make cc something it isn't?

Obviously the guys with 50, (or 500), games at once aren't doing a whole lot of research on their games... I have _two_ active games at the moment... I'm a happy librarian researching the hell out of them. Don't tell me this somehow makes it not chess. For my Bird tournament games I studied the Polar Bear System, watching a couple hours of Danielson's videos. For my current Sicillian Dragon (Yugoslav attack) game, I'll be going to OTB Master games that are published _during_ the play of this game! How cutting edge is that? This is my way of generating enthusiasm for study. Having a game to win (or lose) makes the study a whole hell of a lot more interesting than just poring over books memorizing lines. Is this chess? Feels like chess to me. And eventually the db lines run out, and I'm on my own.... and I start wearing out the analysis board ;) 

Is my cc rating a whole helluva higher than my OTB rating? YES! Am I supposed to be ashamed of that? Or should I be proud that I'm a hard working librarian?  I choose the latter.

Well I think that the more pertinent question is, what should be prohibited here on chess.com -- I believe that is the question you've been alluding to throughout this thread.  The better player versus better researcher argument could be applied to any of the items on the list in my previous post.

For all of the idealistic rhetoric about what is and is not honourable play, there ultimately needs to be a pragmatic approach to establish what resources are permissible because the cost and effort required to police their prohibition either has diminishing returns or because they simply can't be detected, and because they're inclusion on the list of the allowable resources doesn't represent an unreasonably material level of improvement over a players own skill.

I maintain that this pragmatic middle ground has been well chosen here on chess.com with the distinction between dynamic and exhaustive resources such as engines and table-bases versus static ones such as databases and published materials.

JgPyth, you write interesting posts when you don't attack me furiously :)

All you describe are excellent ways of improving your own chess ability. I did the same, I read books/articles and tried to apply in my current games.

But, searching through a database of 7 million games, half of which are cyborg games cannot be compared with that! That's all the point of this thread. I think that this is no longer chess, but searching for the best database.

My reason for playing correspondence chess here: it is very difficult for an active person to go to the local club and play there a long-time control OTB (2 hours). I would like the correspondence games here to allow me to play a long time control game while staying at home. If I will see that this is not worth it since I come across many "database intensive" users, I simply stop playing CC.

Its The exact same! except instead of pouring over books to get theory you are pouring over DB's containing THE EXACT SAME INFORMATION. Heck some DB's ARE books that have been converted over for the new era. The only way this effects chess at all is if you use an engine DURING the game. Anything else is just studying.

PS Kasparov V Deep Blue 2 Rd. 4 result 1/2 - 1/2 Kasparov played all but 2 moves in a 50 move game (ok it's 4% I am still looking through my DB of Kasparovs games to find the 3% one but you see my point) exactly as Fritz predicted after the opening book finished. It's a shame he wouldn't be able to play here. I wqould love to get beat and learn from someone of his calibre

I'd like to share my two cents, and explain to everyone what I do, and my thoughts on everyone's positions here.

First of all, what I do in my online chess games. I have my own unique database that I use, a combination of some that are freely available on-line and an old (like 6 years or something) chessbase set. Mainly, though I use my own notes. I keep extensive notes with variations and sub-variations in every line I play often. I generally play a very limited number of different lines, so those that I do play are fairly well developed. There are of course significant holes. My defenses to anything other than 1. e4 are not what I'd like them to be, there are some lines of the sicilian with white that are also not where I'd like them to be, etc. But my pet lines, like (in particular) everything stemming from the Italian - 1.e4 e5 2.Nf3 Nc6 3. Bc4 - as white, I have many variations written out deeply, beyond where any of my databases have stopped giving useful information. I develop these extensive notes in two ways. When I am not playing, I will analyse the lines with my (somewhat outdated and therefore pretty weak) engine, injecting many of my own ideas into the mix and interpolating them with that of the computer. When I AM playing, I write down my analysis (done with pen, paper, and board) of pretty much all the variations and sub-variations which I've looked at. Later, I write down what's important from them, organise it somewhat better, and after my games are finished, go over these with the engine too. As I've been on a long streak of playing, the up-to-dateness of my engine-work is somewhat behind. After I finish the last two games I'm in now, I plan on taking a break so that I can bring it back up to speed. Now, a note for this. There are certain variations that I have analysed down to final results. For instance, I have a pretty complete analysis of the Fried Liver, all the way down to a black win. If it ever came up in one of my games with black (which won't happen unless I join a theme tourney as I play the sicilian), I would expect to rattle off a over a dozen first choice engine moves in a row, assuming my opponent played any of the more critical variations. This would all be part of my homework. By the way, if I get out of my own notes, I generally revert to MCO 14, assuming it's got anything left in it.

Next, people need to, for the zillionth time, stop comparing Online CC to OTB. They are completely different animals. Now I fully understand people who'd want to have CC without any kind of assistance at all, because it's like your playing on your skill alone but get to use an analysis board and plenty of time. But that's not Chess.com. If you want that, either look elsewhere or find someone who feels the same as you and play against them. Of course, Chess.com isn't real CC either, and thank goodness, because I'd get the pants blown off my by someone using an up-to-date engine. (On a side-note, I don't think real CC is just mindlessly engines playing against each other anyway, because there are still some positions where human insight comes up trumps, and there's also the issue of best using an engine).

To me, slow (i.e. correspondance or modified correspondance, online chess here for example) is about the search for chess truth. I like to play it some, and in areas I think I'm closer to chess truth than a lot of other people. But it's very different than wanting to show I'm the best chessplayer. To me, this can only really be done OTB, as that is what chess really is and, in a way, was meant to be. OTB is the soul of chess. Now, I play live as preparation for OTB; it helps me gauge where I am in remembering my analyses and thinking on my feet. As anyone doing a thorough analysis of my live games could see, my biggest problem as a chessplayer is inconsistency; I'll have spits of brilliance (I've held toe-to-toe with masters and even a couple IMs for a while, based often on good knowledge of my pet opening lines, at my local chess club. I've even beaten a master once at G/5 - a time control I hate but he relishes; mind you, this is one win and maybe three draws out of, oh, say, a hundred games against that particular master alone) but I'm also prone to horrible blunders; I even played badly enough (dropped two pieces in a single move based on a tactic that I saw wrong) that I was losing by far against someone rated around 1000 a couple weeks ago; I ended up winning based several squandered opportunities, the selection by my opponent of THE WORST possible move on at least one occasion, and a complete imbalance of endgame knowledge.

So, to some up, chess is a many-splendid game, with many variations in time control and allowable help (I'd love to see some consultation chess at really high levels) making basically completely different games of it, and you need to find what you're looking for.

As a final note, if anything I've done (as I've described above) is considered cheating (which I'm pretty sure it isn't) here, please let me know, and I'll cease such behaviour in the future and seek a way to forfeit those games, post facto if necessary.

Please excuse my lack of technical knowledge with chess rules.  I classify myself as a new player, but what exactly does database for CC mean.  I use the move back and forth for games sometimes when I forgot where the moves were going.  But, I had no clue you could use machine help with games.  Infact, it feels like cheating to me.  Although I am not good, I play straight, and using help really makes me reassess where I am at ratings wise.

Therefore, ratings with CC chess really don't help you understand where you are as a chess player with databases.  That is unless I don't understand.

To your point about databases only being useful if both sides cooperate, there's a certain amount of irony in that a user of an engine-versus-engine database's Achilles heel is more likely than not a poor or substandard move by their opponent that their opposing engine would never have considered due to its inferiority and which takes the database user "out of book" as a result.

I would caution against blindly considering databases and books equivalent.  Search-ability aside, there are databases that are actually equivalent (if not superior) to engines.  Namely, endgame table-bases.

These are basically a subset of positions for which chess has actually been solved with the aide of engines and as a result can be played perfectly based on these databases.  There is an important distinction, however, between endgame table-bases and engine versus engine databases such as Costelus has described.  That is that endgame table-bases are exhaustive, and cannot be taken out of book -- they will always provide the best possible move no matter what the opposing player does.  This is simply not the case for engine versus engine databases as the sheer complexity of chess prohibits anything close to resembling an exhaustive examination of the possibilities for the stage of the game where they are relevant.

The good news is that the use of these table-bases is also extremely easily detected.

I think you are wrong. Is this the game:

http://www.chessgames.com/perl/chessgame?gid=1070915

I don't have Fritz, the engine I have shows that Kasparov made many mistakes. A more pertinent opinion:

http://www.chesscafe.com/text/yaz09.pdf

Seirawan: "an error-filled game".

Thanks richie, costelus, for the kind words on my post... let me pass it on and say...

Phew, super-post Wanderingwinder :)  I agree with all your points regarding cc and looking for truth, etc. (You really think you've "solved" the fried liver (0-1!) wow!? Have you tested your analysis against Rybka?)

Costelus -- the seven million game cyborg db... Is that really such a problem? the beauty of ratings is that they allow you to find and play people who, for whatever reason, are playing at roughly your level.  To find a level playing field all one really has to do is find someone with a similar rating to one's own (who has 25+ games under their ratings belt with a healthy does of losses) ...

I also think you overstate the impact of dbs -- the best still leave so much choice up to the user; they require skill and knowledge to use well. It just isn't like Rybka. An idiot with Rybka playing for him will play like a super GM. But an idiot with a very big db still has A LOT of ways to go wrong. The idiot-proof db doesn't exist yet AFAIK. I'm not going to say it can't exist, in theory (*although I'm not convinced that it can)... but I don't think it exists yet. 

As for computer researched moves... Costelus, that train left the station a long time ago. Chess-engine researched lines are everywhere -- you could ban all dbs from cc, including the game explorer, and still run into deep engine-researched lines, because every contemporary opening manual -- a book, on ink in paper -- is FULL of them. It's just a fact of chess-life. Or, you could try to ban all research from cc -- there are groups on chess.com that do just that -- I think they're missing what cc has to offer.

I know. I did my own engine analysis for some openings I encountered in live chess. I analize almost all my games - even bullet games. I found lines/moves to play using an engine.

But - the big point - I play as much from those lines as I remember. If I make a database with all the lines then I will remember everything. And, in some sharp opening like KG or Danish, 5 computer moves it's all I need :) 

I have a question for you. Please do not take it as though it contains something malicious. How did you manage to defeat so many cheaters? You crushed many engine users, forcing them to actually resign, not winning on time because their accounts were closed.

And another question: if you like this type of game which resembles chess so much, why you do not compete with others who have the same interest (ICCF players). There you can prove your skills in configuring a chess engine, gathering thousands of games or steering the engine into the right direction. Why do you chose to compete with the players here? Most of them cannot afford to invest time in such activities unrelated with their chess ability.

No, nothing like that.

Gonnosuke, I'm totally with you, though I don't share your love for the building of these DBs. I don't own any DBs or any engines.(*)

Costelus, so you don't stand a chance when you enter a sharp opening against a well prepared opponent who has roughly the same ELO as you. OK, I still don't see the problem.

(*) The one on my Palm Pilot doesn't count :-)

The turn-based chess on this site is for others who have the same interest.  The rules outlining what resources are and are not permissible during turn-based game play are clearly stated, so to presume to speak for the community in trumped-up outrage at the notion that databases are used by some players is preposterous.  Everyone here has already agreed to it.

I think that a similar question could be asked of you and richie_and_oprah and any other like-minded individuals -- perhaps the rule-set you're looking for can be found in Live Chess.

Yes, I know about live chess. But there very few people dare to play long games (the risk of playing an engine is too big). Not only here, at ICC also. Very likely you will find NOBODY to play a serious game of at least 30 minutes per side.

This type of correspondence chess is very convenient. It allows me to play a long, serious game, while not requiring me to go to a club or to find 4 hours to sit continuously on a table. In many cases, this (finding so much spare time in a day in which to play chess without interruption) is not possible.

But, I want to play chess. I want to play against KG, I want to encounter many gambits/risky openings. I am not at the level on which to play only sicilian and closed positional games. I played once a Ng5 in two knights and my opponent wiped the floor with me. That's the problem.

Another problem is that always a little bit of cheating will be impossible to detect. As I said above, it is impossible to know if those 5 decisive moves I made in the game X were made by me, by an engine, or I looked them up in a database.

There are a few groups out there who, based on the honour system, play turn based chess here without any outside assistance.  Perhaps reaching out to them for inclusion and limiting your play to those among their ranks would give you the type of game play you are looking for.

Here is the one that I am aware of:

http://www.chess.com/groups/view/circle-of-trust-otb