Forums

Stockfish dethroned

Sort:
sammy_boi

By the way, I've beaten SF at full strength, no opening book, when I make it give me knight odds and it plays into a French exchange type mid game. We just trade pieces and I have an easily winning endgame.

Maybe this experience makes me biased, but I think no opening book could really skew the results.

Elroch
sammy_boi wrote:

Yes the engine is rendering an eval for a position many moves deep, but it still has to reach this position one move at a time, so I think my point still remains. Its search and eval function weren't tuned with that in mind.

As for filling in the sandwich, sure, I agree, but I'm not sure a 20 move opening book would be good. Since these engines are so strong I was thinking more along the lines of 5 moves of book, 10 at most, then let the engine makes decisions from there. If it's inadvertently getting a passive middlegame all the time, then I think that would skew the results.

As for engine-made opening books, that's more like cheating to me. Basically as if you're giving it more time to think.

Indeed. Well actually, it should combine all top engine games (hundreds of thousands of games, each involving substantial computing time. Centuries of computing time. So it's getting assistance with a huge computational task for move 1. But how is is better to use GM games? Because they are weaker?

As for the endgame, yeah an EGTB is like cheating, so I'd be willing to say no EGTB.

 

DiogenesDue

If engines having opening books is cheating, then humans reading opening books and trying to memorize lines is also a milder form of cheating wink.png.  Maybe you'd propose hamstringing the engine by making it randomly "misremember" the best lines from its opening book?  To simulate fuzzy human memory?  Or maybe artificially limit it to storing only 25 openings to 30 moves and the rest only to 10 moves?

The goal of engine developers is to be the best.  Opening books are currently part of this, though they actually will hurt engines ultimately as they keep improving past 3000+ performance levels by foisting flawed human play on the engine.  Tablebases are also cheating by your definition, but an engine can't and doesn't "cheat", we simply build it to perform the best we can.

If...if TDs had forbidden opening books and tablebases from the very beginning for engine play, then they would have evolved quite differently.  That's one reason it would not have been fair to play Stockfish minus opening book and/or tablebases.  The engine is built around those things and relies on them for its best play.  You might as well perform a partial lobotomy on Carlsen and then tell him to play a match.

Optimissed

That's actually a pretty poor, not very useful definition.  AI is not the study of something.... We may study AI, but it is not in and of itself the study of...

So what you are saying is that the computer scientists who have done all the work on AI have the definition wrong? You should tell them, so they can fix it!>>>>

They're computer scientists, not great philosophers. Having said that, Smyslov should have read my post and from it, even being a non-native English speaker, he should have understood that **The study of** AI is the study of blah blah.

I'm smarting. I just came from a club match, I was top board and black and it was a Torre Attack. As predicted my opponent launched a fearsome K-side attack with all his pieces. I managed to defend and got an opposite colour bishop ending, equal pawns, which was winning for me. I blundered in time trouble and he got a draw. Match drawn. It would be good to be a computer sometimes.

Elroch
Lyudmil_Tsvetkov wrote:
Elroch wrote:
Lyudmil_Tsvetkov wrote:
Elroch wrote:
hairhorn wrote:

The only real connection between number of threads and number of cores is that Stockfish recommends one thread per core. Other settings are possible but sub-optimal. 

If that setting was used, Stockfish was running on 64 cores, which is a very powerful computer. 32 cores is almost as impressive.

Not impressive at all next to what Alpha had.

 

I keep referring to the graphs from the DeepMind paper.

These show that if Stockfish was given thirty times longer per move it would have gained surprisingly few Elo points, and not done much better.

 

This is already bogus.

Doubling time is usually more important than doubling speed, and this is significantly more time.

No, you are wrong by definition.

A computer is twice as fast if it requires half as much time to do the same thing. That is the definition of speed!

I guess they have measured something wrong. We should not believe everything as they wrote it.

 

Infinite_Bishop

This is absolutely incredible! I can't help but wonder what we could learn from alphazero. For instance, is there an undiscovered general rule that makes the english opening superior to others? Or perhaps a small sacrifice in the beginning is best with perfect play. On another note, Stockfish was handicapped because it wasn't allowed an opening database or sufficient memory to play at its strongest. Stockfish would've still lost, but by a smaller margin.

 

I have a few videos analyzing the chess game between alphazero and stockfish here:

Game 1: https://www.youtube.com/watch?v=eMdp0YHUzw4&t=258s

Game 2: https://www.youtube.com/watch?v=6SKK2FPpfGc&t=2s

Game 3: https://www.youtube.com/watch?v=w5CY_ozRwKk

 

Subscribe: https://www.youtube.com/channel/UCrcs75yPaYK6P3nNnyCkBPA

 

matthewgroff

This is non-sense.  Until Alpha Zero's algorithm can be run on a machine without proprietary processors so that is can play Stock Fish 9 (The current strongest engine) on equal machines, we are comparing apples to orangutans.  Plus I would give Stock Fish 9 a book created by playing Stock Fish 9 at fix ply 31 or better against a large database of computer competition games, like can be found through http://www.computerchess.org.uk/ccrl/4040/ .  That would be a better comparison to what Alpha Zero is doing.  My money is on Stock Fish 9.  Until then this is all just propaganda hype for Project Deep Thought.

Godeka

I wouldn’t say a normal PC is hopeless. For Go it works really well even without a fast GPU or any GPU at all. It is not enough to win against the top professional players, and in chess it would not win against Stockfish, but maybe it plays more interesting? At least it can learn different styles and so be a nice and funny player to play with. It can have multiple NN of different strength, and it can be play more human like games.

 

There is some research to make NN smaller by removing the less important neurones. Maybe there is a progress from two sides: the hardware becomes faster and more specialised, and the NN becomes smaller or more efficient.

 

I think in some years it will be possible to buy AI cards. (There are already NVIDIA cards having TPUs, but they are a little bit expensive.)

Cavatine
[COMMENT DELETED]
Elroch
Godeka wrote:

I wouldn’t say a normal PC is hopeless. For Go it works really well even without a fast GPU or any GPU at all. It is not enough to win against the top professional players, and in chess it would not win against Stockfish, but maybe it plays more interesting? At least it can learn different styles and so be a nice and funny player to play with. It can have multiple NN of different strength, and it can be play more human like games.

 

There is some research to make NN smaller by removing the less important neurones. Maybe there is a progress from two sides: the hardware becomes faster and more specialised, and the NN becomes smaller or more efficient.

 

I think in some years it will be possible to buy AI cards. (There are already NVIDIA cards having TPUs, but they are a little bit expensive.)

 No there aren't. There are Nvidia GPUs, which start at a few hundred dollars which will outperform CPUs for AI. The majority are around the same value in terms of computing power per cost.

Godeka

NVIDIA Volta which is expected to be available this year has TPU cores.

Elroch

The Volta microarchitecture is the next generation of Nvidia GPUs. It includes a type of functionality it calls tensor processing, but the term "TPU" has only ever been used for google's hardware that is only available as a cloud service, so your statement is an imprecise use of terminology.

The distinction between GPU and TPU is more a matter of where it comes from than what it can do, as GPUs can do a great deal more than the graphics processing after which they are named and for which they were mainly marketed until recently, and it is clear that Volta is aimed at competing with google's hardware for AI. The extra power of the device probably takes it beyond what even the keenest gamer could seriously take advantage (once an immersive environment reaches a certain level of detail, human vision is not going to benefit from taking it any further!)

DavidForthoffer
I thought the headline should have read, “Google supercomputer beats PC in chess.”

I’d like to see AlphaZero playing Stockfish on equivalent hardware.
Elroch
DavidForthoffer wrote:
I thought the headline should have read, “Google supercomputer beats PC in chess.”

I’d like to see AlphaZero playing Stockfish on equivalent hardware.

I'd like to see a Eurofighter running on the engine from a Porsche. Actually, I wouldn't because that would be as silly as AlphaZero running on a PC. The system was not developed to do so.

As for running Stockfish on a TPU, it hasn't been done. No-one has written the code to do so, and it is not clear that it could. If someone wants to, the hardware is available in the google cloud, and there's even some time for free available.

Until then, I would say AlphaZero has achieved the highest performance seen for an unrestricted chess agent.

prusswan

It is harder for chess to go further since it is one notch below Go as a computing endeavor. Even the chess engine authors prefer to apply their talents to Go (Giraffe's author joined Deepmind, while sjeng's author is behind LeelaZero).

Elroch

It is notable that the relative performance achieved by AlphaZero Go was stupendously higher on a relative scale than that in chess: it was many hundreds of points above human performance. Part of this is that Go is always a decisive game - no draws - and part of it is that it is more complex and offers for scope for AlphaZero's power.