Chess Mastered with a Self-Play Algorithm

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Seriously. Look at the very first game given in the database, where AlphaZero won with black in 67 moves.

On move 35, Stockfish has 4 pawns and a knight, against black's two bishop (with rooks on both sides). It can play 35.Ng6, practically forcing the exchange of the knight for the bishop on f8, which leads to a relatively easy draw, as the only weakness to protect is the pawn on c2.

Instead, SF played 35.Nc4?? Leaving the game completely open where black will be able to use the bishop pair. I can't even understand why SF would play that.

I don't doubt A0's algorithm. It's interesting and all. But the actual chess in there is really fishy (no pun intended). Even I, with my old calculating machine, would play 35.Ng6 or 35.Rc1 without even thinking twice. 35.Nc4 doesn't make an ounce of sense, even for an engine. The knight even had to go back to e5 a few moves later.

I don't see a concrete line that forces the exchange after n-g6. In any event the fact that you or i don't understand SF is neither here nor there. It does nothing to prove your point that SF was playing below standard.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

Seriously. Look at the very first game given in the database, where AlphaZero won with black in 67 moves.

On move 35, Stockfish has 4 pawns and a knight, against black's two bishop (with rooks on both sides). It can play 35.Ng6, practically forcing the exchange of the knight for the bishop on f8, which leads to a relatively easy draw, as the only weakness to protect is the pawn on c2.

Instead, SF played 35.Nc4?? Leaving the game completely open where black will be able to use the bishop pair. I can't even understand why SF would play that.

I don't doubt A0's algorithm. It's interesting and all. But the actual chess in there is really fishy (no pun intended). Even I, with my old calculating machine, would play 35.Ng6 or 35.Rc1 without even thinking twice. 35.Nc4 doesn't make an ounce of sense, even for an engine. The knight even had to go back to e5 a few moves later.

[Egidijus Zeromskis]

Re: Chess Mastered with a Self-Play Algorithm

As far as I know, this is the first successful chess program that uses an approach other than the brute force alpha-beta method. And let's not forget that the reason computer programs have improved is not the number of positions per second they evaluate - it's the quality of the evaluations at the end of the tree. See for example https://en.chessbase.com/post/komodo...ktop-challenge. Up until now, the evaluation algorithms have all been devised by humans. Now we have computers coming up with better evaluation algorithms on their own. What happens when we combine the two together?.

Looking through cited source the Giraffe engine poped out. It was (is?) based on machine learning too. The author is also one of the AZ authors :)
There is his prediction ( http://www.talkchess.com/forum/viewtopic.php?t=59003 ) before he dropped Giraffe after moving to DeepMind:
"It's no fun knowing there are several techniques I can try that will probably drastically improve Giraffe, when I can't try them because they are still trade secret. "
"Machine learning has defeated hand-crafted systems in just about every other field. That will happen in chess, too, and the only question is when. "

[Garland Best]

Re: Chess Mastered with a Self-Play Algorithm

I think the naysayers about Alphazero are missing a key point here. This is a program that started with ZERO information about chess other than the basic rules, and within FOUR HOURS trained itself to the point where it had developed its own set of openings, its own evaluation of the piece values, its own heuristics to evaluate whether one move was better than another, its own endgame strategies, and its own form of calculating move sequences, all without human intervention, and do so well enough to go against one of the best programs in the world and win. Even if Stockfish was somewhat hobbled (no huge opening book, no access to endgame tablebases), it still lost and lost badly. And it lost doing what Alphazero was doing - working from it's own evaluation algorithms, without use of external references.

As far as I know, this is the first successful chess program that uses an approach other than the brute force alpha-beta method. And let's not forget that the reason computer programs have improved is not the number of positions per second they evaluate - it's the quality of the evaluations at the end of the tree. See for example https://en.chessbase.com/post/komodo...ktop-challenge. Up until now, the evaluation algorithms have all been devised by humans. Now we have computers coming up with better evaluation algorithms on their own. What happens when we combine the two together?

So this is a huge leap in computer chess, and more importantly in machine learning. This is a fundamentally different tool that will find use in many applications. As an example, teach it the controls of an airplane, attach it to a flight simulator and set winning to be "take this plane from Toronto to Chicago". Run the simulator enough times and with enough scenarios, and maybe you will have a self-flying plane.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Is it possible to ask a question without gambling on it? That's what scientists do.

To answer your second question no it would not, the very nature of the algo is that it continually gets better and better. You saw the results of 24 hours of learning, imagine after a week on TPU type processors. This algo is important across many fields and represents a major breakthrough in AI.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Don't get me wrong. I agree that an algorithm like AlphaZero has potential. All I'm saying is that this potential is limited to very specific fields.

Call me back when a computer comes up with a new design for a pair of scisors or when it writes and directs a movie.

This very technology makes the tasks you describe as feasible much sooner then you think.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

I once meant somebody who worked for BC Tel in the 1980s who commented that fax machines would never catch on. Steve Balmer thought the Iphone would never go anywhere. Your statement reminds me very much of this.

Don't get me wrong. I agree that an algorithm like AlphaZero has potential. All I'm saying is that this potential is limited to very specific fields.

Call me back when a computer comes up with a new design for a pair of scisors or when it writes and directs a movie.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

Would you care to wager on this...if so how much and what odds?

Is it possible to ask a question without gambling on it? That's what scientists do.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

And to be fair, AlphaZero came up with some real good moves. Like the bishop sacrifice on g5 mentioned in the chessbase article. Also, it showed some crafty manoeuvring in closed position. Definitely interesting.

The big question is: would it get destroyed in the opening, either by a well prepared GM or simply by a computer with a fine tuned opening book?

Would you care to wager on this...if so how much and what odds?

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Sounds real interesting but... I'm looking at the games on a crappy laptop running the latest version of Komodo and it finds multiple mistakes that Stockfish did. Strange.

Looks like the guys were not really willing to use the best possible conditions for the engine.

As for applying this to other problems, not so sure yet. It's gonna be great for anything statistical or anything with a very specific set of rules (like board games or poker). But it certainly won't be able to come up with new, paradigm shifting ideas, IMO.

. But it certainly won't be able to come up with new, paradigm shifting ideas, IMO.

I once meant somebody who worked for BC Tel in the 1980s who commented that fax machines would never catch on. Steve Balmer thought the Iphone would never go anywhere. Your statement reminds me very much of this.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

And to be fair, AlphaZero came up with some real good moves. Like the bishop sacrifice on g5 mentioned in the chessbase article. Also, it showed some crafty manoeuvring in closed position. Definitely interesting.

The big question is: would it get destroyed in the opening, either by a well prepared GM or simply by a computer with a fine tuned opening book?

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

Sounds real interesting but... I'm looking at the games on a crappy laptop running the latest version of Komodo and it finds multiple mistakes that Stockfish did. Strange.

Looks like the guys were not really willing to use the best possible conditions for the engine.

As for applying this to other problems, not so sure yet. It's gonna be great for anything statistical or anything with a very specific set of rules (like board games or poker). But it certainly won't be able to come up with new, paradigm shifting ideas, IMO.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

If you so knowledgeable, can you answer me other questions in my previous message. Though I looked through the paper yesterday.
If the paper is really a scientific one (I am far from the expert in this field to judge), others will be able to repeat the algorithm.

Kotov is not a computer scientist. His method you speak of is for analyzing chess games. It is a tree method for analyzing very specific possible replies to possible moves in a tree like fashion. That is as you know only one aspect of the game, alot of the game is recognizing general patterns as opposed to exact moves. For example when I play the Sicilian defence part of the pattern I am looking for might be an open queen bishop file and perhaps getting control of the d5 square with a flank attack on the queen side chasing away pieces that have influence over that square. It is a more "heuristic" approach to the game. Looking for patterns as opposed to precise moves on a tree is what neural networks and Monte carlo tree searches are really good at. So a system like this will learn by creating probabilistic classifiers for various patterns as opposed to specific moves. That is why it gets away with analyzing far less positions per second then stockfish.
The paper is definitely a scientific one authored by very credible and well known computer scientists.

[Egidijus Zeromskis]

Re: Chess Mastered with a Self-Play Algorithm

Because you have not taken the time to look up what the words mean does not mean they are baloney. Here is the original alpha go zero paper that actually learned to play go in a matter of days at a world championship level starting with only the basic rules. This was a far more impressive feat then mastering chess. As far as the quality of the games go I am very sure that at this point this thing could beat any traditional chess playing program and have no doubt you will see more convincing games shortly. Educate yourself before jumping to conclusions.
https://www.nature.com/articles/natu...wjxeTUgZAUMnRQ

If you so knowledgeable, can you answer me other questions in my previous message. Though I looked through the paper yesterday.
If the paper is really a scientific one (I am far from the expert in this field to judge), others will be able to repeat the algorithm.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

You know this sounds as a total makra-adabra with many fancy words.

Because you have not taken the time to look up what the words mean does not mean they are baloney. Here is the original alpha go zero paper that actually learned to play go in a matter of days at a world championship level starting with only the basic rules. This was a far more impressive feat then mastering chess. As far as the quality of the games go I am very sure that at this point this thing could beat any traditional chess playing program and have no doubt you will see more convincing games shortly. Educate yourself before jumping to conclusions.
https://www.nature.com/articles/natu...wjxeTUgZAUMnRQ