Chess Mastered with a Self-Play Algorithm

[Egidijus Zeromskis]

Re: Chess Mastered with a Self-Play Algorithm

Because you have not taken the time to look up what the words mean does not mean they are baloney. Here is the original alpha go zero paper that actually learned to play go in a matter of days at a world championship level starting with only the basic rules. This was a far more impressive feat then mastering chess. As far as the quality of the games go I am very sure that at this point this thing could beat any traditional chess playing program and have no doubt you will see more convincing games shortly. Educate yourself before jumping to conclusions.
https://www.nature.com/articles/natu...wjxeTUgZAUMnRQ

If you so knowledgeable, can you answer me other questions in my previous message. Though I looked through the paper yesterday.
If the paper is really a scientific one (I am far from the expert in this field to judge), others will be able to repeat the algorithm.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

If you so knowledgeable, can you answer me other questions in my previous message. Though I looked through the paper yesterday.
If the paper is really a scientific one (I am far from the expert in this field to judge), others will be able to repeat the algorithm.

Kotov is not a computer scientist. His method you speak of is for analyzing chess games. It is a tree method for analyzing very specific possible replies to possible moves in a tree like fashion. That is as you know only one aspect of the game, alot of the game is recognizing general patterns as opposed to exact moves. For example when I play the Sicilian defence part of the pattern I am looking for might be an open queen bishop file and perhaps getting control of the d5 square with a flank attack on the queen side chasing away pieces that have influence over that square. It is a more "heuristic" approach to the game. Looking for patterns as opposed to precise moves on a tree is what neural networks and Monte carlo tree searches are really good at. So a system like this will learn by creating probabilistic classifiers for various patterns as opposed to specific moves. That is why it gets away with analyzing far less positions per second then stockfish.
The paper is definitely a scientific one authored by very credible and well known computer scientists.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

Sounds real interesting but... I'm looking at the games on a crappy laptop running the latest version of Komodo and it finds multiple mistakes that Stockfish did. Strange.

Looks like the guys were not really willing to use the best possible conditions for the engine.

As for applying this to other problems, not so sure yet. It's gonna be great for anything statistical or anything with a very specific set of rules (like board games or poker). But it certainly won't be able to come up with new, paradigm shifting ideas, IMO.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

And to be fair, AlphaZero came up with some real good moves. Like the bishop sacrifice on g5 mentioned in the chessbase article. Also, it showed some crafty manoeuvring in closed position. Definitely interesting.

The big question is: would it get destroyed in the opening, either by a well prepared GM or simply by a computer with a fine tuned opening book?

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Sounds real interesting but... I'm looking at the games on a crappy laptop running the latest version of Komodo and it finds multiple mistakes that Stockfish did. Strange.

Looks like the guys were not really willing to use the best possible conditions for the engine.

As for applying this to other problems, not so sure yet. It's gonna be great for anything statistical or anything with a very specific set of rules (like board games or poker). But it certainly won't be able to come up with new, paradigm shifting ideas, IMO.

. But it certainly won't be able to come up with new, paradigm shifting ideas, IMO.

I once meant somebody who worked for BC Tel in the 1980s who commented that fax machines would never catch on. Steve Balmer thought the Iphone would never go anywhere. Your statement reminds me very much of this.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

And to be fair, AlphaZero came up with some real good moves. Like the bishop sacrifice on g5 mentioned in the chessbase article. Also, it showed some crafty manoeuvring in closed position. Definitely interesting.

The big question is: would it get destroyed in the opening, either by a well prepared GM or simply by a computer with a fine tuned opening book?

Would you care to wager on this...if so how much and what odds?

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

Would you care to wager on this...if so how much and what odds?

Is it possible to ask a question without gambling on it? That's what scientists do.

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

I once meant somebody who worked for BC Tel in the 1980s who commented that fax machines would never catch on. Steve Balmer thought the Iphone would never go anywhere. Your statement reminds me very much of this.

Don't get me wrong. I agree that an algorithm like AlphaZero has potential. All I'm saying is that this potential is limited to very specific fields.

Call me back when a computer comes up with a new design for a pair of scisors or when it writes and directs a movie.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Don't get me wrong. I agree that an algorithm like AlphaZero has potential. All I'm saying is that this potential is limited to very specific fields.

Call me back when a computer comes up with a new design for a pair of scisors or when it writes and directs a movie.

This very technology makes the tasks you describe as feasible much sooner then you think.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Is it possible to ask a question without gambling on it? That's what scientists do.

To answer your second question no it would not, the very nature of the algo is that it continually gets better and better. You saw the results of 24 hours of learning, imagine after a week on TPU type processors. This algo is important across many fields and represents a major breakthrough in AI.

[Garland Best]

Re: Chess Mastered with a Self-Play Algorithm

I think the naysayers about Alphazero are missing a key point here. This is a program that started with ZERO information about chess other than the basic rules, and within FOUR HOURS trained itself to the point where it had developed its own set of openings, its own evaluation of the piece values, its own heuristics to evaluate whether one move was better than another, its own endgame strategies, and its own form of calculating move sequences, all without human intervention, and do so well enough to go against one of the best programs in the world and win. Even if Stockfish was somewhat hobbled (no huge opening book, no access to endgame tablebases), it still lost and lost badly. And it lost doing what Alphazero was doing - working from it's own evaluation algorithms, without use of external references.

As far as I know, this is the first successful chess program that uses an approach other than the brute force alpha-beta method. And let's not forget that the reason computer programs have improved is not the number of positions per second they evaluate - it's the quality of the evaluations at the end of the tree. See for example https://en.chessbase.com/post/komodo...ktop-challenge. Up until now, the evaluation algorithms have all been devised by humans. Now we have computers coming up with better evaluation algorithms on their own. What happens when we combine the two together?

So this is a huge leap in computer chess, and more importantly in machine learning. This is a fundamentally different tool that will find use in many applications. As an example, teach it the controls of an airplane, attach it to a flight simulator and set winning to be "take this plane from Toronto to Chicago". Run the simulator enough times and with enough scenarios, and maybe you will have a self-flying plane.

[Egidijus Zeromskis]

Re: Chess Mastered with a Self-Play Algorithm

As far as I know, this is the first successful chess program that uses an approach other than the brute force alpha-beta method. And let's not forget that the reason computer programs have improved is not the number of positions per second they evaluate - it's the quality of the evaluations at the end of the tree. See for example https://en.chessbase.com/post/komodo...ktop-challenge. Up until now, the evaluation algorithms have all been devised by humans. Now we have computers coming up with better evaluation algorithms on their own. What happens when we combine the two together?.

Looking through cited source the Giraffe engine poped out. It was (is?) based on machine learning too. The author is also one of the AZ authors :)
There is his prediction ( http://www.talkchess.com/forum/viewtopic.php?t=59003 ) before he dropped Giraffe after moving to DeepMind:
"It's no fun knowing there are several techniques I can try that will probably drastically improve Giraffe, when I can't try them because they are still trade secret. "
"Machine learning has defeated hand-crafted systems in just about every other field. That will happen in chess, too, and the only question is when. "

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

Seriously. Look at the very first game given in the database, where AlphaZero won with black in 67 moves.

On move 35, Stockfish has 4 pawns and a knight, against black's two bishop (with rooks on both sides). It can play 35.Ng6, practically forcing the exchange of the knight for the bishop on f8, which leads to a relatively easy draw, as the only weakness to protect is the pawn on c2.

Instead, SF played 35.Nc4?? Leaving the game completely open where black will be able to use the bishop pair. I can't even understand why SF would play that.

I don't doubt A0's algorithm. It's interesting and all. But the actual chess in there is really fishy (no pun intended). Even I, with my old calculating machine, would play 35.Ng6 or 35.Rc1 without even thinking twice. 35.Nc4 doesn't make an ounce of sense, even for an engine. The knight even had to go back to e5 a few moves later.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Seriously. Look at the very first game given in the database, where AlphaZero won with black in 67 moves.

On move 35, Stockfish has 4 pawns and a knight, against black's two bishop (with rooks on both sides). It can play 35.Ng6, practically forcing the exchange of the knight for the bishop on f8, which leads to a relatively easy draw, as the only weakness to protect is the pawn on c2.

Instead, SF played 35.Nc4?? Leaving the game completely open where black will be able to use the bishop pair. I can't even understand why SF would play that.

I don't doubt A0's algorithm. It's interesting and all. But the actual chess in there is really fishy (no pun intended). Even I, with my old calculating machine, would play 35.Ng6 or 35.Rc1 without even thinking twice. 35.Nc4 doesn't make an ounce of sense, even for an engine. The knight even had to go back to e5 a few moves later.

I don't see a concrete line that forces the exchange after n-g6. In any event the fact that you or i don't understand SF is neither here nor there. It does nothing to prove your point that SF was playing below standard.

[Paul Bonham]

Re: Chess Mastered with a Self-Play Algorithm

Go was magnitudes more difficult to master then chess and the thing did it. I am fully confident the games were not cherry picked and even beefing up stockfish's hardware would not come even close to beating this type of software. This is very telling here, the thing that beat stockfish had nohing to do with brute force and everything to do with software methodologies. This is an entire new way of AI and chess.

Hi Sid,
I'd really like to discuss all this with you. I am in agreement with some of your points, but not all of them -- the ones I am not in agreement with are just because I think there needs to be more evidence, not because I think it just can't be so.

But first, you shouldn't be surprised that Vlad Drkulec is opposed to the idea that science is conquering vast areas of what was once considered sacrosanct domains of human thought. If AlphaZero were to teach itself climate change science and conclude that man is changing the climate just as 90%+ of climate scientists are claiming, Vlad would dismiss it all as voodoo science and walk away to go read his Flat Earth Society journals. He is resistant to the very idea that science can tell us anything. Nevermind all the evidence from Los Angeles, Houston, Puerto Rico, Miami, Antarctica, India, China, the Far East, etc. Nevermind all the "once in 500 years" storms that are ravaging the planet in just the past few years.

So enough about him. Let's dismiss him the way he dismisses science.

I do think AZ represents a fundamental paradigm shift going on. For years, I argued about the potential of neural net programming for machine learning, and not many were listening. The technology just wasn't there, plus it requires LOTS of training data. I am super happy that people are now working on such systems as AlphaZero. Just the fact that it can completely learn Go or chess in just a few hours is telling us something dramatic is going on. No one can deny that.

But I do have to be skeptical about some claims. First, the one above that "Go was magnitudes more difficult to master then chess". I don't think this can be proven. Yes, it's true that HUMANS have had more trouble with Go than with chess. But that may actually indicate something problematic about the human brain. Now, what could have happened is that AlphaZero studied thousands of Go games betweeen top-level players and discovered something simple that has eluded humans, some principle that is either too complicated OR TOO SIMPLE for humans to get. And it used that information to defeat the Go masters. I'm just saying it's possible, and we have to (as scientists) account for it. So let's leave the Go accomplishments of AlphaZero out of the picture, it's too problematic on that basis.

But the AlphaZero 4-hour mastery of chess and defeating of (a crippled) Stockfish is stunning, no bones to pick there.

What needs to happen now is for the AlphaZero developers to go way beyond chess and Go. These are games of perfect information, and as long as they stick with those games, AZ will be criticized for being limited to perfect information domains in general.

They need to prove that AZ can be applied to stock trading just as easily as to chess, and can earn millions of dollars a year just trading on the major markets as a human would do. It needs to be consistent, over good days and bad. And it needs to teach itself from zero.

EDIT: think about this for a moment: imagine that AZ does conquer stock trading and vastly outperforms humans. Then imagine humans en masse buy a home version of AZ for stock trading and millions of AZ engines log into stock trading platforms and start trading.... ALL of them making money. What do you think would happen? Why, all trading would cease probably within hours, there would be no sellers. All the human sellers would be gone, their accounts emptied of funds. The engines wouldn't trade with each other because they are all geared to making money.

I also think that this spells curtain for anticomputer cheating variants of chess such as those worked on by Paul Bonham. This thing takes a heuristic approach to learning that means that it can easily master games with incomplete information such as No Limit Texas Holdem poker that for the first time can beat poker pro's at their own game.

You are of course referring to my anti-computer and anti-cheating chess variant called Option Chess.

The point about Option Chess, and also about poker, is that it's one thing to have an engine that can learn the game and win a game here or a game there, even against masters / pros. That is a great accomplishment, but in the case of poker, we already have numerous examples of virtual noobs at the game winning major tournaments against poker pros. It happens quite often -- TOO often in fact, making poker pros look bad.

So merely winning the odd event falls short of what really would be desired: total domination of humans by the learning engine. And THAT is something that I claim will not happen for decades to come. Perhaps quantum computers could do it, but not current technology.

So with regards to Option Chess, my claim remains that no engine in the next 40 years will dominate a trained or expert human player. I'll define "dominate" to mean winning 2 out of every 3 decisive games (draws would be ignored). If my claim is true, that would mean that even the best Option Chess engine over those 40 years would not be useful for computer cheating in Option Chess against top level human opposition.

I'm open to have that claim disproven, but who can be the "trained, expert human"? So let me amend this a bit: if Stockfish or Komodo or Fritz or any top-10 computer engine were modified to play Option Chess and play it at a high level as it does regular chess, I would extend this claim to have the "trained, expert human" replaced by such an engine, and to dispel all doubts, the two engines AZ and Stockfish, Komodo, whatever, would have to be playing on hardware that is considered near identical (recognizing that TPUs are not the same beast as CPUs nor GPUs).

And I would remain confident that AZ would not dominate that engine under those conditions. In fact, I would predict a nearly 50% split of the games.

The reason? Neither AZ nor the opposing engine would be able to work out with consistent accuracy the one piece of hidden information that is inherent to Option Chess: the Current Value of an unused Option token. In fact, I remarked to someone the other day, the Current Value of such a token is like the "Current Value" of a Schrodinger Wave Equation: in order to measure it, you have to collapse the wave. For the Option token, that means actually playing the option. If you keep it for later, the wave goes on and Current Value can only be guessed at. The point being, how would the engine know whether or not to keep the token for later use? It can't put a value on that. It could try different values, but the problem is, the value would change with every situation. In the end, I assert that the engine would not know any better than the human or Stockfish opponent whether or not to save the token for later use.