AlphaZero - is this what 4000 rated chess looks like?

[Sid Belzberg]

I will admit my bias upfront that reading this thread I didn't think 4...a6 demonstrates much of anything, though of course asking one move to prove anything is a pretty small sample size. However, I am really impressed with the amount of research that went into this post.

Yes, I was also impressed by that. I was actually surprised that this move was tried at all in any serious game. I was equally surprised that no human in any of the cited games had ever followed up with the move Stockfish did 5 N-g5. The discussion sort of takes away from the main point I was making that we are seeing many new novel ideas like 4a6 that as we can see up to now are rarely considered and indeed the play of Alphazero has been described by many as "other worldly" and I absolutely agree with this characterization.
Already similar methodologies are being deployed for things like near instant diagnosis for possible stroke by comparing a patients CAT scan to large databases of known CAT scans. I consider this technological breakthrough to be almost on the same order of importance as the elucidation of the DNA molecule in 1953.

[Paul Bonham]

....I was equally surprised that no human in any of the cited games had ever followed up with the move Stockfish did 5 N-g5. The discussion sort of takes away from the main point I was making that we are seeing many new novel ideas like 4a6 that as we can see up to now are rarely considered and indeed the play of Alphazero has been described by many as "other worldly" and I absolutely agree with this characterization.....

For all we know, the programmers of AZ could have inserted code in the engine to say that "if you find an early opening move that is very rare but at the same time is about equally strong as the usual move, play the rare move to get the opponent out of book or out of familiar territory" and AZ could have connections to chess databases to find out which moves fit that category. The 4....a6 move definitely fits the bill, as it has been very rarely played and yet SF8 itself considers it almost equal with 4...Nf6.

If the above were to be the case, there is absolutely nothing otherworldy about it. Just clever programming.

The other point I would make is that we don't yet know how often AZ is willing to play these "novelties". Have it play 10,000 games of the Giucco Piano as Black up to move 4.d3 and see how many times it plays 4...a6 versus 4...Nf6. If the 4...a6 move predominates, then either the programming effect I noted above is in effect OR AZ really has found something about 4...a6 by studying complete game results in which the games are generated internally by semi-random move selection.

I have no problem admitting that AZ plays very different stylistically from all other chess engines (excepting LCZero which is modeled after AZ). For that alone, we should all be glad AZ has come along and we should hope many more similar engines can be available to use on typical computer hardware very soon. This would get people studying computer played chess much more than in the past. Computer chess has developed a reputation for being very dry and uninteresting to most.

Sid I do not want to play chess960 as it devolves into regular chess much too quickly. I haven't played competitive chess at all in over 20 years. Option Chess only has short segments where it may become like regular chess, so the entire game throughout is different. I'll leave my challenge open. You really have very little to lose and a lot to gain -- if you took time away from playing regular chess on that app you mentioned and spent it instead on playing a much longer more involved game of Option Chess, you'd be a better person for the experience imo.

I actually thought you were much higher rated than 2032, I thought you were up around 2400 or at least 2300. Maybe at some time you were.

[Sid Belzberg]

For all we know, the programmers of AZ could have inserted code in the engine to say that "if you find an early opening move that is very rare but at the same time is about equally strong as the usual move, play the rare move to get the opponent out of book

Sure, and "for all we know" the Apollo mission to the moon was shot in a movie studio.

Sid I do not want to play chess960 as it devolves into regular chess much too quickly.

That's fine, but if you have that much of an aversion to chess opening theory and even chess theory in general then it underscores my point that your commentary on Stockfish's choice of lines in an opening is of little relevance. I do not want to play option chess for exactly the opposite reason.

[Paul Bonham]

Originally posted by Paul Bonham
For all we know, the programmers of AZ could have inserted code in the engine to say that "if you find an early opening move that is very rare but at the same time is about equally strong as the usual move, play the rare move to get the opponent out of book

Sure, and "for all we know" the Apollo mission to the moon was shot in a movie studio.

Are you actually saying that it is as improbable that programmers of AZ could have written the type of code I mentioned as that the Apollo mission was a hoax?

You've already embarrassed yourself in this thread with your ridiculous claims of "we'll never see 4...a6 in the Giucco Piano by high level chess players in our lifetimes" only to have to admit to being surprised that it was played in at least 8 high level games (probably lots more as Vlad Drkulec claimed).

Why go even further and make yourself look worse? Do you know anything about programming at all? Do you know anything about chess engine programming at all? Do you know that chess engine programmers look for every little advantage they can get? Duh....



Originally posted by Paul Bonham
Sid I do not want to play chess960 as it devolves into regular chess much too quickly.

That's fine, but if you have that much of an aversion to chess opening theory and even chess theory in general then it underscores my point that your commentary on Stockfish's choice of lines in an opening is of little relevance. I do not want to play option chess for exactly the opposite reason.

While I know this isn't going to do any good, it should interest you to know as someone deeply involved in AI research that Option Chess could easily become, and in fact SHOULD easily become, the next big challenge to AZ and other NN chess engines. I could explain why, but based on your history here (and the way you've been posting just in this thread) that anything I say about that will fall on deaf ears. It's become too personal and emotional for you, which has been your entire history on this board.

For someone interested in AI, you have a remarkably closed mind. I have to wonder if your brother in law at that AI company is aware of your posts here. If yes, he's got to be shaking his head and muttering under his breath....

[Sid Belzberg]

Are you actually saying that it is as improbable that programmers of AZ could have written the type of code I mentioned as that the Apollo mission was a hoax?

Yes ,I am actually saying that. I thought YOU knew enough about programming to understand why your scenario is ridiculous. The main claim at Deep Mind is the thing started "Tabula Rasa". They proved in far more complex games like GO that this works. This result is being reproduced with projects like Leela Zero where the code is open source and indeed works TABULA RASA. They do not want to gain an "edge" with external information about openings and what is rarely played and in fact would defeat the purpose of their research proving that a system can start knowing only the rules and teach itself.("duh"!) It is unbelievable how you have missed the point of their entire project and seem to think that their goal is to be yet another chess engine provider.

Your claim is so absurd that if anything the only thing that is embarrassing to me is holding a discussion with someone at your level of ignorance as proven by you suggesting a scenario of fraudulent claims from Deep Mind, that they would not get away with. I feel like having a discussion with you is about as useful as talking with someone from the Flat Earth Society.

What part of TABULA RASA do you not understand?? :-) The methodology has more then proven to work with Open source projects where only the rules of the game are programmed in that starts with a clean slate totally self taught, no other external info, check out the code yourself with the Leela zero open source project if you don't believe it!

By the way, the fact that only 204 games showed up with 4 a6 out of 7.1 million and not one where a Grandmaster deployed it only strengthened my point that a top player (alpha zero) uncorked a novelty that works and that is one of many. I thought it was telling that you had to resort to 30+(!) ply with Stockfish to find this move via brute force and it still did not rank it as a first choice!

Option Chess could easily become, and in fact SHOULD easily become, the next big challenge to AZ and other NN chess engines

Actually, Go was a far bigger challenge then a chess variant like Option chess as was discussed at length in an earlier thread.. I actually wasted some time studying the rules of your silly Option Chess and unless they have changed as I understand it after 8 moves each player is allowed to use 12 tokens issued to them where they can use up a token at at any time to make two consecutive moves instead of one with qualifiers like allowing the opponent to defend against check etc. More tokens are issued at a later stage of the game. So,yes, far more complex then chess but not as complex as Go and it is still a game of complete information with a set of fixed rules unlike a game of incomplete information like Texas Holdem where two cards are hidden.

The algo is designed to be for general purpose zero sum games of complete information. So Paul ,why talk to me about it , go ahead and modify the leela zero code. You complain that you need "option chess masters" first, well ,here is your ideal first student for that.,,you know... that Tabula Rasa thing we have discussed at length in other threads

You will need volunteers to offer up computational resources like with Leela Zero but I suspect that no one involved in this will view the opportunity as "interesting" as you do. They will all be as "closed minded" as I am, in view of the fact that they have more then proven their point with more complex zero sum games of complete information like Go. Initially your variant might have been interesting as a way of playing online where one did not have to worry about a chess engine assisting them.It is now even more obvious then the last thread when we discussed this (when only alphagozero was out) that it is now too late for your variant to have a viable business built around it. The truth is that contrary to your predictions Option Chess does not have wide spread appeal and without the other aspect of always being 100 percent sure that it is impervious to machine assisted cheating it is useless and always will remain that way.

You talk about me getting personal? At least you mercifully spared me in this thread from harassing me about my ethnic origin as is your usual MO but still could not resist trying to insult me. About the only "personal " thing I suggested in this thread is that as a B player it is ridiculous for you to second guess the move choices of a 2800+ rated chess program. This thread is exhausted but no doubt you will have some glib nonsensical self serving reply that I will choose to ignore as with all of your threads has devolved into trolling.

[Peter McKillop]

Sid, to give you some idea of my computer programming expertise, my last experience with coding happened nearly 50 years ago. If memory serves, the language I was taught was fortran. So, I''m asking my questions purely from a layman's perspective. What does 'tabula rasa' really mean in this AZ context? It seems awfully hard to believe, from my coding-illiterate perspective, that it could mean a totally blank slate with only the exception of the rules. For example, wouldn't the AZ program need some means of understanding the objectives of the game? Also, how would AZ evaluate posiitions without some kind of programmed methodology? And if AZ needs 'something' beyond just the rules, is it not possible that the programmers' preconceptions/biases could enter into the picture, thus distorting any AI effectiveness conclusions?

[Sid Belzberg]

It seems awfully hard to believe, from my coding-illiterate perspective, that it could mean a totally blank slate with only the exception of the rules.

Believe it, because that is exactly what it is and that is the heart of the whole thing. As a member of The American Association For The Advancement of Science I have access to the entire paper that was published only a few weeks ago.
Here are a few selected excerpts from it.

COMPUTER SCIENCE
A general reinforcement learning algorithm that masters chess, shogi,and Go through self-play
David Silver1,2*†, Thomas Hubert1*, Julian Schrittwieser1*, Ioannis Antonoglou1,
Matthew Lai1, Arthur Guez1, Marc Lanctot1, Laurent Sifre1, Dharshan Kumaran1,
Thore Graepel1, Timothy Lillicrap1, Karen Simonyan1, Demis Hassabis1†

The game of chess is the longest-studied domain in the history of artificial intelligence.
The strongest programs are based on a combination of sophisticated search techniques,
domain-specific adaptations, and handcrafted evaluation functions that have been refined
by human experts over several decades. By contrast, the AlphaGo Zero program recently
achieved superhuman performance in the game of Go by reinforcement learning from self-play.
In this paper, we generalize this approach into a single AlphaZero algorithm that can achieve
superhuman performance in many challenging games. Starting from random play and given
no domain knowledge except the game rules, AlphaZero convincingly defeated a world
champion program in the games of chess and shogi (Japanese chess), as well as Go.

A long-standing ambition of artificial intelligence
has been to create programs that can instead
learn for themselves from first principles
(5, 6). Recently, the AlphaGo Zero algorithm
achieved superhuman performance in the game
of Go by representing Go knowledge with the
use of deep convolutional neural networks (7, 8),
trained solely by reinforcement learning from
games of self-play (9). In this paper, we introduce
AlphaZero, a more generic version of the AlphaGo
Zero algorithm that accommodates, without
special casing, a broader class of game rules.
We apply AlphaZero to the games of chess and
shogi, as well as Go, by using the same algorithm
and network architecture for all three games.
Our results demonstrate that a general-purpose
reinforcement learning algorithm can learn,
tabula rasa—without domain-specific human
knowledge or data, as evidenced by the same
algorithm succeeding in multiple domains—
superhuman performance across multiple challenging
games.

[Paul Bonham]

Yes ,I am actually saying that. I thought YOU knew enough about programming to understand why your scenario is ridiculous. The main claim at Deep Mind is the thing started "Tabula Rasa". They proved in far more complex games like GO that this works. This result is being reproduced with projects like Leela Zero where the code is open source and indeed works TABULA RASA. They do not want to gain an "edge" with external information about openings and what is rarely played and in fact would defeat the purpose of their research proving that a system can start knowing only the rules and teach itself.("duh"!) It is unbelievable how you have missed the point of their entire project and seem to think that their goal is to be yet another chess engine provider.

Your claim is so absurd that if anything the only thing that is embarrassing to me is holding a discussion with someone at your level of ignorance as proven by you suggesting a scenario of fraudulent claims from Deep Mind, that they would not get away with. I feel like having a discussion with you is about as useful as talking with someone from the Flat Earth Society.

What part of TABULA RASA do you not understand?? :-) The methodology has more then proven to work with Open source projects where only the rules of the game are programmed in that starts with a clean slate totally self taught, no other external info, check out the code yourself with the Leela zero open source project if you don't believe it!

By the way, the fact that only 204 games showed up with 4 a6 out of 7.1 million and not one where a Grandmaster deployed it only strengthened my point that a top player (alpha zero) uncorked a novelty that works and that is one of many. I thought it was telling that you had to resort to 30+(!) ply with Stockfish to find this move via brute force and it still did not rank it as a first choice!

Haha! There were not 7.1 million games of the Giucco Piano with 4.d3. Nice try, but you can't fool all of the people all of the time.

Here's the part about programming that you do not understand Sid: computers don't do ANYTHNG on their own. You spout this tabula rasa stuff as if the programmers simply turned the hardware on and voila it was learning chess. Or maybe you think they fed in a text file with the rules of chesss printed in it, and it used AI to read the rules, and it figured out all on its own that from that moment on it had to start thinking of chess moves to play.

The engine had to be PROGRAMMED to play chess, which means it had to be programmed to choose one move out of many. And part of that programming could easily have been code that says "in early opening play, when 2 or more moves are about equal in score, look them up in databases and choose the one that has been least played". They could code that and STILL say their engine is learning tabula rasa!

Let me repeat that: they could have put that code in and STILL BE SAYING THEIR ENGINE IS LEARNING TABULA RASA.

If you disagree with that, then you still don't understand programming nor do you understand the motivations of engine programmers. First of all, understand that there has to be code somewhere in the engine to tell it to find the move with the "highest score" as obtained by mcts methods and then to play that move. The mcts method itself had to be coded. THE ENGINE DIDN'T WRITE ITS OWN CODE FROM NOTHING!

The whole tabula rasa thing in this context only means that the engine doesn't have any eval() function to tell it the value of a particular chess position. It doesn't know the value of passed pawns or king safety or any other human-gained knowledge of the game. But it is still TOLD BY PROGRAMMER CODE to play moves that abide by chess rules. Otherwise it would sit there and do nothing. COMPUTERS DON'T DO ANYTHING WITHOUT BEING TOLD TO DO IT BY MEANS OF MACHINE INSTRUCTIONS.

And it is TOLD BY PROGAMMER CODE how to score the game when no more moves are possible as specified by the rules or when draw conditions as determined by the rules have materialized. And it is TOLD BY PROGRAMMER CODE how to feed that score back and use it in the mcts method of aggregating results and scoring the originating moves.

So it could easily be TOLD BY PROGRAMMER CODE that when after all that has been done for a particular move, and 2 or more candidate moves are about equal in score, look them up in databases and choose the one that has been least played". That code would not violate tabula rasa, because the engine is already being TOLD BY PROGRAMMER CODE to choose the move with the best score! If what I'm saying could have been coded in amounts to being "fraudulent" (a word I never brought up), then the programmers telling it to choose the move with the best score would ALSO be fraudulent!

DUH!

And by the way, even if the programmers didn't insert code to say choose the best move when 2 or more moves are tied for best score by doing database comparisions... they still had to have SOME MEANS of telling AZ which one to choose. That means could be simply random number generation. Each move gets 50% chance of being chosen. In that case, you'd still have AZ choosing "novelties" when they are scored as good as the standard moves, and there could be a tolerance given to that. So when Stockfish 8 scores 4....a6 within 0.04 points of 4...Nf6, then if that is within AZ's tolerance (0.04 is meaningless, it is 1/25 of a Pawn), it uses whatever technique the programmers coded it to use.

PROGRAMMING 101: COMPUTER CPUs DON'T DO ANYTIHNG WITHOUT MACHINE INSTRUCTIONS PROVIDED TO THEM. THEY WILL ONLY DO WHAT THE MACHINE INSTRUCTIONS TELL THEM TO DO.

"Tabula rasa" isn't "abaracadabara". AZ may have learned to play better-than-GM chess in 4 hours, but it still had to be PREPARED to do that, and that means being FED WITH CODE.

This whole business of you proclaiming that AZ is choosing novelties because they are somehow better than humans or even Stockfish have ever realized is ABSOLUTE RUBBISH.

You have shown yourself throughout this conversation to be quite lacking in thought. I have out-researched you to the point of totally surprising you, and I've proven you are wrong in your proclamations. I haven't seen anyone else making your type of claims. On talkchess.com (a well-known chess programming site) no one is talking about 4....a6 in that game. This is all YOUR invention and you are wrong.

I mentioned Stockfish 8 making 4...a6 its second choice (a mere 0.04 points behind 4...Nf6, and 0.04 again is meaningless in a 30-ply deep search) to show that 4...a6 is a totally respectable, playable move. If humans haven't chosen it very much, it says far more about humans than it does about AZ.

Actually, Go was a far bigger challenge then a chess variant like Option chess as was discussed at length in an earlier thread.. I actually wasted some time studying the rules of your silly Option Chess and unless they have changed as I understand it after 8 moves each player is allowed to use 12 tokens issued to them where they can use up a token at at any time to make two consecutive moves instead of one with qualifiers like allowing the opponent to defend against check etc. More tokens are issued at a later stage of the game. So,yes, far more complex then chess but not as complex as Go and it is still a game of complete information with a set of fixed rules unlike a game of incomplete information like Texas Holdem where two cards are hidden.

The algo is designed to be for general purpose zero sum games of complete information. So Paul ,why talk to me about it , go ahead and modify the leela zero code. You complain that you need "option chess masters" first, well ,here is your ideal first student for that.,,you know... that Tabula Rasa thing we have discussed at length in other threads

You will need volunteers to offer up computational resources like with Leela Zero but I suspect that no one involved in this will view the opportunity as "interesting" as you do. They will all be as "closed minded" as I am, in view of the fact that they have more then proven their point with more complex zero sum games of complete information like Go. Initially your variant might have been interesting as a way of playing online where one did not have to worry about a chess engine assisting them.It is now even more obvious then the last thread when we discussed this (when only alphagozero was out) that it is now too late for your variant to have a viable business built around it. The truth is that contrary to your predictions Option Chess does not have wide spread appeal and without the other aspect of always being 100 percent sure that it is impervious to machine assisted cheating it is useless and always will remain that way.

You talk about me getting personal? At least you mercifully spared me in this thread from harassing me about my ethnic origin as is your usual MO but still could not resist trying to insult me. About the only "personal " thing I suggested in this thread is that as a B player it is ridiculous for you to second guess the move choices of a 2800+ rated chess program. This thread is exhausted but no doubt you will have some glib nonsensical self serving reply that I will choose to ignore as with all of your threads has devolved into trolling.

I'm impressed you remember the rules of Option Chess (although you fail miserably to fully understand them). But you are grossly mistaken: I'm not building any business around Option Chess. In fact Option Chess was something I "gifted" to the chess world. I made fully public everything about it because I wasn't going to patent or license it in any way. So you are simply mistaken as usual.

The only thing about Go that makes it more complex than chess is a much bigger game tree. That's it. Option Chess also expands the game tree, and if you did the necessary mathematics, you'd discover that Option Chess game tree is much bigger than even Go. In fact the Option Chess game tree size is beyond calculation. Go ahead, give it a try. I've already been through it with much smarter people than you.

But game tree size isn't the only aspect that makes Option Chess much more challenging. Again, I could go into it but why bother with someone who has a totally closed mind. You've already decided AZ is invincible to any kind of zero sum game, it will conquer them all. My claim is this: in the case of Option Chess, AZ could and likely would become as good as the best humans AND IF IT BECAME BETTER, IT WOULD BE ONLY MARGINALLY BETTER. I'll define "marginally" as within 100 ELO points better.

Now if you were really the AI enthusiast you claim to be, you would want to fully explore that. That's the mark of a true genius in any particular field: someone who eagerly wants to explore all avenues within that field and learn as much as they can, even if a particular avenue seems to them to be questionable. But with you, it's already game over, AZ is guaranteed to conquer all zero sum games of perfect information. You've written off any other possibility. Just like you assumed there were no games of the Giucco Piano played by masters or experts that had 4...a6.

Surprise!!!!

[Sid Belzberg]

So it could easily be TOLD BY PROGRAMMER CODE that when after all that has been done for a particular move, and 2 or more candidate moves are about equal in score, look them up in databases and choose the one that has been least played

No, that is not how the thing works. Here is the statement from the abstract.

Starting from random play and given
no domain knowledge except the game rules, AlphaZero convincingly defeated a world
champion program in the games of chess and shogi (Japanese chess), as well as Go.

It is given NO domain knowledge . In fact it's self play algo involves playing a given position with itself all the way to the end of the game starting with Random moves and then classifying the position based on win lose or draw and assigning a probability to the particular "pattern" to it and adjusting the neural network accordingly and then start the process again. It is looking for general patterns and ranking the probability of the result being a win or loss or draw rather then scoring each individual move or scoring each individual position based on a subjective evaluation function. In other words it does not evaluate based on any of the things a standard chess engine does. It truly is a clean slate!

I have taken the liberty of emailing you the full Alphazero paper. It is truly a great read and hopefully will clarify for you exactly how the thing works.


I have been involved in programming for over thirty years with financial technology projects and was awarded a patent that was for the first product to seamlessly execute trades based on analytics with no human intervention. Larger broker dealers and banks used the product for things like seamless automated cross border interlisted equitiy arbitrage. I did the original beta version for that and later on we hired far better programmers then myself that improved it.
We built our company around that invention in 1993 and sold our interest in it in 2009. Among other things we did in the company was create the first hybrid trading platform (floor and electronic trading) for the CBOE in 2003.

In 2012 we started a company that is heavily involved in AI and in the last several years we also used our domain expertise with enterprise wide secure trading systems to rearchitect the blockchain to allow transactions that are not only secure and truly decentralized but also scalable (thus capable of rapid transactions) the latter being the missing feature with first generation blockchains like bitcoin and ethereum. https://www.ig17.xyz/white-paper. The network can also be used for not only validating transactions but the unused computational resources are ideal for large scale training networks like the volunteer one used for Leela zero.
I also assist in AI projects such as one already mentioned with a relative.


i would also mention that Shogi is more complicated then chess as it uitilizes a bigger board with more pieces and the captured pieces can be reused by the opponent similar to bughouse. The number of possible games is not infinite but estimated to be around 10^226 compared to the Shannon number of around 10^120 possible chess games. The Shannon number, named after Claude Shannon, is a conservative lower bound (not an estimate) of the game-tree complexity of chess of 10^{^120}, based on an average of about 10^3 possibilities for a pair of moves consisting of a move for White followed by one for Black, and a typical game lasting about 40 such pairs of moves. Introducing the two consecutive move option I would guess increases the average to around 10^6 (?) possibilities for a pair of moves and gives us a game tree complexity of 10^240 but that is based on an average of 40 moves in a game of regular chess. In option chess we don't have enough data to know the average number of moves per game but even if it is slightly lower say 30 movers per game keeping in mind that the option possibility for two consecutive moves here does not kick in on move 1 but only until after eight moves and is only available 12 times for the first 40 moves. then you would get a comparable game tree size to Shogi.

None the less the methodology used by Alpha Zero was able to master Shogi. If you have a mathematical proof that shows that your game tree is even bigger then this (infinite???) I would like to see it. I would submit that even if a player had the option to make two consecutive moves anytime they want it would not have a game tree bigger then Shogi's. I do have some relatives that are world class mathematicians at several Universities including Berkley that I would like to share this proof with if you have this.

The idea of a chess variant where an opponent makes multiple consecutive moves is neither new or original. Here is a link to chess variant, Quest Chess, invented by a child hood friend of mine the late Donald Benge that allows each opponent to make 10(!) consecutive moves allowing opponents only to recapture pieces or defend against check. https://www.chessvariants.com/multim.../conquest.html. this was invented back in 1974 and would have an even bigger game tree then Option chess. By your logic AI research should be directed to this instead of your variant as the game tree is bigger. At least this variant did actually have some limited audience that plays it as it is actually a derivative of a game called Conquest that does have a community of some sort that enjoys the game.

As stated before no matter how you want to frame it 4.a6 it is a rarely played move that is now introduced into top level play thanks to Alphazero and its unique methodology that does not involve looking at outside databases in any way shape or form.

If you want to discuss technology i am happy to do it via email as your goal here appears to be only to publicly disparage people and anger them (trolling) and promote option chess that by the way you discussed as a business opportunity numerous times in numerous threads here as well as in private correspondence.

[Erik Malmsten]

No, that is not how the thing works. Here is the statement from the abstract.

Starting from random play and given
no domain knowledge except the game rules, AlphaZero convincingly defeated a world
champion program in the games of chess and shogi (Japanese chess), as well as Go.

It is given NO domain knowledge .
I have taken the liberty of emailing you the full Alphazero paper as well as a bit about my background that hopefully will clarify things. If you want to discuss technical things i would prefer to do it offline.

Thanks Sid for your research. It's amazing that the program has no evaluation for the center or material, it just play only for mate.

[Paul Bonham]

No, that is not how the thing works. Here is the statement from the abstract...

Starting from random play and given
no domain knowledge except the game rules, AlphaZero convincingly defeated a world
champion program in the games of chess and shogi (Japanese chess), as well as Go.

It is given NO domain knowledge . In fact it's self play algo involves playing a given position with itself all the way to the end of the game starting with Random moves and then classifying the position based on win lose or draw and assigning a probability to the particular "pattern" to it and adjusting the neural network accordingly and then start the process again. It is looking for general patterns and ranking the probability of the result being a win or loss or draw rather then scoring each individual move or scoring each individual position based on a subjective evaluation function. In other words it does not evaluate based on any of the things a standard chess engine does. It truly is a clean slate!

I have taken the liberty of emailing you the full Alphazero paper. It is truly a great read and hopefully will clarify for you exactly how the thing works.

...

i would also mention that Shogi is more complicated then chess as it uitilizes a bigger board with more pieces and the captured pieces can be reused by the opponent similar to bughouse. The number of possible games is not infinite but estimated to be around 10^226 compared to the Shannon number of around 10^120 possible chess games. The Shannon number, named after Claude Shannon, is a conservative lower bound (not an estimate) of the game-tree complexity of chess of 10^{^120}, based on an average of about 10^3 possibilities for a pair of moves consisting of a move for White followed by one for Black, and a typical game lasting about 40 such pairs of moves. Introducing the two consecutive move option I would guess increases the average to around 10^6 (?) possibilities for a pair of moves and gives us a game tree complexity of 10^240 but that is based on an average of 40 moves in a game of regular chess. In option chess we don't have enough data to know the average number of moves per game but even if it is slightly lower say 30 movers per game keeping in mind that the option possibility for two consecutive moves here does not kick in on move 1 but only until after eight moves and is only available 12 times for the first 40 moves. then you would get a comparable game tree size to Shogi.

None the less the methodology used by Alpha Zero was able to master Shogi. If you have a mathematical proof that shows that your game tree is even bigger then this (infinite???) I would like to see it. I would submit that even if a player had the option to make two consecutive moves anytime they want it would not have a game tree bigger then Shogi's. I do have some relatives that are world class mathematicians at several Universities including Berkley that I would like to share this proof with if you have this.

The idea of a chess variant where an opponent makes multiple consecutive moves is neither new or original. Here is a link to chess variant, Quest Chess, invented by a child hood friend of mine the late Donald Benge that allows each opponent to make 10(!) consecutive moves allowing opponents only to recapture pieces or defend against check. https://www.chessvariants.com/multim.../conquest.html. this was invented back in 1974 and would have an even bigger game tree then Option chess. By your logic AI research should be directed to this instead of your variant as the game tree is bigger. At least this variant did actually have some limited audience that plays it as it is actually a derivative of a game called Conquest that does have a community of some sort that enjoys the game.

As stated before no matter how you want to frame it 4.a6 it is a rarely played move that is now introduced into top level play thanks to Alphazero and its unique methodology that does not involve looking at outside databases in any way shape or form.

If you want to discuss technology i am happy to do it via email as your goal here appears to be only to publicly disparage people and anger them (trolling) and promote option chess that by the way you discussed as a business opportunity numerous times in numerous threads here as well as in private correspondence.

Sid, I am not trolling you. If you look at this thread you will see that I only started mentioning your history on this board (of being very bullying and overly emotional) AFTER you brought up my B level of chess rating which was meant to be personally critical. You categorized me, so I categorized you.

If it makes you angry that I argue against your views, that isn't something I can help. It doesn't mean that I'm TRYING to anger you. Your reaction is anger and that is something YOU have to deal with. It's very apparent you are an overly emotional person. Just like someone whose name rhymes with "dump" .... but I digress.

No, once again I am NOT promoting Option Chess. Since I released its details publicly in early 2014, I have done absolutely NOTHING to promote it. I am not creating any business around it. You are confusing it with another game that I am creating a business around and which I haven't mentioned at all in this thread.

Now about variants and game trees.... I"m sure there are many variants that allow multiple piece moves per turn. I don't claim Option Chess is the first to have that. But the difference with Option Chess is that Option Chess is the first variant (afaik) to make extra moves optional and additionally to make such options limited in number.

From an AI point of view, this is very very important. I don't think you grasp its importance. Perhaps it would help if I mentioned that saving a double move for the endgame is a worthy goal because double moves even with the restrictions imposed by Option Chess INCREASE IN EFFECTIVENESS as the game approaches the endgame (i.e. as pieces get removed from play). However, it is a risk to try and save as many double moves for the endgame as possible, because you may get mated in the middlegame and never reach the endgame. Perhaps that will make you realize that Option Chess is a delicate balancing of when to use double moves versus when not to use double moves.

AI has trouble dealing with that kind of concept.... the whole idea of a "delicate balancing act" is something AI has trouble with, unless it can calculate everything numerically. Because after all, everything computers do comes down to NUMBERS. That's all computers can work with internally.

Also, let me dispel you of something you wrote above. If a chess variant allowed each player to make double moves without restriction from the beginning of the game, the game tree keeping the assumption of 80 ply average game length WOULD be bigger than Shogi game tree. And the math is very simple in that case:

Assume: average 35 move choices per ply.
Assume: average game length of 80 plies (40 moves per player)
Understand: logBase10(x * x) = 2 logBase10(x)

Size of Shogi game tree
= 10 ^ 226 (according to Wikipedia)

Size of chess game tree after 80 plies as power of 10 with 35 moves per ply
= 10 ^ (80 * logBase10(35))
= 10 ^ 123

Size of chess game tree if double moves allowed on all plies
= 10 ^ (80 * logBase10(35 * 35)
= 10 ^ 247

This is the Shogi game tree * 10 ^ 21. Very simple math, I'm surprised Sid you didn't do this before jumping in with your claim.

Now let's say I decided that we allow for triple moves on a player's turn without limitation.
Now the math becomes

Size of chess game tree if triple moves allowed on all plies
= 10 ^ (80 * logBase10(35 * 35 * 35)
= 10 ^ 370

and this is beyond even the Go game tree, by 10 ^ 10.

But Sid, I want you to think outside of the box. In all the above math, there were assumptions made. I want you to tell me the one assumption that is totally faulty, if you can. Use all your powers of research, intuition, logic etc. and tell me what's wrong with all the above, meaning what assumption is wrong. Not only wrong, but potentially GROSSLY wrong.

I'm going to give you one hint:
Bell Curve.

Edit: Just realized I made a slight mistake in my previous post. I said the Option Chess game tree size is beyond even the Go game tree. I should have said "potentially" beyond even the Go game tree. More on that later.

[Sid Belzberg]

Perhaps that will make you realize that Option Chess is a delicate balancing of when to use double moves versus when not to use double moves.

AI has trouble dealing with that kind of concept.... the whole idea of a "delicate balancing act" is something AI has trouble with, unless it can calculate everything numerically. Because after all, everything computers do comes down to NUMBERS.

Yes, I agree that with any normal chess engine this is a huge problem. In a normal chess engine it is easy to have an evaluation function that looks at things like doubled pawns , control of the center , material count etc. All of the sudden when you add a new thing to evaluate like when to use double moves and when not to this is very difficult to numerically evaluate as you correctly point out. With AZ we do not worry about evaluation functions.

For a general purpose zero sum complete information game engine like Alphazero the only problem as you mentioned (that as you will see below is not much of a problem at all) is simply a size of the game tree as the evaluation function is not a problem because it is only a function of the results the machine gets with self play.

<sup>The game tree for chess is on the order of 10^120+ possible games and the game tree of Go is 10^360+ possible games. It took 4 hours of self training for Alphazero to defeat Stockfish and 72 hours of self training for Alphagozero to defeat Alphago. The difference in training times is a factor of 18 times and yet the game tree size is 10^240 bigger for Go then Chess. If the time relationship for training Go versus chess was proportional to the gametree complexity you would not see results at all with a self training Go engine in our lifetime or for that matter the planet Earth's lifetime! I see the MTCT/NN as a variant of genetic algos and is much more effective then standard brute force hill climbing algos and therefore complexity of the game tree size is not nearly as big a factor as it would be with traditional chess engines, Shogi engines etc.

The reason it works is is that evolution whether naturally occurring or machine made is very good at zeroing in on relevant lines only. That is why a fully trained Alphazero looks at only 80000 positions per second versus Stockfish 80,000,000 positions per second and still wins by large margins.

I remember having an interesting discussion about 15 years ago with AI Chess researcher/pioneer David Levy president of the International Computer Games association (his successor will be AI/Chess/Poker pioneer U of Alberta's Dr Jonathan Schaeffer in August 2019) about deploying Genetic algos for chess engines. Even at that time he said they actually had some degree of success using genetic algo's with backgammon that had a much smaller game tree then chess at a time when the technology was not there for the size of game tree complexity with chess. It took the innovation of having MCTS's combined with Deep Convolutional Neural networks and armies of networked computational resources to make the dealing with enormous game trees of almost any size a reality.

I don't see any useful purpose served in mastering option chess using AZ style technology as other then game tree size it has nothing new to offer. The evaluation problems you speak of are not there with AZ. As stated above the game tree size has proven to be effectively dealt with, else the time gap between mastering Go and Chess would be near infinite. Moreover in games like Go Chess and Shogi it was easy to benchmark the program against the best. Option chess does not have enough players if any to benchmark the result and the days of traditional game engines with human input evaluators to benchmark against are now over.
Here is an example of an insect robot that teaches itself to walk using Genetic Algo's.</sup>
https://www.youtube.com/watch?v=KHV7fWvnn_0

[Paul Bonham]

Yes, I agree that with any normal chess engine this is a huge problem.....

Sid, thanks for sending me the latest whitepaper. So I looked it over and found what I was looking for: an explanation as to why AZ was playing opening "novelties". Here are the relevant sections:
references section:
28. The Stockfish variant used the Cerebellum opening book downloaded from https://zipproth.de/#Brainfish. AlphaZero did not use an opening book.
To ensure diversity against a deterministic opening book, AlphaZero used a small amount of randomization in its opening moves (10); this avoided
duplicate games but also resulted in more losses.

This further referenced a "supplementary" paper, which I downloaded, and I found an even better explanation there:
(section "Opponents"):
"Stockfish does not have an opening book of its own and all primary evaluations were performed without an opening book. We also performed one
secondary evaluation in which the opponent’s opening moves were selected by the Brainfish program, using an opening book derived from Stockfish.
However, we note that these matches were low in diversity, and AlphaZero and Stockfish tended to produce very similar games throughout the match,
more than 90% of which were draws. When we forced AlphaZero to play with greater diversity (by softmax sampling with a temperature of 10.0 among
moves for which the value was no more than 1% away from the best move for the first 30 plies) the winning rate increased from 5.8% to 14%."

I knew they had to use some mechanism to change up opening moves in order to vary the games. I mentioned the possibility that they did a database search and chose the least played move that remained within a scoring range, but they describe something else. Nevertheless, this certifies that these "novelties" are NOT chosen because they are decidedly better than what humans and other engines have been playing. So 4...a6 in the Giucco Piano was not such a great move, it was a DIFFERENT move. For Stockfish 8 4...a6 was only 0.04 points worse than 4...Nf6, so that indicates AZ would consider it a "chooseable" move based on the specifications above.


On your points about game tree size, yes, AZ has proven that it can adjust magnificently to increased game size... "as far as we know". I had to add that because first of all, I don't know the games Shogi nor Go, so I can't say anything about whether AZ has actually "mastered" those games. AZ is the best in those 2 games as of now, but what if the bar is really low? What if the human players and the engines before AZ are actually not playing all that great to begin with? With such unimaginable game tree sizes, who is to say?

It is even difficult to say that AZ has "mastered" chess. Do we REALLY know that?

You quoted the chess game tree size which is the typical number given.... what if it is wrong... by several orders of magnitude?

The shortest game of chess possible is 4 plies, the Fool's Mate. The longest game of chess possible (because of 50 move rule) is 5898 moves! What if it turned out that if we actually had a database of all possible unique and legal chess games, these two extremes would be the beginning and end points of a Bell curve plotting length in plies of each game, and the midpoint of the Bell curve was (5898 -4) / 2 = 2947 plies.

Yes, 2947 plies INSTEAD OF the number assumed, 80 plies!

Where did that number of 80 plies come from? Probably a sampling of human games. But humans make mistakes, humans trade off pieces easily, and even today's minimax engines do the same... what if the sampling of all unique and legal chess games we have right now is the extreme low end of the spectrum?

It could mean that even in chess, AZ is merely "mastering" play against the very very very low end all possible opponents. Magnus Carlsen has gained a reputation for playing out to "extreme" game lengths when a game seems drawn. Could there be engines in the future that will do this to AZ and find ways to beat it in 300 ply games? And other engines that would beat THOSE engines in 3000 ply games?

You see, Sid, this is thinking out of the box.

I don't see any useful purpose served in mastering option chess using AZ style technology as other then game tree size it has nothing new to offer. The evaluation problems you speak of are not there with AZ. As stated above the game tree size has proven to be effectively dealt with, else the time gap between mastering Go and Chess would be near infinite. Moreover in games like Go Chess and Shogi it was easy to benchmark the program against the best. Option chess does not have enough players if any to benchmark the result and the days of traditional game engines with human input evaluators to benchmark against are now over.....

I totally dispute that Option Chess only offers increase in game tree size (compared to chess) to the AI community. Poker is full of calculations known as EV (Expected Value) and Option Chess brings in those type of calculations that don't get used in chess. That is to say, there might someday be a calculation for EV of playing an option on move N of opening XYZ versus saving that option for a later time. Such calculations would have to know how to evaluate the option itself, both now and in the future stages of the game.

There are many life choices we humans make related to this. Graduating high school students decide whether to pursue an undergraduate degree now or work / travel a few years first. Couples decide whether to marry / have kids now or wait a few years or even longer. Chess has no analogy to this kind of thinking. There is no calculation of "what if I play Nf3 later" because later Nf3 might not even be possible... the knight could be captured or pinned or f3 might be occupied by a friendly piece....but in Option Chess, options are always possible if you have the option tokens.

[Sid Belzberg]

Sid, thanks for sending me the latest whitepaper.

My pleasure, I hope you found it enjoyable.

"We also performed one secondary evaluation in which the opponent’s opening moves were selected by the Brainfish program, using an opening book derived from Stockfish." Stockfish does not have an opening book of its own and all primary evaluations were performed without an opening book. We also performed one
secondary evaluation in which the opponent’s opening moves were selected by the Brainfish program, using an opening book derived from Stockfish.
However, we note that these matches were low in diversity, and AlphaZero and Stockfish tended to produce very similar games throughout the match,
more than 90% of which were draws. When we forced AlphaZero to play with greater diversity (by softmax sampling with a temperature of 10.0 among
moves for which the value was no more than 1% away from the best move for the first 30 plies) the winning rate increased from 5.8% to 14%."

The deliberate randomization of moves at early stages appeared to be only as part of a secondary evaluation. In any case this is exactly how a genetic algo works where the best solutions are only slightly altered ('mutated") and tested for "fitness again". Moreover, no outside domain knowledge was used as advertised. Whatever the cause the methodology has produced many interesting novelties at all stages of the game including a6 even if it came about as being a ( "different move") .

. So 4...a6 in the Giucco Piano was not such a great move, it was a DIFFERENT move. For Stockfish 8 4...a6 was only 0.04 points worse than 4...Nf6, so that indicates AZ would consider it a "chooseable" move based on the specifications above.

I would not expect any human whether D player or super grandmaster to second guess or judge either a 3300(!) rated stockfish or (X rated :-)) Az's move choices. We are not in that league and never will be. Apparently Stockfish's evaluation failed as AZ trounced stockfish with 4...a6 so i would say that AZ success and actual result is a better barometer of whether or not the move is good as opposed to Stockfish's evaluation.

That is to say, there might someday be a calculation for EV of playing an option on move N of opening XYZ versus saving that option for a later time. Such calculations would have to know how to evaluate the option itself, both now and in the future stages of the game.

Maybe so but the AZ methodology does not work that way. More likely just like in all the other games it might zero in on patterns for example where exercising options are saved for the ending (if probabilities based on self play showed that this is advantageous ) and would not bother with expected value calcs at all. It would simply give this pattern a higher place in the hierarchical rankings.In a game like Poker i expect the reason the Az methodology has had consistent success in heads up poker but not multiplayer table poker is in heads up the thing can find patterns through it's opponents historical pattern of betting decisions. In option chess AZ would have no problem finding patterns and moreover unlike poker it is a game of complete information.

Where did that number of 80 plies come from? Probably a sampling of human games. But humans make mistakes, humans trade off pieces easily, and even today's minimax engines do the same... what if the sampling of all unique and legal chess games we have right now is the extreme low end of the spectrum?

The research is based on what we are aware of, not the what ifs of the uncharted universe or improbable scenarios. Random moves through self play showed that in all three games the average game size whether its 40 moves in chess, 70 moves in Shogi or 250 moves in Go has a very high probability (approaching 100 percent!) of being correct as the self play mode started with random moves that evolved and optimized to an average game size with best play.

So your idea that the game size is bigger because play has not been optimized is not "thinking out of the box" it is just wrong. By the way I did mention that I made assumptions about the Option chess game size knowing that not enough data if any exists to know.Submitting Optionchess to a AZ self play algo would answer that question. As mentioned I doubt if the Leela Zero community would be interested in volunteering their computational resources and time to make this happen for the reasons I have previously stated.

"DeepMind's Demis Hassabis, a chess player himself, called AlphaZero's play style "alien": It sometimes wins by offering counterintuitive sacrifices, like offering up a queen and bishop to exploit a positional advantage. "It's like chess from another dimension."^[8]

"Danish grandmaster Peter Heine Nielsen likened AlphaZero's play to that of a superior alien species."

[Paul Bonham]

Originally posted by Paul Bonham
That is to say, there might someday be a calculation for EV of playing an option on move N of opening XYZ versus saving that option for a later time. Such calculations would have to know how to evaluate the option itself, both now and in the future stages of the game.

Maybe so but the AZ methodology does not work that way. More likely just like in all the other games it might zero in on patterns for example where exercising options are saved for the ending (if probabilities based on self play showed that this is advantageous ) and would not bother with expected value calcs at all. It would simply give this pattern a higher place in the hierarchical rankings.In a game like Poker i expect the reason the Az methodology has had consistent success in heads up poker but not multiplayer table poker is in heads up the thing can find patterns through it's opponents historical pattern of betting decisions. In option chess AZ would have no problem finding patterns and moreover unlike poker it is a game of complete information.

The example you give of using a pattern of saving all options for the endgame would imo fail against a strong player who can make good strategical (positional) moves using options in the middlegame. In essence the endgame that the AZ engine was expecting would never come. But of course, that is but one example. It would be of great interest to the field of AI as to exactly what patterns AZ would actually pick up on. We know that a minimax engine would want to use all options immediately, because it wouldn't be seeing through to the game's conclusion. It would see N plies ahead, where N is maybe 30, and consider that both sides would be playing their options to maximize their immediate positional strength. How exactly would AZ cope against that?

I'm saying that this would be of great interest in the general field of AI, because we humans tend to be faced with "should I do this now or do it later" kind of situations and would like some insight into any patterns no matter how nebulous dealing with that question. Of course, taking chess analysis and extrapolating that to general life situations is not exactly easy nor correct, but even if we keep everything in the chess context, it would be interesting to see what AZ does. There could be many patterns we could learn. Also, I would say that if AI is ever to get a grasp of general real world situations versus contrived perfect-information games, it must deal with this "now or later" dilemma, and so AI researchers should be on that track asap.



Originally posted by Paul Bonham
Where did that number of 80 plies come from? Probably a sampling of human games. But humans make mistakes, humans trade off pieces easily, and even today's minimax engines do the same... what if the sampling of all unique and legal chess games we have right now is the extreme low end of the spectrum?

The research is based on what we are aware of, not the what ifs of the uncharted universe or improbable scenarios. Random moves through self play showed that in all three games the average game size whether its 40 moves in chess, 70 moves in Shogi or 250 moves in Go has a very high probability (approaching 100 percent!) of being correct as the self play mode started with random moves that evolved and optimized to an average game size with best play.

So your idea that the game size is bigger because play has not been optimized is not "thinking out of the box" it is just wrong....

Wait, are you privy to some information about the self play of AZ? Do you have information that in its internal self play, when it was optimizing lines of play, it produced complete games of a particular move length (say 80 or 90 plies)? If yes, do you even have some lines of self play from its internal calculations? This would be VERY interesting for everyone to see if you have it, because it would be in effect seeing AZ playing against AZ.

If you don't have such information, then you are making baseless claims.

But even if such information were to come out, that AZ versus AZ in internal self play gives typical game lengths of N plies where N is some reasonable number like 80 or 90, even 100, there is still a fault with your argument. You are basing everything on AZ being the strongest chess engine ever, even into the future. You are using the conclusions of your own argument to prove your argument. It is like Donald Trump saying that IF we spend $5 billion on a Mexican border wall, Americans will be safer BECAUSE the wall will reduce crime in America. There is no proof that the wall will reduce crime, that is a conjecture. But Trump uses it to say we need the wall. There is no proof of that conjecture until the money is spent... and it could turn out false.

AZ may turn out in some future world to have been merely a stepping stone towards the strongest chess engine imaginable. Therefore if AZ is playing itself and game length is say 90 plies average, that could still turn out to be the very extreme low end of what chess engines will play in the future.

I'm not claiming I am absolutely correct but.... you haven't proven me wrong at all, and there is no way you can prove me wrong. You are not capable of thinking out of the box. You are boxed in by your beliefs and biases.