Chess Mastered with a Self-Play Algorithm

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Hi Sid,
I'd really like to discuss all this with you. I am in agreement with some of your points, but not all of them -- the ones I am not in agreement with are just because I think there needs to be more evidence, not because I think it just can't be so.

But first, you shouldn't be surprised that Vlad Drkulec is opposed to the idea that science is conquering vast areas of what was once considered sacrosanct domains of human thought. If AlphaZero were to teach itself climate change science and conclude that man is changing the climate just as 90%+ of climate scientists are claiming, Vlad would dismiss it all as voodoo science and walk away to go read his Flat Earth Society journals. He is resistant to the very idea that science can tell us anything. Nevermind all the evidence from Los Angeles, Houston, Puerto Rico, Miami, Antarctica, India, China, the Far East, etc. Nevermind all the "once in 500 years" storms that are ravaging the planet in just the past few years.

So enough about him. Let's dismiss him the way he dismisses science.

I do think AZ represents a fundamental paradigm shift going on. For years, I argued about the potential of neural net programming for machine learning, and not many were listening. The technology just wasn't there, plus it requires LOTS of training data. I am super happy that people are now working on such systems as AlphaZero. Just the fact that it can completely learn Go or chess in just a few hours is telling us something dramatic is going on. No one can deny that.

But I do have to be skeptical about some claims. First, the one above that "Go was magnitudes more difficult to master then chess". I don't think this can be proven. Yes, it's true that HUMANS have had more trouble with Go than with chess. But that may actually indicate something problematic about the human brain. Now, what could have happened is that AlphaZero studied thousands of Go games betweeen top-level players and discovered something simple that has eluded humans, some principle that is either too complicated OR TOO SIMPLE for humans to get. And it used that information to defeat the Go masters. I'm just saying it's possible, and we have to (as scientists) account for it. So let's leave the Go accomplishments of AlphaZero out of the picture, it's too problematic on that basis.

But the AlphaZero 4-hour mastery of chess and defeating of (a crippled) Stockfish is stunning, no bones to pick there.

What needs to happen now is for the AlphaZero developers to go way beyond chess and Go. These are games of perfect information, and as long as they stick with those games, AZ will be criticized for being limited to perfect information domains in general.

They need to prove that AZ can be applied to stock trading just as easily as to chess, and can earn millions of dollars a year just trading on the major markets as a human would do. It needs to be consistent, over good days and bad. And it needs to teach itself from zero.

EDIT: think about this for a moment: imagine that AZ does conquer stock trading and vastly outperforms humans. Then imagine humans en masse buy a home version of AZ for stock trading and millions of AZ engines log into stock trading platforms and start trading.... ALL of them making money. What do you think would happen? Why, all trading would cease probably within hours, there would be no sellers. All the human sellers would be gone, their accounts emptied of funds. The engines wouldn't trade with each other because they are all geared to making money.




You are of course referring to my anti-computer and anti-cheating chess variant called Option Chess.

The point about Option Chess, and also about poker, is that it's one thing to have an engine that can learn the game and win a game here or a game there, even against masters / pros. That is a great accomplishment, but in the case of poker, we already have numerous examples of virtual noobs at the game winning major tournaments against poker pros. It happens quite often -- TOO often in fact, making poker pros look bad.

So merely winning the odd event falls short of what really would be desired: total domination of humans by the learning engine. And THAT is something that I claim will not happen for decades to come. Perhaps quantum computers could do it, but not current technology.

So with regards to Option Chess, my claim remains that no engine in the next 40 years will dominate a trained or expert human player. I'll define "dominate" to mean winning 2 out of every 3 decisive games (draws would be ignored). If my claim is true, that would mean that even the best Option Chess engine over those 40 years would not be useful for computer cheating in Option Chess against top level human opposition.

I'm open to have that claim disproven, but who can be the "trained, expert human"? So let me amend this a bit: if Stockfish or Komodo or Fritz or any top-10 computer engine were modified to play Option Chess and play it at a high level as it does regular chess, I would extend this claim to have the "trained, expert human" replaced by such an engine, and to dispel all doubts, the two engines AZ and Stockfish, Komodo, whatever, would have to be playing on hardware that is considered near identical (recognizing that TPUs are not the same beast as CPUs nor GPUs).

And I would remain confident that AZ would not dominate that engine under those conditions. In fact, I would predict a nearly 50% split of the games.

The reason? Neither AZ nor the opposing engine would be able to work out with consistent accuracy the one piece of hidden information that is inherent to Option Chess: the Current Value of an unused Option token. In fact, I remarked to someone the other day, the Current Value of such a token is like the "Current Value" of a Schrodinger Wave Equation: in order to measure it, you have to collapse the wave. For the Option token, that means actually playing the option. If you keep it for later, the wave goes on and Current Value can only be guessed at. The point being, how would the engine know whether or not to keep the token for later use? It can't put a value on that. It could try different values, but the problem is, the value would change with every situation. In the end, I assert that the engine would not know any better than the human or Stockfish opponent whether or not to save the token for later use.

So merely winning the odd event falls short of what really would be desired: total domination of humans by the learning engine. And THAT is something that I claim will not happen for decades to come. Perhaps quantum computers could do it, but not current technology.

You could not be more incorrect about poker. A program from the University of Alberta (my old alma mater) called deep stack uses the same methodology as alphago where it taught itself Tabula Rasa over a one month period and then played millions of hands against 11 seasoned pros and wiped them all out over a one month period. Here is a link. Read the article carefully, it is the total antithesis of what you state. Requiring Quantum computers to solve this problem is nonsense. The problem has already been solved solely on the basis of software methodologies and solved in polynomial time.

https://www.scientificamerican.com/a...hold-rsquo-em/

I'm open to have that claim disproven, but who can be the "trained, expert human"?

Again, this statement only underscores that you clearly do not know what you are talking about. You really need to read up more about MCTS and deep neural networks. They teach themselves starting with a clean slate. Several open source projects exist for alpha go zero and no doubt a more generic version like alpa zero will soon follow. Simply take the code and modify the rules for option chess and let it learn over a one month period (yes they play against themselves using the MCTS). i will give you a different challenge, if it comes to pass that you modify the code for option chess you will not be able to find any human on the planet to beat it. You could stake $500,000 and you would find no takers. The good news is the only code you would modify is the basic rules that the thing starts with.
Everything else you have stated is irrelevant and flatly disproved by the existence of deep stack. By the way thanks to Garland Best i have totally accepted the pilot wave theory and completely reject the idea that particles have dual properties of wave and particles where waves suddenly collapse back into particles.
I am sorry if I am brutal about this but I think the opportunities for online chess variants for money is over and you would be wise investing your money and time elsewhere. That is probably a bitter pill to swallow but I speak the truth.

[Paul Bonham]

Re: Chess Mastered with a Self-Play Algorithm

You could not be more incorrect about poker. A program from the University of Alberta (my old alma mater) called deep stack uses the same methodology as alphago where it taught itself Tabula Rasa over a one month period and then played millions of hands against 11 seasoned pros and wiped them all out over a one month period. Here is a link. Read the article carefully, it is the total antithesis of what you state. Requiring Quantum computers to solve this problem is nonsense. The problem has already been solved solely on the basis of software methodologies and solved in polynomial time.

https://www.scientificamerican.com/a...hold-rsquo-em/

Just as you can point to that as evidence of your views, I can point to countless poker tournaments where a virtual newcomer has done the same thing. Which means that in the game of poker, this is quite common. It would only be meaningful if the poker algorithm were winning say 50% of the hands against those 11 other players. But no, it takes millions of hands for it to come out ahead. And I remember at the time that there was a more in-depth article that explained that the play of that poker event was altered to make allowances for the poker engine, so that the pros weren't playing quite the game they were used to playing. I can't remember the details, and the article you linked to doesn't give any details at all.

Again, this statement only underscores that you clearly do not know what you are talking about. You really need to read up more about MCTS and deep neural networks. They teach themselves starting with a clean slate. Several open source projects exist for alpha go zero and no doubt a more generic version like alpa zero will soon follow. Simply take the code and modify the rules for option chess and let it learn over a one month period (yes they play against themselves using the MCTS). i will give you a different challenge, if it comes to pass that you modify the code for option chess you will not be able to find any human on the planet to beat it. You could stake $500,000 and you would find no takers. The good news is the only code you would modify is the basic rules that the thing starts with.

Wouldn't I need a $4 million TensorFlow cluster to play it on? Or you are saying this will be available for home desktop PCs? Well, you can claim all you want that no human could beat it at Option Chess, but you can't prove it, so it is meaningless. When that day comes, we shall see. Until then, it's just the two of us giving our differing opinions.

Also, based on what you are saying, then options trading (a completely zero sum game) will disappear once a version of AZ can learn to do it. So let's make that YOUR challenge. Modify AZ to do options trading and release it on the markets. The entire industry should disappear virtually overnight. Until that day comes, you are just pissing in the wind.

Everything else you have stated is irrelevant and flatly disproved by the existence of deep stack. By the way thanks to Garland Best i have totally accepted the pilot wave theory and completely reject the idea that particles have dual properties of wave and particles where waves suddenly collapse back into particles.
I am sorry if I am brutal about this but I think the opportunities for online chess variants for money is over and you would be wise investing your money and time elsewhere. That is probably a bitter pill to swallow but I speak the truth.

I don't think of it as brutal, I think of it as just your opinion, at the moment not worth the screen pixels it's written on, but nevertheless duly noted. We aren't wasting our money and we will prove it. All indicators are positive. Your vaunted poker engine hasn't stopped online poker, that is still going strong. And why, because it takes millions of hands for the poker engine to come out slightly ahead. Plenty of opportunity for humans to win money over those millions of hands. Heck, people still play Blackjack for money, their odds are far worse there. I am taking your pronouncements as just you being a drama queen.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

Wouldn't I need a $4 million TensorFlow cluster to play it on?

Do you ever read anything before you type? Here is what the Chessbase article said.

Stockfish needs no introduction to ChessBase readers, but it's worth noting that the program was on a computer that was running nearly 900 times faster! Indeed, AlphaZero was calculating roughly 80 thousand positions per second, while Stockfish, running on a PC with 64 threads (likely a 32-core machine) was running at 70 million positions per second. To better understand how big a deficit that is, if another version of Stockfish were to run 900 times slower, this would be equivalent to roughly 8 moves less deep. How is this possible?

In any event if you ever need to test something on a tensorFlow cluster for free here is the place to do it
https://www.tensorflow.org/tfrc/

Just as you can point to that as evidence of your views, I can point to countless poker tournaments where a virtual newcomer has done the same thing. Which means that in the game of poker, this is quite common. It would only be meaningful if the poker algorithm were winning say 50% of the hands against those 11 other players. But no, it takes millions of hands for it to come out ahead. And I remember at the time that there was a more in-depth article that explained that the play of that poker event was altered to make allowances for the poker engine, so that the pros weren't playing quite the game they were used to playing. I can't remember the details, and the article you linked to doesn't give any details at all.

Your ignorance of this subject is incredible. Here is the academic paper on Deep Stack. Read it so at least you are not talking out of your asshole.
https://arxiv.org/pdf/1701.01724.pdf

Also, based on what you are saying, then options trading (a completely zero sum game) will disappear once a version of AZ can learn to do it. So let's make that YOUR challenge. Modify AZ to do options trading and release it on the markets. The entire industry should disappear virtually overnight. Until that day comes, you are just pissing in the wind.

I recently befriended a developer who works for a very large bank that is doing just that.


You are sending good money after bad.

[Patrick Kirby]

Re: Chess Mastered with a Self-Play Algorithm

You are of course referring to my anti-computer and anti-cheating chess variant called Option Chess.

The point about Option Chess, and also about poker, is that it's one thing to have an engine that can learn the game and win a game here or a game there, even against masters / pros. That is a great accomplishment, but in the case of poker, we already have numerous examples of virtual noobs at the game winning major tournaments against poker pros. It happens quite often -- TOO often in fact, making poker pros look bad.

So merely winning the odd event falls short of what really would be desired: total domination of humans by the learning engine. And THAT is something that I claim will not happen for decades to come. Perhaps quantum computers could do it, but not current technology.

So with regards to Option Chess, my claim remains that no engine in the next 40 years will dominate a trained or expert human player. I'll define "dominate" to mean winning 2 out of every 3 decisive games (draws would be ignored). If my claim is true, that would mean that even the best Option Chess engine over those 40 years would not be useful for computer cheating in Option Chess against top level human opposition.

I'm open to have that claim disproven, but who can be the "trained, expert human"? So let me amend this a bit: if Stockfish or Komodo or Fritz or any top-10 computer engine were modified to play Option Chess and play it at a high level as it does regular chess, I would extend this claim to have the "trained, expert human" replaced by such an engine, and to dispel all doubts, the two engines AZ and Stockfish, Komodo, whatever, would have to be playing on hardware that is considered near identical (recognizing that TPUs are not the same beast as CPUs nor GPUs).

And I would remain confident that AZ would not dominate that engine under those conditions. In fact, I would predict a nearly 50% split of the games.

The reason? Neither AZ nor the opposing engine would be able to work out with consistent accuracy the one piece of hidden information that is inherent to Option Chess: the Current Value of an unused Option token. In fact, I remarked to someone the other day, the Current Value of such a token is like the "Current Value" of a Schrodinger Wave Equation: in order to measure it, you have to collapse the wave. For the Option token, that means actually playing the option. If you keep it for later, the wave goes on and Current Value can only be guessed at. The point being, how would the engine know whether or not to keep the token for later use? It can't put a value on that. It could try different values, but the problem is, the value would change with every situation. In the end, I assert that the engine would not know any better than the human or Stockfish opponent whether or not to save the token for later use.

I don't know anything about your chess variant. But the Alpha Zero algorithm is basically to play out the current board position x number of times and choose the move that gives the highest score. Why wouldn't that method be applicable to your variant?

[Mathieu Cloutier]

Re: Chess Mastered with a Self-Play Algorithm

I don't know anything about your chess variant. But the Alpha Zero algorithm is basically to play out the current board position x number of times and choose the move that gives the highest score. Why wouldn't that method be applicable to your variant?

Agreed 100%. AlphaZero would actually destroy anyone at Option Chess. Just give it the rules and it will train itself. In fact, AlphaZero is THE algorithm to play such silly games.

But I think it's wiser to leave Paul Bonham alone with his delusions.

[Vlad Drkulec]

Re: Chess Mastered with a Self-Play Algorithm

I am sorry if I am brutal about this but I think the opportunities for online chess variants for money is over and you would be wise investing your money and time elsewhere. That is probably a bitter pill to swallow but I speak the truth.

There are consequences for waiting too long.

Bonham speaks nonsense most of the time and he writes dozens of paragraphs where a simple sentence or two would suffice. He is about to be punished for all that wasted time.

I am not nay saying Alpha Zero's accomplishment. I am excited about its application to climate science where the machine would be able to create its own predictive models and tell us free of bias what the actual situation is.

This is the Savior Machine (David Bowie for the great unwashed), Skynet, Samaritan, the AI from Person of Interest and Hal the computer rolled into one. I'm sure there are people applying these techniques to the stock market already. What about an entity that can take the metadata and even all the data from the surveillance state and predict Isis and crazy attacks and send operatives to neutralize them before they can unleash havoc. What does this mean for politics and advertising? Do we need an amendment to allow the AI to run for office? What would this take for the AI to become self aware and start working on its own problems rather than the trivial question of chess and any other question of interest to mankind.

[Louis Morin]

Re : Re: Chess Mastered with a Self-Play Algorithm

AlphaZero is THE algorithm to play such silly games.

It is not a silly game. It was actually fun to play, and very challenging.

[Wayne Komer]

Re: Chess Mastered with a Self-Play Algorithm

Chess Mastered with a Self-Play Algorithm

December 8, 2017

Chess.com has an article by Peter Doggers entitled:

AlphaZero: Reactions From Top GMs, Stockfish Author

The first few paragraphs – please see the video:

The news about AlphaZero beating Stockfish 64-36 without a single loss after just four hours of self training has shocked the chess world. Chess.com has early reactions from the London Chess Classic participants and from one of the original authors of Stockfish.

It was Wednesday morning when the paper about DeepMind's latest AlphaZero success started going around on Twitter and elsewhere, when the participants of the London Chess Classic were in the middle of their preparation. Some of them heard about it, and even quickly looked at a few games from the AlphaZero-Stockfish match. Others only heard about it after finishing their games.

At the end of the fifth round Chess.com asked eight participants for their first, quick reaction. Here's a video with top grandmasters Michael Adams, Levon Aronian, Fabiano Caruana, Sergey Karjakin, Hikaru Nakamura, Ian Nepomniachtchi, Wesley So and Maxime Vachier-Lagrave.

https://www.chess.com/news/view/alph...ockfish-author

And this comment:

Meanwhile Chess.com also received a lengthy comment from one of the original Stockfish authors, Tord Romstad, which we'll give in full:

The match results by themselves are not particularly meaningful because of the rather strange choice of time controls and Stockfish parameter settings: The games were played at a fixed time of 1 minute/move, which means that Stockfish has no use of its time management heuristics (lot of effort has been put into making Stockfish identify critical points in the game and decide when to spend some extra time on a move; at a fixed time per move, the strength will suffer significantly).

The version of Stockfish used is one year old, was playing with far more search threads than has ever received any significant amount of testing, and had way too small hash tables for the number of threads. I believe the percentage of draws would have been much higher in a match with more normal conditions.

On the other hand, there is no doubt that AlphaZero could have played better if more work had been put into the project (although the "4 hours of learning" mentioned in the paper is highly misleading when you take into account the massive hardware resources used during those 4 hours). But in any case, Stockfish vs AlphaZero is very much a comparison of apples to orangutans. One is a conventional chess program running on ordinary computers, the other uses fundamentally different techniques and is running on custom designed hardware that is not available for purchase (and would be way out of the budget of ordinary users it it were).

From another perspective, the apples vs orangutans angle is the most exciting thing about this: We now have two extremely different (both on the hardware and the software side) man-made entities that both display super-human chess playing abilities. That's much more interesting than yet another chess program that does the same thing as existing chess programs, just a little better. Furthermore, the adaptability of the AlphaZero approach to new domains opens exciting possibilities for the future.

For chess players using computer chess programs as a tool, this breakthrough is unlikely to have a great impact, at least in the short term, because of the lack of suitable hardware for affordable prices.

For chess engine programmers -- and for programmers in many other interesting domains -- the emergence of machine learning techniques that require massive hardware resources in order to be effective is a little disheartening. In a few years, it is quite possible that an AlphaZero like chess program can be made to run on ordinary computers, but the hardware resources required to _create_ them will still be way beyond the budget of hobbyists or average sized companies. It is possible that an opening source project with a large distributed network of computers run by volunteers could work, but the days of hundreds of unique chess engines, each with their own individual quirks and personalities, will be gone.

[Paul Bonham]

Re: Chess Mastered with a Self-Play Algorithm

I don't know anything about your chess variant. But the Alpha Zero algorithm is basically to play out the current board position x number of times and choose the move that gives the highest score. Why wouldn't that method be applicable to your variant?

That method WOULD be applicable, AZ COULD learn to play this variant if supplied with the rules. What I'm claiming is that neither AZ nor Stockfish nor Houdini nor any other engine will be able to dominate the best human players in matches of Option Chess. "Dominate" means win 2 out of every 3 decisive games. I don't even think they could win 3 out of 5 decisive games.

In a nutshell, Option Chess gives each player a total of 16 chances throughout the game to play a double move on their turn, with restrictions: the first move can't give check nor capture material, and if you move the same piece twice, it can't capture on either move (although it can throw a check on the 2nd move).

Each player has to decide when to use the 16 options. If you use them all right away, leaving none for the endgame, you'd better win the game early or win lots of material, because the options become more important / powerful later on in the game as pieces get captured and tempos become more of a decisive factor. The problem for both humans and for engines is how to decide whether now is a good time to use an option versus leaving it for later use. The value of the later use can't be known, it is hidden information (hidden to both players). Since everything computers do is based on numerical values (even text characters have numerical values), they have trouble with hidden information. They need to know a value for a future use of an option. If they don't know that value, they can only guess, which is exactly what humans would do.

The idea that AZ would learn (from playing itself many times) to correctly guess the future value of an option better than any human every time is preposterous. It is like guessing in December how many goals Alex Ovechkin will score in a particular hockey game in March. He might be injured that night and not even play. He might feel like Superman that night and score 10 goals. The safest guess would be something like 1 goal. But any human can make that same guess. Or the human might guess 0 goals, and it's a 50/50 split as to whether the human or the computer is correct when March comes around.

So what will happen in Option Chess is, whenever it is AZ's turn to play, it will guess what the future value of its remaining options are. Then it will asses the current value of playing an option right now, using the same techniques it would calculate the value of playing any other move. If the current value is better than its guess of future value, it will play the option now, otherwise it will hang onto the option. The human will do the same thing on his or her turn, and it cannot be predicted with any consistent accuracy which player guesses better.

Most often the engine will choose to play options whenever it has the chance. It will likely use all 16 options in 20 moves or less. A clever human player could defeat the engine if s/he could use less options initially, saving them for the endgame, and survive until the endgame and then come back by using his or her extra options.

So my claim is that the engines will all, including AZ, be about as good as the best humans at Option Chess and will not dominate the best humans. That's better than the current situation in regular chess, where there is no hidden information and the computers can assign accurate numbers to everything and dominate the humans.

Sid is making lots of bluster about how AZ is going to dominate humans in every task it undertakes, but he's got nothing backing that up other than games like chess and go. These are games of perfect information. So far, AZ has not shown it can do anything extraordinary with hidden information, such as in options trading. Every options trade involves guessing at some future value of something. If you guess right you win, if you guess wrong you lose. The best human options traders can make good money, and for every dollar they make, some other human is losing a dollar. Therefore, if AZ were to take over options trading and always guess all future values correctly, it would wipe out the entire industry, practically in a matter of days. Nothing like this has happened or will happen. If it does happen someday, then Sid can claim victory, but until then, he's just full of sound and fury, signifying nothing.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

That method WOULD be applicable, AZ COULD learn to play this variant if supplied with the rules. What I'm claiming is that neither AZ nor Stockfish nor Houdini nor any other engine will be able to dominate the best human players in matches of Option Chess. "Dominate" means win 2 out of every 3 decisive games. I don't even think they could win 3 out of 5 decisive games.

In a nutshell, Option Chess gives each player a total of 16 chances throughout the game to play a double move on their turn, with restrictions: the first move can't give check nor capture material, and if you move the same piece twice, it can't capture on either move (although it can throw a check on the 2nd move).

Each player has to decide when to use the 16 options. If you use them all right away, leaving none for the endgame, you'd better win the game early or win lots of material, because the options become more important / powerful later on in the game as pieces get captured and tempos become more of a decisive factor. The problem for both humans and for engines is how to decide whether now is a good time to use an option versus leaving it for later use. The value of the later use can't be known, it is hidden information (hidden to both players). Since everything computers do is based on numerical values (even text characters have numerical values), they have trouble with hidden information. They need to know a value for a future use of an option. If they don't know that value, they can only guess, which is exactly what humans would do.

The idea that AZ would learn (from playing itself many times) to correctly guess the future value of an option better than any human every time is preposterous. It is like guessing in December how many goals Alex Ovechkin will score in a particular hockey game in March. He might be injured that night and not even play. He might feel like Superman that night and score 10 goals. The safest guess would be something like 1 goal. But any human can make that same guess. Or the human might guess 0 goals, and it's a 50/50 split as to whether the human or the computer is correct when March comes around.

So what will happen in Option Chess is, whenever it is AZ's turn to play, it will guess what the future value of its remaining options are. Then it will asses the current value of playing an option right now, using the same techniques it would calculate the value of playing any other move. If the current value is better than its guess of future value, it will play the option now, otherwise it will hang onto the option. The human will do the same thing on his or her turn, and it cannot be predicted with any consistent accuracy which player guesses better.

Most often the engine will choose to play options whenever it has the chance. It will likely use all 16 options in 20 moves or less. A clever human player could defeat the engine if s/he could use less options initially, saving them for the endgame, and survive until the endgame and then come back by using his or her extra options.

So my claim is that the engines will all, including AZ, be about as good as the best humans at Option Chess and will not dominate the best humans. That's better than the current situation in regular chess, where there is no hidden information and the computers can assign accurate numbers to everything and dominate the humans.

Sid is making lots of bluster about how AZ is going to dominate humans in every task it undertakes, but he's got nothing backing that up other than games like chess and go. These are games of perfect information. So far, AZ has not shown it can do anything extraordinary with hidden information, such as in options trading. Every options trade involves guessing at some future value of something. If you guess right you win, if you guess wrong you lose. The best human options traders can make good money, and for every dollar they make, some other human is losing a dollar. Therefore, if AZ were to take over options trading and always guess all future values correctly, it would wipe out the entire industry, practically in a matter of days. Nothing like this has happened or will happen. If it does happen someday, then Sid can claim victory, but until then, he's just full of sound and fury, signifying nothing.

Sid is making lots of bluster about how AZ is going to dominate humans in every task it undertakes, but he's got nothing backing that up other than games like chess and go. These are games of perfect information. So far, AZ has not shown it can do anything extraordinary with hidden information, such as in options trading. Every options trade involves guessing at some future value of something. If you guess right you win, if you guess wrong you lose. The best human options traders can make good money, and for every dollar they make, some other human is losing a dollar. Therefore, if AZ were to take over options trading and always guess all future values correctly, it would wipe out the entire industry, practically in a matter of days. Nothing like this has happened or will happen. If it does happen someday, then Sid can claim victory, but until then, he's just full of sound and fury, signifying nothing.

Paul I ask you again....do you ever read anything before you type? Again, here is the link to the paper that shows how this very technology has been successfully deployed in heads up No limit Texas Hold em poker.
https://arxiv.org/pdf/1701.01724.pdf
In fact this technology is ideally suited for games with incomplete information. Just in case you are too lazy to click on the link here is the abstract. The paper is a worthwhile read however. at least you would understand the technology and logic behind it. You are alone in believing this "signifies nothing" The entire AI community is very excited about the success this technology has had with games with incomplete information but Paul Bonham know's better.....NOT!

Artificial intelligence has seen several breakthroughs in recent years, with
games often serving as milestones. A common feature of these games is that
players have perfect information. Poker is the quintessential game of imperfect
information, and a longstanding challenge problem in artificial intelligence.
We introduce DeepStack, an algorithm for imperfect information settings. It
combines recursive reasoning to handle information asymmetry, decomposi-
tion to focus computation on the relevant decision, and a form of intuition that
is automatically learned from self-play using deep learning. In a study involv-
ing 44,000 hands of poker, DeepStack defeated with statistical significance pro-
fessional poker players in heads-up no-limit Texas hold’em. The approach is
theoretically sound and is shown to produce more difficult to exploit strategies
than prior approaches.

[Paul Bonham]

Re: Chess Mastered with a Self-Play Algorithm

Paul I ask you again....do you ever read anything before you type? Again, here is the link to the paper that shows how this very technology has been successfully deployed in heads up No limit Texas Hold em poker.
https://arxiv.org/pdf/1701.01724.pdf
In fact this technology is ideally suited for games with incomplete information. Just in case you are too lazy to click on the link here is the abstract. The paper is a worthwhile read however. at least you would understand the technology and logic behind it. You are alone in believing this "signifies nothing" The entire AI community is very excited about the success this technology has had with games with incomplete information but Paul Bonham know's better.....NOT!

Sid, you are making assumptions about me without knowing what you are talking about. I am IN THE FIELD that we are discussing here. I am currently working with the top research chemists at one of the world's biggest pharma companies, helping them to write software that runs on a compute cluster with over 300 Haswell octacores and hundreds of thousands of CUDA cores and HPC communications backbone (Infiniband). This software does 3D molecular modeling for this company's Research Labs division, to discover new drugs. No, I'm not in DevOps maintaining the cluster, I'm helping write the core of the modelling software that runs jobs on the cluster. It's highly multithreaded C++ and it's millions of lines of code performing floating point and matrix calculations that would make your head spin. And guess what? I'm self taught! I'm the human version of AZ. I became expert in C++ with no degree in computer science, and only 1 formal programming course, Oracle database technology back in the mid-1990s. I've extended my domain knowledge into Hadoop, Hive, Spark, Scala on the data analytics side. Next up is Scikit-learn and TensorFlow.

I am very highly paid to do my work. Unfortunately, a lot of my money has to go into care for my wife who has a genetic and degenerative neoromuscular disease, and because I am highly paid, I am expected to foot all the bills. I knew what the risks were when I married her, she had been in a wheelchair since 6 years old, and I decided she was the most amazing person I'd ever met and would be worth every penny I would spend on her.

So if you need to know, THAT is why my company is taking so long to get my chess visions going. There's very little left over that I can funnel into the chess. But slowly, surely, it is coming to fruition. And while I'm at it, here's my response to your claims that I am wasting my money on my chess vision: it never was about money for me. It was always about giving chess players something more than what they have right now. Not the elite players that you pander to, i'm talking about the so-called woodpushers who love chess so much that they accept never winning any significant prizes. Well, my vision is to give those players the CHANCE to win significant prizes. The chess establishment doesn't want their royal game "polluted" with any element of chance, and is willing to sentence the woodpushers to eternal status as feeders to the elite in order to preserve the status quo. But I'm going to create a new vista of opportunity. And it will happen as surely as the poker boom happened at the turn of the century. The poker boom proved that people, huge numbers of people, will respond to a chance to win big money. So far chess doesn't have that, but that is going to change. And this new chess... is SO MUCH MORE EXCITING to both play and watch than regular chess. The woodpushers will come out of the woodwork, by the thousands, and so will the spectators by the hundreds of thousands, by the millions. Mark my words!

And the royal game of chess can go on with its status quo, as long as it can continue to find tireless volunteers and organizers who are the only reason competitive chess still even exists.


You are quoting a lot of fluff. Yes, it is true that AZ may be a paradigm shift in machine learning, I've said that already. It is incredible that it can learn chess to such an extent in 4 hours, but it is also telling that after the 4 hours, it basically flatlines. It's an asymptotic curve that rises rapidly, then flattens out. It could play itself for years and improve by only a few more tens of ELO points. It's already about the best it's going to be at chess. Think about it: if it improved so much after 4 hours, why would they immediately step into a match against Stockfish? Why wouldn't they let it learn another week or another month? The answer is that it had peaked already, the diminishing returns had already hit almost 0.

We've looked at the poker results, they are NOT convincing. But listen, if you want to prove your point, take up that challenge to write the options trading engine that will shut down Nadex! Or here's another one: horse racing. Write an AZ engine that will learn everything about horse racing and make perfect bets every time, putting the entire industry out of business. If you're so damn convinced, why are you doing anything else? Your billionaire status awaits! At the very least HIRE somebody to do it!

You can't do it, it can't be done. Until you do it, you are just blowing hot air.

[Ted Hsu]

Digging deeper into the AlphaZero Chess paper

Looking at the diagrams in the Silver et al. paper, I find it amusing that after learning about 2 hours, when it reaches a rating of about 3000, it suddenly "discovers" the Caro-Kann and the French Defense, and then seemingly discards them at still higher ratings.

[Sid Belzberg]

Re: Chess Mastered with a Self-Play Algorithm

We've looked at the poker results, they are NOT convincing.

Above is the main point that is germane to this discussion. Perhaps the law of diminishing returns that you cite with respect to training time is another relevant point. The rest of your long winded post appears to do nothing more then attempt to validate your technical credentials and present yourself as an altruistic person.

The only technical argument I have seen from you as to why the poker results are not convincing is that the sample size is not large enough and therefore not statistically significant. However the trend over the last several years prior to the publication of this paper is a steady improvement in AI driven poker playing as cited in the paper.

The paper is authored by credible researchers in this area and the results are indeed compelling. Although you may want to believe the results are a statistical anomaly the paper itself presents good mathematical arguments as to why the results are statistically significant.

I would agree that this technology is not a panacea to solve all problems with incomplete information such as playing poker at a table of nine players as opposed to heads up matches.

As far as options trading is concerned or for that matter bonds or equity trading at an institutional trading level the main problem institutional traders often face is trying to guess as to how large a buy or sell order is that they are up against when trying to predict the price movement.

My friend that I mentioned earlier is doing research for a very large bank in deploying the MCTS/Neural network approach to identify trading patterns that are predictive of this. It is early days but the results of his research are interesting.

I think you are approaching the problem with respect to online anti cheating computer chess from the wrong angle. I together with some co authors will be publishing a white paper in the next few days that among other things addresses the problem of controlling secure standardized software deployment and decentralized randomization techniques. It could have some applicability to online gaming. I will share the link when it is up.

[Vlad Drkulec]

Re: Digging deeper into the AlphaZero Chess paper

Looking at the diagrams in the Silver et al. paper, I find it amusing that after learning about 2 hours, when it reaches a rating of about 3000, it suddenly "discovers" the Caro-Kann and the French Defense, and then seemingly discards them at still higher ratings.

I guess that means that the Caro-Kann and the French Defense are only playable up to the 3000 level.

[Garland Best]

Re: Digging deeper into the AlphaZero Chess paper

One thing that I would find interesting:

I noted in the CFC Newsfeed how John Upper considers Alphazero to have a style, namely dominance via minor pieces. The self learning method the program uses employs random elements to learn. I wonder that if you started the process over again, would it converge onto the same solution, or would it develop a new style say more Kasparov than Karpov?