Detecting Cheating in Chess by Computer Analysis

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Re: Detecting Cheating in Chess by Computer Analysis

    Originally posted by Ed Zator View Post
    Nevertheless, there was probably some cheating at the 2011 Canadian Open!

    My opponent who was posting his moves to the internet via one of those Monroi devices,
    began running out of the hall after every move once he got a lost position.
    He won that game.

    I also noticed this happening with one other player using a Monroi recorder.

    Neither the organizers or the TDs prevented such players from entering and
    exiting the hall repeatedly. Either player could have gotten advice from
    someone watching over the internet, such as happened at the Dubai Open
    not long ago.

    In that case, the player was caught because of steps taken by the organizers there.
    No such luck here.
    You are the same person who accused someone of colluding/cheating in the previous 2010 Canadian open because they couldn't win a Q vs R + P endgame even though they tried for 95 moves. I know for a fact that in that case you were wrong. I don't know you, but that makes me wonder if you are just overly paranoid, or one of those players who always has to come up with some excuse when you lose.

    That being said, if your are not exaggerating, and your opponent actually did leave the playing area after every move I can see how someone might become suspicious. Did you bring it up with the TD during or after the game? It is hard for a TD watch every player in a large open but keeping an eye on one might be possible. Also having a delay in the broadcasting of games on the internet seems reasonable. It has been done in some European tournaments already so it must be possible. That would remove the possibility of receiving help from an internet accomplice but it still would not stop someone with a chess program on a smart phone hidden in their pocket.

    Comment


    • #17
      Re: Detecting Cheating in Chess by Computer Analysis

      Originally posted by Terry Chaisson View Post
      You are the same person who accused someone of colluding/cheating in the previous 2010 Canadian open because they couldn't win a Q vs R + P endgame even though they tried for 95 moves. I know for a fact that in that case you were wrong. I don't know you, but that makes me wonder if you are just overly paranoid, or one of those players who always has to come up with some excuse when you lose.

      That being said, if your are not exaggerating, and your opponent actually did leave the playing area after every move I can see how someone might become suspicious. Did you bring it up with the TD during or after the game? It is hard for a TD watch every player in a large open but keeping an eye on one might be possible. Also having a delay in the broadcasting of games on the internet seems reasonable. It has been done in some European tournaments already so it must be possible. That would remove the possibility of receiving help from an internet accomplice but it still would not stop someone with a chess program on a smart phone hidden in their pocket.
      LOL Terry, I was just going to mention that but you beat me to the punch. Ed, you should quit chess, you obviously can't take a loss. I hope you don't take up poker, I can only imagine the accusations that you'll shoot out if you get sucked out on the river.
      Shameless self-promotion on display here
      http://www.youtube.com/user/Barkyducky?feature=mhee

      Comment


      • #18
        Re: Detecting Cheating in Chess by Computer Analysis

        Evidence, please.

        The organizer has a duty of care, if they're going to allow broadcast
        devices, to monitor what those using them are doing. Any suspicious
        behaviour should be investigated without a players asking, or
        giving evidence that something is going on.

        I also was posting my moves to the internet via one of those Monroi devices and I did manage to win a game against an IM where I was totally lost but if he had taken my rook instead of my queen he would have won. He also happened to walk into a similar tactic that had been covered in a chess lesson I had two weeks prior to the game. Accusing someone of cheating without proof is wrong. It would be simple enough to stop cheating of the kind that you are alleging. Ban cell phones at tournaments. I don't own one so my win against the IM must have been accomplished with some kind of telepathy.

        Banning cellphones doesn't stop players from carrying them, or using
        land lines.

        Did the organizers or TDs prevent you from entering and leaving the hall repeatedly when you were playing with a person with a Monroi? The Monroi puts the game on the internet so by your logic then the organizers should have prevented you AND your opponent from leaving the room. Couldn't your opponent allege that you gained your advantage by cheating in the first place?

        They can certainly try. But when was the last time a teller robbed a bank
        with the gun a bank robber brought to a holdup?

        If the organizers had prevented me from leaving the room they might have been faced with a situation similar to the one that happened in an Ottawa tournament a few years ago. When you gotta go, you gotta go. In addition, I probably could have made some extra money with a human rights complaint.

        Again, once the game was 'lost' my opponent just repeatedly left the hall
        after every move. The TD should have been watching and followed them at
        least once.

        And so could their opponent, which means that your initial advantage against a superior opponent could put you under a cloud of suspicion of cheating

        No it wouldn't. It's not my device, and I'm not the one coming and going.

        Comment


        • #19
          Re: Detecting Cheating in Chess by Computer Analysis

          The back-and-forth allegations above---without any concrete evidentiary foundation---are exactly what it is half my mission to combat.

          I have updated my site with several pages that begin to answer concerns and queries raised here and in other forums: FAQ, General Remarks, and "The Parable of the Golfers". Some of you may remember the last from my public lecture at the Canadian Open.

          Replying to Egidijus Zeromskis on Rybka and Houdini, first note that the analysis is not done in the usual playing mode but to a fixed depth. In a recent post on the Rybka forums I reported tests showing that Rybka 3 beats Rybka 4.1 in this fixed-depth mode, and that Multi-PV beats the Single_PV playing mode at any depth. As for Houdini, my preliminary tests show it needs depth 16 or 17 to beat Rybka's reported depth 13 (which many allege is "really" 16 or 17 anyway), and then also note the font colour given to Houdini's name in those rating lists.. .
          Last edited by Kenneth Regan; Thursday, 22nd March, 2012, 10:22 AM. Reason: "accusations"-->"allegations", though to be sure the mission is combatting accusations.

          Comment


          • #20
            Re: Detecting Cheating in Chess by Computer Analysis

            [QUOTE=Terry Chaisson;48748]You are the same person who accused someone of colluding/cheating in the previous 2010 Canadian open because they couldn't win a Q vs R + P endgame even though they tried for 95 moves. I know for a fact that in that case you were wrong. I don't know you, but that makes me wonder if you are just overly paranoid, or one of those players who always has to come up with some excuse when you lose.
            [QUOTE]

            If anything looks suspicious, the TD should investigate, not the players.
            The TDs should know who has these devices by watching the broadcast.

            Tournaments aren't, nor should they be, run on the honour system. If that
            were the case, then we could all stay home and play via the Internet.

            Accusing people of paranoia is just paranoid itself, and usually the last
            resort of someone who has no argument.

            Comment


            • #21
              Re: Detecting Cheating in Chess by Computer Analysis

              Originally posted by Gary Ruben View Post
              I don't have much to keep me busy since I retired from playing and I wouldn't mind running the game on a fast multi core computer with various software.
              I am positive you will think of something better to
              do.

              Comment


              • #22
                Re: Detecting Cheating in Chess by Computer Analysis

                Originally posted by Kenneth Regan View Post
                The back-and-forth accusations above---without any concrete evidentiary foundation---are exactly what it is half my mission to combat.

                I have updated my site with several pages that begin to answer concerns and queries raised here and in other forums: FAQ, General Remarks, and "The Parable of the Golfers". Some of you may remember the last from my public lecture at the Canadian Open.

                Replying to Egidijus Zeromskis on Rybka and Houdini, first note that the analysis is not done in the usual playing mode but to a fixed depth. In a recent post on the Rybka forums I reported tests showing that Rybka 3 beats Rybka 4.1 in this fixed-depth mode, and that Multi-PV beats the Single_PV playing mode at any depth. As for Houdini, my preliminary tests show it needs depth 16 or 17 to beat Rybka's reported depth 13 (which many allege is "really" 16 or 17 anyway), and then also note the font colour given to Houdini's name in those rating lists.. .
                Your program sounds interesting, but I don't see its use in large tournaments,
                or ones with strong enough players. Can it be used to estimate ratings?

                Comment


                • #23
                  Re: Detecting Cheating in Chess by Computer Analysis

                  "Accusing people of paranoia is just paranoid itself, and usually the last
                  resort of someone who has no argument."

                  HOLY CRAP THIS IS IRONIC#$@$# AHHH#$@$#

                  Comment


                  • #24
                    Re: Detecting Cheating in Chess by Computer Analysis

                    Originally posted by Ed Zator View Post
                    Your program sounds interesting, but I don't see its use in large tournaments,
                    or ones with strong enough players. Can it be used to estimate ratings?
                    The papers I mention at the top of this thread do exactly that. Here are direct links to my paper with the Canadian Open study and my ongoing Intrinsic Performance Ratings compendium---note the latter intends to fill in other performances by the high-IPR outliers from the Canadian Open. The other tests show nothing amiss anywhere.
                    Last edited by Kenneth Regan; Thursday, 22nd March, 2012, 07:47 AM. Reason: Gave links

                    Comment


                    • #25
                      Re: Detecting Cheating in Chess by Computer Analysis

                      Originally posted by Kenneth Regan View Post
                      first note that the analysis is not done in the usual playing mode but to a fixed depth.
                      Do cheaters used the same mode too? :D Did you run with normal (factory) engine settings and compare results? (I imagine it is more difficult to simulate)

                      Curious - did you do "blind" tests for your method with games were players occasionally used engines?

                      As for a fixed depth - 13 - imho, the number is too low to compare strength. Do you have a plot: wins percentage vs depth (1-2x) for various engines?

                      Sorry, if there are too many questions for you and maybe answers are on your webpage. I'll try to read thoroughly at least your mentioned paper. :)

                      Comment


                      • #26
                        Re: Detecting Cheating in Chess by Computer Analysis

                        Originally posted by Lucas Davies View Post
                        "Accusing people of paranoia is just paranoid itself, and usually the last
                        resort of someone who has no argument."

                        HOLY CRAP THIS IS IRONIC#$@$# AHHH#$@$#
                        Glad I'm not the only one that noticed how amazing this comment is haha.

                        Comment


                        • #27
                          Re: Detecting Cheating in Chess by Computer Analysis

                          Hi Ken,

                          I just wanted to say that your papers are very interesting... thanks for posting the link.

                          Comment


                          • #28
                            Re: Detecting Cheating in Chess by Computer Analysis

                            Originally posted by Egidijus Zeromskis View Post
                            Do cheaters used the same mode too? :D Did you run with normal (factory) engine settings and compare results? (I imagine it is more difficult to simulate)
                            I've made a consistent and extensive statistical study of one phenomenon: Rybka 3 to depth 13. A systematic deviation from normal playing practice---with any engine---may cause a detectable deviation in this phenomenon. Online playing sites have access to better forensic tools, such as the exact time taken to play a move and ability to simulate many engines and settings---I've been disappointed not even to be able easily to obtain the time taken for each move at major events.

                            This aligns with the general difference between phenomenology and epistemology in science---for my experiments I can only be concerned with the former. Public testimony in the Sebastien Feller case has related tests with the engine Fire or Firebird, not sure which. Those and Houdini are evidently close enough to Rybka that a fair bit of "fidelity" is preserved---actually my public-feedback communication to Frederic Friedel previous to the one he posted was about using my work to cluster engines according to "personality".


                            Curious - did you do "blind" tests for your method with games where players occasionally used engines?
                            I would like to get more of them.

                            As for a fixed depth - 13 - imho, the number is too low to compare strength. Do you have a plot: wins percentage vs depth (1-2x) for various engines?
                            Larry Kaufman estimated Rybka 3 depth 14 (Single-PV) to be about 2750. Privately and I think publicly, Rybka experts have stated that for a fairly long range, one ply translates to about 70 rating points, with "no diminishing returns visible yet", which would place depth 13 in the 2650--2700 range. However, my tests reported above show the Multi-PV mode to be tangibly stronger, so that may put me back in the 2700s. Slides from my talks explain two competing desiderata: (1) to have the engine analysis be authoritatively strong, but also (2) to have the analysis be in the fattest middle slice of the strength spectrum of players being modeled. Depth 13 seems to be an adequate compromise---the proof of the pudding is how well the model's results and theoretically projected error bars "jive" with real results.


                            In my opinion---my testing on this is still in an unsettled stage---the true strength of Rybka 3 depth 13 is 2900's in the late opening sliding down to 2400's in the endgame. This aligns with informal observations of the mean time taken to reach depth 13 at various stages of a game. Having fewer pieces makes it much quicker, and the rubric of Elo 50 points per doubling of time corresponds to the 2400's--2900's range. My paper with GM Macieja and Guy Haworth includes a graphic showing average-error ramping up steadily to Move 40, then suddenly falling halfway-but-only-halfway. Taking selected game-turn ranges from my training sets shows that the IPR falls off steeply as the game goes on, and here I agree that use of higher depths at later stages (in particular for endgames) will be needed to judge whether this decline in strength is actual or illusory.

                            Sorry, if there are too many questions for you and maybe answers are on your webpage. I'll try to read thoroughly at least your mentioned paper. :)
                            I am trying to put information like this on my site---new links mentioned above---I have been insanely busy with other professional obligations too... Thank you for good questions to prompt me!

                            Comment


                            • #29
                              Re: Detecting Cheating in Chess by Computer Analysis

                              I think I found the game. I went to the MonRoi site and looked up the games from the 2011 Canadian Open. I only found 1 of Ed's game that was broadcast - a round 4 loss. Ed only lost 2 games in the tournament so it's either this game or his round 7 loss. Ed can verify.

                              I ran it quickly through Fritz 12 on my laptop and to me it looks like black lost because he missed a mate threat on the back rank and had to give up a piece. There are several mistakes on both sides (I included ??'s in the pgn). I am not an expert, but I don't see any evidence of outside assistance.

                              That being said, if games are being broadcast live on the internet there is always a risk that the kind of cheating Ed suspected might happen. What can be done to prevent it is a valid question, but wild unsubstantiated accusations don't help.

                              [Event "Pinnacle 48th COCC 2011"]
                              [Site "Toronto"]
                              [Date "2011.07.12"]
                              [Round "4"]
                              [White "Moran-venegas, Mario"]
                              [Black "Zator, Ed"]
                              [Result "1-0"]
                              [WhiteELO "1864"]
                              [WhiteTitle ""]
                              [BlackELO "1902"]
                              [BlackTitle ""]
                              [Source "MonRoi"]

                              1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 a6 6.Be2 e6 7.Be3 Be7 8.Qd2 b5 9.a3 Bb7 10.f3 Nc6 11.O-O Nxd4 12.Bxd4 O-O 13.Rad1 e5 14.Bf2 Qc7 15.Bd3 Rad8 16.Ne2 d5 17.exd5 Bxd5 18.Ng3 Be6 19.Qe2 Nd5 20.Nf5 Bf6 21.g3 g6 22.Ne3 Bg7 23.Nxd5 Bxd5 24.c3 f5 25.Bc2? Kh8? 26.Rfe1? Qb7 27.Kg2 g5 28.Bc5 g4 29.Rxd5 gxf3 30.Qxf3 Rxd5 31.Kh3?? Rf6?? 32.Be7 e4?? 33.Bxf6 Qd7?? 34.Bxg7 Kxg7 35.Qf4 Rd2 36.Qg5 Kf8? 37.Qf6 Ke8 38.Qh8 Ke7 39.Qxh7 Kd8 40.Qxd7 1-0

                              Comment


                              • #30
                                Re: Detecting Cheating in Chess by Computer Analysis

                                My brain is turning to mush after looking at that game. Second last blunder wins.
                                Last edited by Vlad Drkulec; Friday, 23rd March, 2012, 12:36 AM.

                                Comment

                                Working...
                                X