CFC Ratings Auditor

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #46
    It's been an interesting discussion. I can contribute a little on CFC rating stability. I examine the health of the system every three months and have been doing so for about ten years. I have access to a great diagnostic program built by Vincent Chow which in turn replaced a similar program built by Roger Patterson.

    Starting with the average rating of our strong players: Since the introduction of the bonus point formula in March 2012, the average rating of the top 100 players started at 2356 and stood at 2358 at the end of June 2020. During this period, the high was 2370 and the low was 2336. No evidence of strong players benefiting from the bonus formula.

    Moving to the intermediate players, I monitor all players rated above 1200 active in the preceding year. This group started in March 2012 at 1732 and stood at 1710 at the end of June 2020. During this period the high was 1750 and the low was 1690. There might be a tiny bit of deflation there, but not enough to warrant action yet.

    The situation when considering all players shows more volatility as expected because we now include the lower end - beginners and fading players. This group started in March 2012 at 1210 and stood at 1292 at the end of March 2020. I used March in this case as there seems to be a data anomaly for June 2020 (a jump to 1373). This group has gained rating points whichever way you look at it.

    The first conclusion is that the top 100 are stable. This is very important for players competing in outside Canada.
    The second conclusion is that the lower rated players have moved up a bit compared to the 1200+ players. The bonus formula may have had a role in this but there are other factors at play that make it difficult to form a conclusion. We used to turn a blind eye, for example, to junior events submitted for regular rating that did not meet the necessary conditions. The elimination of this practice may have had a role in keeping very low rated players out of the system until they have more experience.




    Paul Leblanc
    Treasurer Chess Foundation of Canada

    Comment


    • #47
      That is impressive dedicated work.

      Comment


      • #48
        Adult Player Province Rating Jan 1 2013 Rating Current Games Played Bonus Points Earned Average Bonus Per Game Comments
        Bob Armstrong ON 1618 1604 545 622 1.2 Active player added to sample
        Paul Leblanc BC 1829 1739 397 532 1.3
        John Brown ON 1586 1450 257 294 1.1 Half of bonus came from a 10-round tournament
        Roger Patterson BC 2084 2085 227 82 0.4 Primarily low seed in Premier section
        Fred McKim PEI 1918 1984 167 131 0.8
        Vlad Drkulec ON 2041 2020 140 189 1.3
        Tony Li ON 1902 1981 85 88 1.0
        Erik Malmsten ON 1892 1926 70 13 0.2 Infrequent player
        Aris Marghetis ON 1893 1869 41 0 0 Infrequent player
        Totals 1929 1951 1.0
        I didn't expect this, but looking at the tournament histories of the players who posted here (plus Bob Armstrong) suggests that the current bonus system acts similar to participation bonus.
        • The players are all stable adult players, possibly in slight decline (see current rating vs Jan 1 2013)
        • The average bonus point per game is 1.0, with the main exception being Roger who is primarily the low seed in the Premier section. We know from statistics that the players in the middle of a section have the most volatility. Since bonus points are substantial and occasional, infrequent players like Erik and Aris also have different averages.
        • The bonus per game only rises slightly based on number of rounds. Once the database is available, I would be interested to look more into this for juniors.

        Roger

        1. I think you are taking the word crossover too seriously. There is a substantial bias - the underdog is favored at every rating difference. This cannot statistically arise naturally since the spread will close itself over time.
        2a. I understand the circularity agree that an adjustment would be not reasonable. However, scaling down the bonus system would. Also, I believe you mentioned there is no better spread than 400 in the provisional rating calculation. I disagree and believe the actual spread of 600 is better.
        2b. The prior bonus system had a direct participation bonuse which would also increase the spread.
        3. See above chart. You appear to be quoting personal experience. You play primarily as a low seed in the Premier section of tournaments. Noise is driven by volatility, and the highest volatility comes from being in the middle of a section.
        Last edited by Tony Li; Monday, 17th August, 2020, 09:53 PM.

        Comment


        • #49
          Originally posted by Tony Li View Post
          Adult Player Province Rating Jan 1 2013 Rating Current Games Played Bonus Points Earned Average Bonus Per Game Comments
          Bob Armstrong ON 1618 1604 545 622 1.2 Active player added to sample
          Paul Leblanc BC 1829 1739 397 532 1.3
          John Brown ON 1586 1450 257 294 1.1 Half of bonus came from a 10-round tournament
          Roger Patterson BC 2084 2085 227 82 0.4 Primarily low seed in Premier section
          Fred McKim PEI 1918 1984 167 131 0.8
          Vlad Drkulec ON 2041 2020 140 189 1.3
          Tony Li ON 1902 1981 85 88 1.0
          Erik Malmsten ON 1892 1926 70 13 0.2 Infrequent player
          Aris Marghetis ON 1893 1869 41 0 0 Infrequent player (enemy of Caissa?)
          Totals 1929 1951 1.0
          I didn't expect this, but looking at the tournament histories of the players who posted here (plus Bob Armstrong) suggests that the current bonus system acts similar to participation bonus.
          • The players are all stable adult players, possibly in slight decline (see current rating vs Jan 1 2013)
          • The average bonus point per game is 1.0, with the main exception being Roger who is primarily the low seed in the Premier section. We know from statistics that the players in the middle of a section have the most volatility. Since bonus points are substantial and occasional, infrequent players like Erik and Aris also have different averages.
          • The bonus per game only rises slightly based on number of rounds. Once the database is available, I would be interested to look more into this for juniors.

          Roger

          1. I think you are taking the word crossover too seriously. There is a substantial bias - the underdog is favored at every rating difference. This cannot statistically arise naturally since the spread will close itself over time.
          2a. I understand the circularity agree that an adjustment would be not reasonable. However, scaling down the bonus system would. Also, I believe you mentioned there is no better spread than 400 in the provisional rating calculation. I disagree and believe the actual spread of 600 is better.
          2b. The prior bonus system had a direct participation bonuse which would also increase the spread.
          3. See above chart. You appear to be quoting personal experience. You play primarily as a low seed in the Premier section of tournaments. Noise is driven by volatility, and the highest volatility comes from being in the middle of a section.
          1) I was under the impression that you had claimed a crossover (based on a misreading of the link you provided) and thought it was important. My point was that as FIDE also shows the same behaviour (the underdog performing better than the theoretical curve) and so bonus points (which FIDE does not have) are not the reason for this behaviour.
          2) Provisional ratings are calculated for the first 25 games. Most players do not play so many games of that with people 400 points beyond their strength. I am dubious that this is in any way an important factor in the rating system
          2b) The spread of what? (you are apparently talking about spread of the theoretical and actual results in (1), call using the rating cuttoff for provisional ratings of 400 and your preference 600 as a spread in 2(a) and I see no connection between either of those and and bonus and/or participation points

          3) You seem to be argueing something like:
          - the Premier section is usually 2000+
          - I am close to the rating bottom for the Premier section of most events
          - therefore I am probably scoring few points
          - people scoring 50% will have greater volatility in their results compared to low scoring players as a matter of statistics
          - therefore I get fewer bonus points.
          - therefore ...... (not sure what point you are trying to make)

          but.....
          - some events (e.g BC Open) the boundary is at 1900, in any case there are always lots of class A players in the section
          - most events I am paired down (ie. am the higher rated) in the first round, although recently not always, so not near the bottom. I'm usually these days in the middle somewhere
          - my scores in most events are +0, +1 , or +2 probably you don't know but I frequently take 0 point byes in the last round) so again, not near the bottom , actually in the middle, and hard to square with your statement about volatility.

          so I don't think what ever argument you are trying to make works but maybe I misunderstand your argument.


          You seem to be quite convinced that bonus points are the cause of a number of ills but provide no data to support this. By data, I mean a comparison between a system without bonus points and a system with bonus points. You repeatedly make statements like " bonus points increase the spread" with no actual evidence this is true.

          Fortunately there is an answer for you if you are so convinced, Do what was done when the last bonus point system was set up. Run whatever calculation you want on the CFC data for the last 10 years and see what happens and if you are right. From my perspective, the current system is working well, there is no inflation to speak of, no one complains about under rated juniors any more, and generally, everything seems stable.

          It is an interesting chart though. I'm not sure I would call something that varies from 0.2 to 1.3 with a standard deviation of 0,5 as being equivalent to a constant participation point bonus.
          Last edited by Roger Patterson; Monday, 17th August, 2020, 02:38 AM.

          Comment


          • #50
            Originally posted by Tony Li View Post
            Adult Player Province Rating Jan 1 2013 Rating Current Games Played Bonus Points Earned Average Bonus Per Game Comments
            Bob Armstrong ON 1618 1604 545 622 1.2 Active player added to sample
            Paul Leblanc BC 1829 1739 397 532 1.3
            John Brown ON 1586 1450 257 294 1.1 Half of bonus came from a 10-round tournament
            Roger Patterson BC 2084 2085 227 82 0.4 Primarily low seed in Premier section
            Fred McKim PEI 1918 1984 167 131 0.8
            Vlad Drkulec ON 2041 2020 140 189 1.3
            Tony Li ON 1902 1981 85 88 1.0
            Erik Malmsten ON 1892 1926 70 13 0.2 Infrequent player
            Aris Marghetis ON 1893 1869 41 0 0 Infrequent player (enemy of Caissa?)
            Totals 1929 1951 1.0

            I didn't expect this, but looking at the tournament histories of the players who posted here (plus Bob Armstrong) suggests that the current bonus system acts similar to participation bonus.
            • The players are all stable adult players, possibly in slight decline (see current rating vs Jan 1 2013)
            • The average bonus point per game is 1.0, with the main exception being Roger who is primarily the low seed in the Premier section. We know from statistics that the players in the middle of a section have the most volatility. Since bonus points are substantial and occasional, infrequent players like Erik and Aris also have different averages.
            • The bonus per game only rises slightly based on number of rounds. Once the database is available, I would be interested to look more into this for juniors.

            Roger

            1. I think you are taking the word crossover too seriously. There is a substantial bias - the underdog is favored at every rating difference. This cannot statistically arise naturally since the spread will close itself over time.
            2a. I understand the circularity agree that an adjustment would be not reasonable. However, scaling down the bonus system would. Also, I believe you mentioned there is no better spread than 400 in the provisional rating calculation. I disagree and believe the actual spread of 600 is better.
            2b. The prior bonus system had a direct participation bonuse which would also increase the spread.
            3. See above chart. You appear to be quoting personal experience. You play primarily as a low seed in the Premier section of tournaments. Noise is driven by volatility, and the highest volatility comes from being in the middle of a section.
            Oh, my name in a list.

            Missing from your table are, I guess inactive, Hugh Brodie 2100 to 1851, Frank Dixon to 2090 to 1864, Ken Craft 1785 to 1780.

            Back in 2012 I found 100 Toronto players on both 1980 and today's database, but 28 of them are inactive Life members. 43 are now in a higher class, 49 have fallen back to the same class they were in 1980, and only 6 now are in a lower class than they were in 1980. However old-timers are past their peak years, only 13 are still in the same class as their rating high. Of the 60 still active players Scarborough has 7 while Hamilton has 4 plus 3 in Burlington.

            Player Current CFC Rating /Highest Rating since 1995
            1 Lawrence Day 2275 /2510
            2 Brian Hartman 2438 /2487
            3 Kevin Pacey (Ottawa) 2206 /2406
            4 Michael Dougherty 2275 /2404
            5 Doug Bailey 2268 /2400
            6 Brett Campbell 2216 /2388
            7 David Filipovich 2229 /2370
            8 Vinny Puri 2320 /2350
            9 Stephen Boyd (France) 2323 /2323
            10 Andrei Moffat 2181 /2321
            11 Ruperto Frilles 2034 /2302
            12 Bill Evans 1989 /2300
            13 Brian Fiedler 2045 /2300
            14 Lorne Yee 2259 /2285
            15 Nava Starr 2138 /2277
            16 Fima Rakhinshteyn 2117 /2262
            17 John Wright 2232 /2232
            18 Greg Stavropoulos 1739 /2220
            19 Ralph Gregorz 1997 /2218
            20 Imtiaz Husain (USA) 2134 /2211

            Since then Robert Morrison returned and his rating dropped from 2344 to 2276. And I thought Peter Murray 2289 had played, maybe only blitz.

            Comment


            • #51
              I tread very cautiously into this technical rating discussion.

              But for me personally, I think the number of bonus points I garnered is not due to participation........it is due to my historically unstable ability to play!!

              Anyone who looks at my rating stats above will see that I have appeared to hold somewhat for 7 years (from 68 y.o. to 75 y.o.).

              In fact I have, from time to time, gone down, and then regained, by going on a tear all of a sudden in some particular tournament.

              Here are my peak/nadir ratings by year:
              Year Peak Nadir

              2013 - 1637 1516
              2014 - 1674 1516
              2015 - 1641 1511
              2016 - 1727 1503
              2017 - 1711 1503
              2018 - 1669 1498
              2019 - 1645 1447
              2020 - 1631 1536
              (March 12) (Feb. 5)

              So I was always being pleasantly surprised whenever I'd do one of my very temporary comeback charges! Because I exceeded expectations by winning when I shouldn't have been (If my rating was true, I shouldn't have been doing what I was doing), I got BONUS POINTS! Then, almost immediately, I'd start distributing them back into the pool, to my opponents, for a year, until I'd resurrect from being brain dead again.

              Love to have one of the stats wizards evaluate my analysis.......

              Bob A
              Last edited by Bob Armstrong; Monday, 17th August, 2020, 06:25 PM.

              Comment


              • #52
                Originally posted by Tony Li View Post
                Adult Player Province Rating Jan 1 2013 Rating Current Games Played Bonus Points Earned Average Bonus Per Game Comments
                Aris Marghetis ON 1893 1869 41 0 0 Infrequent player (enemy of Caissa?)
                I don't see any upside to the petty insult-joke on my name. Just sayin' lol

                Comment


                • #53
                  Originally posted by Aris Marghetis View Post

                  I don't see any upside to the petty insult-joke on my name. Just sayin' lol
                  I apologize and have removed the comment. It was an unfortunate coincidence as I did not think of your name.

                  Comment


                  • #54
                    I am withdrawing from this year's election to the Ratings Auditor position.

                    I have not succeeded in generating meaningful discussion regarding the shock to the rating system due to the COVID shutdown. I will be investing my time instead on improving my communication skills.

                    Comment


                    • #55
                      Originally posted by Bob Armstrong View Post
                      I tread very cautiously into this technical rating discussion.

                      But for me personally, I think the number of bonus points I garnered is not due to participation........it is due to my historically unstable ability to play!!

                      So I was always being pleasantly surprised whenever I'd do one of my very temporary comeback charges! Because I exceeded expectations by winning when I shouldn't have been (If my rating was true, I shouldn't have been doing what I was doing), I got BONUS POINTS! Then, almost immediately, I'd start distributing them back into the pool, to my opponents, for a year, until I'd resurrect from being brain dead again.
                      Hi Bob,

                      If you gain 1 extra point per game, it works out to about 25 rating points (no matter how many games you've played!). So in the end, it has very little impact on your rating if your opponents are moving through the section quickly. If they are moving up sections, then they are exporting the bonus points to the higher sections.

                      I will find some later to analyze your results to see if you are actually a volatile player.

                      Comment


                      • #56
                        I find that table very interesting. Like Bob, I generously gave away all those bonus points!
                        It would be even more interesting to include some fast rising juniors and see how many bonus points per game they have earned. Try Nicholas Vettese or Mark Plotkin or Jason Cao or Qiyu Zhou
                        Paul Leblanc
                        Treasurer Chess Foundation of Canada

                        Comment


                        • #57
                          Here are the bonus points for active players from the latest IM norm tournament, Veterans and Seniors championships. Guess the player!

                          Juniors: Eugene Hua, Jason Cao, Mark Plotkin, Nicolas Vettese, Qiyu Zhou
                          Veterans: Bob Gillanders, Brian Clarke, David Cummings, Ian Findlay, Michael Barron, Michael Dougherty, Tomas Krnan, Victor Plotkin
                          Seniors: Isai Berengolts, Richard Glew

                          For comparison purposes, bonus points over 2200 are multiplied by 2.

                          Player # Games Total Bonus Points Bonus per Game
                          A 586 812 1.4
                          B 378 334 0.9
                          C 517 433 0.8
                          D 721 596 0.8
                          E 221 132 0.6
                          F 171 93 0.5
                          G 335 176 0.5
                          H 359 187 0.5
                          I 374 152 0.4
                          J 362 130 0.4
                          K 238 85 0.4
                          L 278 74 0.3
                          M 113 11 0.1
                          N 147 0 0
                          O 144 0 0

                          Comment


                          • #58
                            After 10 years as CFC Rating Officer I will be stepping down this year. I will do my best to support my successor during the transition.
                            Paul Leblanc
                            Treasurer Chess Foundation of Canada

                            Comment


                            • #59
                              Thanks for all the good work Paul. CFC volunteers usually do not get enough appreciation.

                              Comment


                              • #60
                                Hi Paul:

                                You've been a stellar volunteer.......thanks from this member!

                                Bob

                                Comment

                                Working...
                                X