CFC rating system broken?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Re: CFC rating system broken?

    Earlier in this thread, Tom reminded chesstalk viewers of his proposal for the CFC to go to a rating system mimicking the way the ACBL does for Bridge players, i.e. collect rating (e.g. "master") points as you play over the years, and therefore Bridge 'ratings' never go down.

    I'm not sure if the British still have their own rating system that's radically different from FIDE's (e.g. a rating of 150, as in the '150 Attack' variation of the Pirc, is typical of what would be an average player's, I think[?!]). Thus their rating system is kind of a precident for a radically different rating system used by a Federation, so why not have the CFC adopt a rating system like the ACBL's?

    [edit: The CFC could try to have the best of both worlds by having the convential rating system it does now, plus an ACBL style system, and/or a system with innovative features of the sort Alan suggests.]
    Last edited by Kevin Pacey; Wednesday, 23rd February, 2011, 05:23 PM.
    Anything that can go wrong will go wrong.
    Murphy's law, by Edward A. Murphy Jr., USAF, Aerospace Engineer

    Comment


    • #32
      Re: CFC rating system broken?

      Originally posted by Kevin Pacey View Post
      Earlier in this thread, Tom reminded chesstalk viewers of his proposal for the CFC to go to a rating system mimicking the way the ACBL does for Bridge players, i.e. collect rating (e.g. "master") points as you play over the years, and therefore Bridge 'ratings' never go down.

      I'm not sure if the British still have their own rating system that's radically different from FIDE's (e.g. a rating of 150, as in the '150 Attack' variation of the Pirc, is typical of what would be an average player's, I think[?!]). Thus their rating system is kind of a precident for a radically different rating system used by a Federation, so why not have the CFC adopt a rating system like the ACBL's?

      [edit: The CFC could try to have the best of both worlds by having the convential rating system it does now, plus an ACBL style system, and/or a system with innovative features of the sort Alan suggests.]
      To be honest people need to stop putting so much emphasis on their ratings it's just a number. Alot of the talk is based on sore egos and not wanting to admit that they're not as strong as they are in their minds. I have to admit I'm at times a little sore about where I am in chess, but thats part of the game.

      The rating problems that I've seen that I think are legitimate, come when ratings are incorporated into tournaments. Turning away a player from any tournament or section because his cfc rating is too low...well now we have problems since the rating system must be accurate and equal across Canada to ensure fairness. Maybe I'm wrong but this is the only time I can really see ratings impacting anything worthwhile.

      Comment


      • #33
        Re: CFC rating system broken?

        Originally posted by Robert Clark View Post
        To be honest people need to stop putting so much emphasis on their ratings it's just a number. Alot of the talk is based on sore egos and not wanting to admit that they're not as strong as they are in their minds. I have to admit I'm at times a little sore about where I am in chess, but thats part of the game.

        The rating problems that I've seen that I think are legitimate, come when ratings are incorporated into tournaments. Turning away a player from any tournament or section because his cfc rating is too low...well now we have problems since the rating system must be accurate and equal across Canada to ensure fairness. Maybe I'm wrong but this is the only time I can really see ratings impacting anything worthwhile.
        Hi Robert

        Ratings are at least useful for establishing whether someone is eligble for a class prize (class prizes are an old sore point with Tom :)). Plus, if it wasn't for having a rating service, the CFC would need to charge a lot less for membership to avoid losing lots of members, since a CFC rating system is one of the few services they offer to average members at the moment. Not only that, but if there was no CFC rating system, organizers would have little reason to insist that participants in their events be CFC members, since submitting a tournament report for rating by the CFC would obviously no long occur (unless an event had to be FIDE rated, and then that could be deemed to require CFC membership for all participants for the CFC to process the report and send it to FIDE).
        Anything that can go wrong will go wrong.
        Murphy's law, by Edward A. Murphy Jr., USAF, Aerospace Engineer

        Comment


        • #34
          Re: CFC rating system broken?

          If you want to use ratings to encourage activity, then something along the lines of what Tennis uses could be good - you only get points for games played roughly in the past year.
          Christopher Mallon
          FIDE Arbiter

          Comment


          • #35
            Re: CFC rating system broken?

            Originally posted by Kevin Pacey View Post
            Hi Robert

            Ratings are at least useful for establishing whether someone is eligble for a class prize (class prizes are an old sore point with Tom :)). Plus, if it wasn't for having a rating service, the CFC would need to charge a lot less for membership to avoid losing lots of members, since a CFC rating system is one of the few services they offer to average members at the moment. Not only that, but if there was no CFC rating system, organizers would have little reason to insist that participants in their events be CFC members, since submitting a tournament report for rating by the CFC would obviously no long occur (unless an event had to be FIDE rated, and then that could be deemed to require CFC membership for all participants for the CFC to process the report and send it to FIDE).
            Hi Kevin,

            I agree that class prizes are a little ridiculous to be honest. That being said, I'm not saying that we should get away with the rating system, just that people need to take ratings with a grain of salt. Although i certainly agree that more should be one to justify the large membership fee paid by the average player.

            Comment


            • #36
              Re: CFC rating system broken?

              Originally posted by Tom O'Donnell View Post
              I tried running a tournament rated FIDE without also being rated CFC and this was not allowed.
              I gather then, the published info for the 2011 Canadian Open on their
              website is a grammatical error. They don't mean that the Open section
              will only be FIDE rated. They mean it will be the only section that will
              be FIDE rated (in addition to CFC rated).

              Comment


              • #37
                Re: CFC rating system broken?

                Originally posted by Alan Baljeu View Post
                The problem with the Elo system is too often I hear people say they don't want to play because they might lose points. That's terrible when you're trying to promote chess!

                Instead of a zero-sum rating system, we should instead have a system that rewards participation. Win games, get points. Place in tournaments, get points. Have a player of the year race, and prizes at the end. Big tournaments offer more points. Collect stats on player performance. Keep lifetime points and points per event.

                That would incentivize participation instead of Elo which incenses only rising newcomers.
                They kind of had that with participation points which were recently done away with and at least they have a results bonus which appears to be on death row as well now.

                Comment


                • #38
                  Re: CFC rating system broken?

                  The trouble with participation points is its inflationary. It's only fine if you are willing to accept declining standards of what skill equates to 2000, etc. I'd promote an alternate system that's emphasizes recent results, not estimated winning potential.

                  Comment


                  • #39
                    Re: CFC rating system broken?

                    The CFC's rating system was deflating, as was apparent in most people's opinion, before the CFC overcorrected in 2007 with an over-generous boon, plus possibly over-generous participation points. Anecdotal evidence of this deflation, for example, is that in Ottawa in the immediate years before the boon, about 10 Ottawa masters I knew of had dropped to Expert level in terms of their CFC ratings.

                    Halving the rate at which participation points were awarded, for example, rather than eliminating them altogether, might have been a better way to avoid what I think will be a coming period of renewed deflation of the CFC's rating system.

                    I think the Result Bonus Points need not be eliminated necessarily either. Bonus points under the Result Bonus Point scheme are usually a bit less generously awarded than under the old Bonus Point formula used say in the 1990's. That was when the CFC rating system seemed to me (and to a past CFC president I know) to be pretty satisfactory in terms of stability, and in terms of players' CFC ratings being normally approximately similar to their FIDE ratings. There were also participation points back then, but note that there was a bigger pool of CFC members in the system.

                    At any rate, the CFC will do what it will do, and after the next round of deflation, one thing the CFC could do is re-introduce participation points (say at half the previous rate), possibly award (normally quickly improving) juniors double the participation points adults get, award Bonus points of some kind, plus possibly add a rating floor of some kind, like the USCF has. Do all this before deflation gets so bad that a 'boon' is required. In other words, monitor and study the rating system on an ongoing basis, and be faster to act next time.
                    Last edited by Kevin Pacey; Friday, 25th February, 2011, 09:31 PM. Reason: Grammar
                    Anything that can go wrong will go wrong.
                    Murphy's law, by Edward A. Murphy Jr., USAF, Aerospace Engineer

                    Comment


                    • #40
                      Re: CFC rating system broken?

                      I should note that the CFC never did fix the deflation to Active ratings. I think I won't bother to resume my boycott of Active events, now that Active ratings presumably get no participation points added to them nowadays (like regular ratings). That's because the Active events in my club now involve prizes (books/cash), so for me there is sufficient bait, in such minor club events, to outweigh principle :).
                      Anything that can go wrong will go wrong.
                      Murphy's law, by Edward A. Murphy Jr., USAF, Aerospace Engineer

                      Comment


                      • #41
                        Re: CFC rating system broken? [over/under2200]

                        I didn't see this one mentioned here, so:

                        As I understand it, when a player crosses over 2200 the volatility of the rating is cut in half (I think this is sometimes explained as the K value is cut in half). e.g. a performance that would earn a 2199 player 20 points would earn a 2200 player only 10, and a perfomance that would cost a 2199 player 30 points would cost a 2200 player only 15.

                        Bill Doubleday and I have both noticed that CFC calulations do not seem to follow this when a player goes from above 2200 to below 2200.

                        Here are the two examples we noticed (our own ratings, not surprisingly):
                        my rating after the Ottawa Winter Open http://www.chess.ca/xtableSQL.asp?TNum=201101033

                        Bill's rating after 2010 Canadian Open http://www.chess.ca/xtableSQL.asp?TNum=201007034

                        [I haven't done the calculations here, but it might be an illustration to compare Bill's rating change at this event with Micheal Barron's: Barron's performance was about 400 points lower than his rating, which dropped 35; Doubleday's performance was 260 poinst lower than his rating, but he lost 72 rating points.]


                        Apart from being inconsistent with how the ratings are claimed to be calculated (assuming I've understood it right), it is unfair to players around 2200-2250 who have their rating gains cut in half but (with one really bad tournament) can have their rating drop at the full U2200 rate.

                        It might seem that overall this is a wash, since every player whose rating increases from under 2200 to over 2200 gets to keep all of the points earned, rather than all the point up to 2200 and only half of those once over 2200. There's probably a clearer way to put that, but here's an illustration: a 2190 player whose tournment perfomance earns 40 points has his rating go to 2230, not 2215 as it would be if all the points after the first 10 (which brought the player up to 2200) were cut in half.

                        But for this to really be a wash would depend on how far above 2200 the player's rating had jumped.
                        Here's an example:
                        Player A is 2190 and earns 12 points to end up with a rating of 2202.
                        Player B is 2190 and earns 80 points to end up with a rating of 2270.

                        [Player A is only 1 point ahead of where he would have been if the lower K value was applied to all rating gains over 2200, while Player B is 35 point ahead of where he would have been.]

                        Both players compete in several events, scoring the same performance rating in each, raising their ratings each time, so that Player A is rated 2220 and B is rated 2285. Then they each lose a one game match vs a 2000 player.
                        The way the CFC ratings are supposed to work:
                        A's rating should drop 12.4 points to 2208,
                        B's rating should drop 13.7 to 2272.

                        The way the CFC ratings actually do work:
                        B's rating does go to 2272, :o
                        A's rating drops to 24.8 points to 2195.

                        Obviously this is not "a wash" for Player A, who benefitted 1 whole point in his step over 2200, but dropped 24.8 points on his way down.

                        Suggestion: it might be interesting to compare the number of active players between say 2199-2170 with the number between 2200-2229 to see if it's the Bell-shaped drop off you'd expect, or if there are a lot more active players just below 2200. If so, that would show a systematic effect of this rating error.

                        I don't have any suggestion about fixing this apart from rewriting the rating program and copying whatever wins this competition: http://www.kaggle.com/chess
                        Last edited by John Upper; Saturday, 26th February, 2011, 02:03 PM.

                        Comment


                        • #42
                          Re: CFC rating system broken? [over/under2200]

                          Originally posted by John Upper View Post
                          I didn't see this one mentioned here, so:

                          As I understand it, when a player crosses over 2200 the volatility of the rating is cut in half (I think this is sometimes explained as the K value is cut in half). e.g. a performance that would earn a 2199 player 20 points would earn a 2200 player only 10, and a perfomance that would cost a 2199 player 30 points would cost a 2200 player only 15.

                          Bill Doubleday and I have both noticed that CFC calulations do not seem to follow this when a player goes from above 2200 to below 2200.

                          Here are the two examples we noticed (our own ratings, not surprisingly):
                          my rating after the Ottawa Winter Open http://www.chess.ca/xtableSQL.asp?TNum=201101033

                          Bill's rating after 2010 Canadian Open http://www.chess.ca/xtableSQL.asp?TNum=201007034

                          [I haven't done the calculations here, but it might be an illustration to compare Bill's rating change at this event with Micheal Barron's: Barron's performance was about 400 points lower than his rating, which dropped 35; Doubleday's performance was 260 poinst lower than his rating, but he lost 72 rating points.]


                          Apart from being inconsistent with how the ratings are claimed to be calculated (assuming I've understood it right), it is unfair to players around 2200-2250 who have their rating gains cut in half but (with one really bad tournament) can have their rating drop at the full U2200 rate.

                          It might seem that overall this is a wash, since every player whose rating increases from under 2200 to over 2200 gets to keep all of the points earned, rather than all the point up to 2200 and only half of those once over 2200. There's probably a clearer way to put that, but here's an illustration: a 2190 player whose tournment perfomance earns 40 points has his rating go to 2230, not 2215 as it would be if all the points after the first 10 (which brought the player up to 2200) were cut in half.

                          But for this to really be a wash would depend on how far above 2200 the player's rating had jumped.
                          Here's an example:
                          Player A is 2190 and earns 12 points to end up with a rating of 2202.
                          Player B is 2190 and earns 80 points to end up with a rating of 2270.

                          [Player A is only 1 point ahead of where he would have been if the lower K value was applied to all rating gains over 2200, while Player B is 35 point ahead of where he would have been.]

                          Both players compete in several events, scoring the same performance rating in each, raising their ratings each time, so that Player A is rated 2220 and B is rated 2285. Then they each lose a one game match vs a 2000 player.
                          The way the CFC ratings are supposed to work:
                          A's rating should drop 12.4 points to 2208,
                          B's rating should drop 13.7 to 2272.

                          The way the CFC ratings actually do work:
                          B's rating does go to 2272, :o
                          A's rating drops to 24.8 points to 2195.

                          Obviously this is not "a wash" for Player A, who benefitted 1 whole point in his step over 2200, but dropped 24.8 points on his way down.

                          Suggestion: it might be interesting to compare the number of active players between say 2199-2170 with the number between 2200-2229 to see if it's the Bell-shaped drop off you'd expect, or if there are a lot more active players just below 2200. If so, that would show a systematic effect of this rating error.

                          I don't have any suggestion about fixing this apart from rewriting the rating program and copying whatever wins this competition: http://www.kaggle.com/chess
                          In Bill's case the "loss or gain cut in half" would only apply against the 1st 12 points, taking him to 2200, after that the loss would be at the normal rate.

                          Ditto for you.

                          By the same token someone 2190 who gains 30 points, is only going to be 2210.

                          Having said that, I'm not sure how the program is actually working . Bill has a printed copy of the program, so he could verify this.

                          Comment


                          • #43
                            Re: CFC rating system broken? [over/under2200]

                            Originally posted by Fred McKim View Post
                            In Bill's case the "loss or gain cut in half" would only apply against the 1st 12 points, taking him to 2200, after that the loss would be at the normal rate.

                            Ditto for you.

                            By the same token someone 2190 who gains 30 points, is only going to be 2210.

                            Having said that, I'm not sure how the program is actually working . Bill has a printed copy of the program, so he could verify this.
                            That's a relief.
                            Last edited by Gary Ruben; Saturday, 26th February, 2011, 06:26 PM.
                            Gary Ruben
                            CC - IA and SIM

                            Comment


                            • #44
                              Re: CFC rating system broken? [over/under2200]

                              Originally posted by Fred McKim View Post
                              Having said that, I'm not sure how the program is actually working . Bill has a printed copy of the program, so he could verify this.
                              It is clearly not working that way, as you can see from calculating the change to Bill's rating after the 2010 Canadian Open [I did this before making my previous post].

                              Bill isn't, AFAIK, a computer programmer, and didn't say anything about having a copy of the programming code, so I'm not sure he could discover the actual calculations being used... but we can do the calculations ourselves to see that the current CFC rating calculations are wrong.

                              I talked to Bill about this and, although he is the ratings auditor, he hasn't been able to get the CFC to fix this.

                              Is it possible that the rating calculation code is so old, or exists only in a compiled version with no copy of the source code, that noone now working for the CFC can figure out how it works?
                              Last edited by John Upper; Saturday, 26th February, 2011, 03:17 PM.

                              Comment


                              • #45
                                Re: CFC rating system broken? [over/under2200]

                                Originally posted by John Upper View Post
                                It is clearly not working that way, as you can see from calculating the change to Bill's rating after the 2010 Canadian Open [I did this before making my previous post].

                                Bill isn't, AFAIK, a computer programmer, and didn't say anything about having a copy of the programming code, so I'm not sure he could discover the actual calculations being used... but we can do the calculations ourselves to see that the current CFC rating calculations are wrong.

                                I talked to Bill about this and, although he is the ratings auditor, he hasn't been able to get the CFC to fix this.

                                Is it possible that the rating calculation code is so old, or exists only in a compiled version with no copy of the source code, that noone now working for the CFC can figure out how it works?
                                1) You can't rely on the published performance ratings as they are the "raw" performance that is used for unrated/provisional calculation. For example if you beat a 1400 player your pr for that one game is only 1800.

                                You have to go through game by game and do the math. Without using a calculator, I got about -87 points. Use 12 of those to to bring him from 2206 down to 2200 and then subtract 75 and he would be 2125, add 8 points for participation and you get 2133. Pretty close.

                                2) At the moment any changes done to the code, are by Vincent Chow on a contract basis. Removing the participation points was relatively simple, totally redesigning the code would be a major capital expense.
                                Last edited by Fred McKim; Saturday, 26th February, 2011, 04:08 PM.

                                Comment

                                Working...
                                X