CFC rating system broken?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • CFC rating system broken?

    On his blog, Kevin Spraggett presents some provocative arguments to suggest that the CFC's rating system is broken. He points to the recent performance of a top Canadian player and states that the CFC rating formula is much more generous than the formula used by FIDE. This he says will lead to high ratings that are just nonsensical.

    I trust that Kevin has done his calculations correctly and that the CFC is more generous with rating points. That raises an interesting question for me, namely, why does the CFC use a different rating formula than FIDE? Given that there seems to be an unstated preference among chess players to have consistent ratings across different systems (i.e. CFC 2348 should be comparable to FIDE 2348, for instance), using different rating formulas would seem to doom this.

    Perhaps some individuals who know more about the CFC rating system than I do can explain why it is different than what FIDE uses.

  • #2
    Re: CFC rating system broken?

    Originally posted by Steve Karpik View Post
    On his blog, Kevin Spraggett presents some provocative arguments to suggest that the CFC's rating system is broken. He points to the recent performance of a top Canadian player and states that the CFC rating formula is much more generous than the formula used by FIDE. This he says will lead to high ratings that are just nonsensical.

    I trust that Kevin has done his calculations correctly and that the CFC is more generous with rating points. That raises an interesting question for me, namely, why does the CFC use a different rating formula than FIDE? Given that there seems to be an unstated preference among chess players to have consistent ratings across different systems (i.e. CFC 2348 should be comparable to FIDE 2348, for instance), using different rating formulas would seem to doom this.

    Perhaps some individuals who know more about the CFC rating system than I do can explain why it is different than what FIDE uses.
    There is a bonus point system in place. For a long time the system awarded a certain number of points depending on how many points you ganied over a certain threshold. This was later modified, so that you wouldn't be given any bonus points if you were at your maximum achieved rating.

    In 2007/2008 the latest change to the bonus points were made

    B: Result Bonus (Performance Rating must exceed highest CFC Rating ever)
    0% - 59%: No points
    60% - 70%: 5 Points
    71% - 89%: 10 Points
    90%+: 15 Points

    I'm sure nobody thought about this at the time, but there probably should have been a cap put in place (ie only players below 2200 are eligible for bonus points). I will be putting forward such a motion at the next CFC online meeting, in early April.

    I have estimated that Bator's rating has gone up about 50 points because of bonus points, since Sept 1. He has probably just about reached a point where it will be impossible for him to get any more bonus points. He has never had a performance rating above 2800, but has been 2740+ in 4 of his last 7 events, probably unmatched in CFC rating history.

    I'd like to thank Kevin for being the person to pick up on this anomaly in the rating system. However, to suggest the rating system is broken ... I wouldn't go quite that far.

    Comment


    • #3
      Re: CFC rating system broken?

      Hi Steve

      Assuming the CFC needs to run its own rating system to help justify its existence, even if the CFC rating system went to an identical formula to FIDE's rating system, players' CFC and FIDE ratings would naturally be bound to be different sooner or later (if not immediately) after at least some events were rated under just one system (say the CFC's), as I think you might be saying.
      Anything that can go wrong will go wrong.
      Murphy's law, by Edward A. Murphy Jr., USAF, Aerospace Engineer

      Comment


      • #4
        Re: CFC rating system broken?

        Actually Steve, the rating system is working just fine. :) Kevin does tend to exaggerate on occasion. I have said on numerous occasion, and will repeat here again, occasionally the system kicks out a obvious rating error. When these are brought to our attention, they are corrected.

        As for Bator's high rating, that is mostly due to the fact that wins almost all his games. For those convinced he doesn't deserve his CFC rating, I challenge you to prove it by actually beating him over the board. :D

        We have discovered recently that Bator's rating has been helped a bit by the bonus point system that was implemented in 2008. To illustrate, if he wins all his games, and his performance rating is greater than his lifetime high CFC rating, then he would get an additional 15 rating points. This is an unintended side effect of the bonus point system. The purpose of the bonus points was to help increase the rating of underrated juniors. Unfortunately, there was never a upper rating limit placed on the formula. I believe Fred and Bill will be seeking a remedy soon.

        I would also point out that our rating auditor, Bill Doubleday, is examining the source code in the rating system to identify problems.

        The picture that some paint that we don't care about the integrity of the rating system is just wrong. We do, but we just can't fix everything first! :)

        ps. I see Fred can type faster than me. See his explanation above.

        Comment


        • #5
          Re: CFC rating system broken?

          Originally posted by Bob Gillanders View Post
          We have discovered recently that Bator's rating has been helped a bit by the bonus point system that was implemented in 2008. To illustrate, if he wins all his games, and his performance rating is greater than his lifetime high CFC rating, then he would get an additional 15 rating points. This is an unintended side effect of the bonus point system. The purpose of the bonus points was to help increase the rating of underrated juniors. Unfortunately, there was never a upper rating limit placed on the formula. I believe Fred and Bill will be seeking a remedy soon.
          IIRC, in the old bonus point system you couldn't earn bonus points if you were rated over 2000. I have no idea why this was removed.

          Comment


          • #6
            Re: CFC rating system broken?

            Originally posted by Bob Gillanders View Post
            Actually Steve, the rating system is working just fine. :) Kevin does tend to exaggerate on occasion. I have said on numerous occasion, and will repeat here again, occasionally the system kicks out a obvious rating error. When these are brought to our attention, they are corrected.

            As for Bator's high rating, that is mostly due to the fact that wins almost all his games. For those convinced he doesn't deserve his CFC rating, I challenge you to prove it by actually beating him over the board. :D
            Who you beat to get your rating points and the kind of event you play to get them matters. It really does.

            Players who take their high CFC ratings, crappy openings and cheapo tricky traps to Europe and try them out against strong players in decent events find this out quite quickly.
            Gary Ruben
            CC - IA and SIM

            Comment


            • #7
              Re: CFC rating system broken?

              Originally posted by Bob Gillanders View Post
              Actually Steve, the rating system is working just fine. :) Kevin does tend to exaggerate on occasion. I have said on numerous occasion, and will repeat here again, occasionally the system kicks out a obvious rating error. When these are brought to our attention, they are corrected.
              Clearly the rating system is broken because my rating is far lower than I want it to be...
              ...Mike Pence: the Lord of the fly.

              Comment


              • #8
                Re: CFC rating system broken?

                Perhaps we should arbitrarily multiply all rating by 100.... then people will be less inclined to compare them with fide ratings.

                People need to understand that because the pools are different, fide ratings and cfc ratings are not comparable.

                The fact that the top Canadians have higher cfc ratings then fide has nothing no significance whatever.

                Comment


                • #9
                  Re: CFC rating system broken?

                  Originally posted by Gary Ruben View Post
                  Who you beat to get your rating points and the kind of event you play to get them matters. It really does.

                  Players who take their high CFC ratings, crappy openings and cheapo tricky traps to Europe and try them out against strong players in decent events find this out quite quickly.
                  Hi Gary

                  Back in 1985 I had a 2300 range CFC rating and a high 2200 range FIDE rating. Not sure if this is what you have in mind for a *high* CFC rating. :)

                  I played in an open swiss event in Croatia (then part of Yugoslavia), along with a CFC class player I traveled with. There were at least 5 GMs, I think (the eventual winner was a young Danish IM).

                  We met up with a young British player with a 2300+ FIDE rating. He played openings like the London System as White and double king pawn as Black. I played 1.e4 as White and often more eccentric stuff, like 1.e4 Nc6, with Black. He got slightly tougher pairings throughout the event than I did. My toughest opponent may have been a wgm (I only drew her, after getting in a sound Bh7+ sac with White in a Sicilian, which won material). I finished half a point ahead of the Brit (we both had a plus score, but won no $). The locals 'thought' the Brit and I were not the greatest at endgames (he winced in disagreement when my traveling companion, a Croatian-Canadian, told him this, and I said nothing). Apparently there is sometimes stereotyping of players on both sides of the world. :)
                  Anything that can go wrong will go wrong.
                  Murphy's law, by Edward A. Murphy Jr., USAF, Aerospace Engineer

                  Comment


                  • #10
                    Re: CFC rating system broken?

                    Originally posted by Stuart Brammall View Post
                    Perhaps we should arbitrarily multiply all rating by 100.... then people will be less inclined to compare them with fide ratings.

                    People need to understand that because the pools are different, fide ratings and cfc ratings are not comparable.

                    The fact that the top Canadians have higher cfc ratings then fide has nothing no significance whatever.
                    I agree. I think that eliminating the results bonus will tend to discourage people who are concerned about their rating from playing in tournaments as much. There is always the gambler's thought that if I have a good day, I get a few extra points. If it is a zero sum game well maybe I only play when conditions are optimal which will tend to reduce entries and revenues for tournament organizers and the CFC.

                    Comment


                    • #11
                      Re: CFC rating system broken?

                      I still think, even more than 10 years after posting this for the first time, if the CFC must have a rating system then it should be one like bridge. Ratings only go up which increases participation. Why have the CFC duplicate something that FIDE does better?
                      "Tom is a well known racist, and like most of them he won't admit it, possibly even to himself." - Ed Seedhouse, October 4, 2020.

                      Comment


                      • #12
                        Re: CFC rating system broken?

                        Originally posted by Tom O'Donnell View Post
                        Why have the CFC duplicate something that FIDE does better?
                        Is it clear that the fide system works better?

                        Certainly it should be obvious that rating systems works better with large amounts of input data, and there are many more CFC games played then Fide games, to the point that everyone who is active in Canada probably has a more accurate cfc rating then a fide rating, provided those ratings are interpereted within the system.

                        Does anyone here know what the average standard deviation of performance-rating is for both the the fide and cfc systems?

                        Comment


                        • #13
                          Re: CFC rating system broken?

                          Originally posted by Stuart Brammall View Post
                          Is it clear that the fide system works better?

                          Certainly it should be obvious that rating systems works better with large amounts of input data, and there are many more CFC games played then Fide games, to the point that everyone who is active in Canada probably has a more accurate cfc rating then a fide rating, provided those ratings are interpereted within the system.

                          Does anyone here know what the average standard deviation of performance-rating is for both the the fide and cfc systems?
                          1) A CFC rating in Halifax cannot be compared to one in Calgary. Since there is almost no cross-pollination of players saying someone is "1800 CFC" is almost totally meaningless.

                          2) The CFC rating system has been manipulated, not to make it more accurate but because certain people "felt" their ratings were "too low" based on ... get this ... their own totally objective belief in their skills. At that point it ceased to be a way of measuring anything other than people's own sense of self-entitlement.

                          3) The only thing preventing there being more input data is that, understandably, people feel they don't want to pay to have the same games rated twice. They can't rate games with only FIDE because the CFC won't let them!
                          "Tom is a well known racist, and like most of them he won't admit it, possibly even to himself." - Ed Seedhouse, October 4, 2020.

                          Comment


                          • #14
                            Re: CFC rating system broken?

                            Originally posted by Tom O'Donnell View Post
                            1) A CFC rating in Halifax cannot be compared to one in Calgary. Since there is almost no cross-pollination of players saying someone is "1800 CFC" is almost totally meaningless.

                            2) The CFC rating system has been manipulated, not to make it more accurate but because certain people "felt" their ratings were "too low" based on ... get this ... their own totally objective belief in their skills. At that point it ceased to be a way of measuring anything other than people's own sense of self-entitlement.

                            3) The only thing preventing there being more input data is that, understandably, people feel they don't want to pay to have the same games rated twice. They can't rate games with only FIDE because the CFC won't let them!
                            I recognize all these points, but I do not still do not think it is obvious the fide system is "better" unless someone can show that the standard deviation of performance is smaller (that is to say, the ratings are more accurate) in the fide system then in the CFC system.

                            Also, I think it is obvious that switching to fide would not eliminate the regional pool problem.

                            There certainly have been rediculous modifications to the system in the past, but the mechanics of the system cause it to correct over time... provided we don't allow anymore rating boons, the system should maintain a respectable degree of integrity...

                            Comment


                            • #15
                              Re: CFC rating system broken?

                              On this topic, I did notice something (probably a typo) in the CFC Handbook.

                              Regarding the estimated ratings of unrated players used to calculate the performance of unrated players against other unrated players, I think the formula for this estimated pre-rating of unrateds should be AR-200+400(W-L)/N, not AR-200+400(W-L) as stated in the handbook.

                              AR in this formula is a sort of modified average rating of the section. Then I assume the -200+400(W-L) is supposed to give the unrated player an estimated rating somewhere between 200 above and 600 below this modified average, based on performance - which it does with the /N at the end. Otherwise, the estimate drops 400 pts for each loss, which could rapidly generate a negative estimated rating!

                              Here's the text:

                              717. Order of Calculation. When an event is rated, performance ratings under equation 714a are obtained, first for unrated players, then for provisionally rated players. For each rated player is calculated: Ro-400(W-L)/N. The mean of these is AR. When unrated plays unrated in this first pass, the opponent’s rating is estimated as AR-200+400(W-L). After this, the previously unrated players are rerated on the basis of the ratings of their opponents just calculated. Finally, the players having established ratings are rated, first those with ratings below 800. A game with a previously unrated or previously provisionally rated player enters the rating of a player with an established rating on the basis of the opponent’s post-event (new) rating. In the case of a player having an established rating below 800, the following is used to calculate the rating of opponents having established ratings 800 and above: the higher of the player’s pre- and post-event ratings.
                              Marcus Wilker
                              Annex Chess Club
                              Toronto, Ontario

                              Comment

                              Working...
                              X