Page 1 of 20 12311 ... LastLast
Results 1 to 15 of 293
  1. #1
    CC Grandmaster
    Join Date
    Jun 2004
    Posts
    5,672

    Rating computation suggestion

    I have a suggestion that I may have mentioned before.

    Is it possible to implement the Glicko calculations such that ratings are updated after every game? That is, instead of using the last published rating for every rating calculation in a period, you instead keep track of updated ratings and use the "current rating" for each calculation.

    I believe this is the way Glicko is intended to be calculated, and indeed the way it is calculated on FICS.

    This method of calculation is one way of avoiding the large swings in rating that can occur due to the time between lists. E.g if you have a number of poor performances throughout a rating list, you will lose points based on your initial high rating for each game you play. If ratings are recalculated after each game, then losses later in the list will be lower due to the reduction in rating that has already occurred (argument also holds for increases in rating due to prolonged good performance).

    In order for this to work, you ideally need some "time" information in order to place the games in order for processing. The best result would be if games are sequenced individually, but I realize it may not be possible to extract separate date info for each game. Sequencing can be done on a tournament by tournament basis - it may lead to slightly inaccurate ordering (for example if people are playing weekly in one tournament and also in a weekender), but this would still be better than the present batch processing.

    Processing the games this way would definitely take longer, but should still be feasible.

    The main downside is that it becomes more difficult for people to estimate their rating change through a period, since they don't know what rating will be used for their opponent in a particular game. This isn't a big deal at the moment however, since any calculation is approximate unless you know everybody's RD and volatility precisely.

    Any thoughts on this approach in principle?

    Pax


    p.s

    Come to think of it, the pronciple difficulty is probably getting tournament organisers to submit their results punctually. When tournaments are submitted months late, this would obviously mess with the system. Still, it messes with the system as it stands in any case.

  2. #2
    Illuminati Bill Gletsos's Avatar
    Join Date
    Jan 2004
    Location
    Sydney
    Posts
    17,093
    Quote Originally Posted by pax
    I have a suggestion that I may have mentioned before.

    Is it possible to implement the Glicko calculations such that ratings are updated after every game? That is, instead of using the last published rating for every rating calculation in a period, you instead keep track of updated ratings and use the "current rating" for each calculation.

    I believe this is the way Glicko is intended to be calculated, and indeed the way it is calculated on FICS.
    Glicko's documents make it clear he expects a group of results to be batched and not calculated one game at a time.

    Quote Originally Posted by pax
    This method of calculation is one way of avoiding the large swings in rating that can occur due to the time between lists.
    Actually it doesnt.
    I looked at this years ago and it makes very little difference for the majority of players. It has an adverse effect on rapidly improving players.

    Quote Originally Posted by pax
    E.g if you have a number of poor performances throughout a rating list, you will lose points based on your initial high rating for each game you play. If ratings are recalculated after each game, then losses later in the list will be lower due to the reduction in rating that has already occurred (argument also holds for increases in rating due to prolonged good performance).
    This isnt correct.

    Quote Originally Posted by pax
    In order for this to work, you ideally need some "time" information in order to place the games in order for processing. The best result would be if games are sequenced individually, but I realize it may not be possible to extract separate date info for each game. Sequencing can be done on a tournament by tournament basis - it may lead to slightly inaccurate ordering (for example if people are playing weekly in one tournament and also in a weekender), but this would still be better than the present batch processing.
    This is nigh impossible to organise.

    Quote Originally Posted by pax
    Processing the games this way would definitely take longer, but should still be feasible.
    There is no actual advantage in doing this.

    Quote Originally Posted by pax
    The main downside is that it becomes more difficult for people to estimate their rating change through a period, since they don't know what rating will be used for their opponent in a particular game. This isn't a big deal at the moment however, since any calculation is approximate unless you know everybody's RD and volatility precisely.
    The main downside is that your approach has significant impact on rapidly improving players.


    Quote Originally Posted by pax
    Any thoughts on this approach in principle?
    Yes, it isnt going to happen.

  3. #3
    CC Grandmaster Alan Shore's Avatar
    Join Date
    Jan 2004
    Location
    Crane, Poole & Schmidt
    Posts
    3,871
    Looks like someone's been reading up on their Edwards, eh Bill?

    I'm guessing it's the suggestions to change the rating system that's given you ants in your pants, right?
    "I can't go back to yesterday because I was a different person then."
    - White Queen, Alice through the Looking-Glass

  4. #4
    CC FIDE Master bobby1972's Avatar
    Join Date
    Dec 2003
    Posts
    576
    gliko is ok leave it alone ,me and 2 of my mates have been over 2000 many times thanks to gliko .we love it

  5. #5
    Illuminati Bill Gletsos's Avatar
    Join Date
    Jan 2004
    Location
    Sydney
    Posts
    17,093
    Quote Originally Posted by Bruce Dickinson
    Looks like someone's been reading up on their Edwards, eh Bill?

    I'm guessing it's the suggestions to change the rating system that's given you ants in your pants, right?
    Nope.
    I'd looked at this issue ages ago.
    Originally back before we implemented Glikco and again before implementing Glicko2.

  6. #6
    CC Grandmaster
    Join Date
    Jun 2004
    Posts
    5,672
    Quote Originally Posted by Bill Gletsos
    Glicko's documents make it clear he expects a group of results to be batched and not calculated one game at a time.
    Theoretically, it is far better to calculate one game at a time. Can you reference the quote? It certainly is true that the calculations can still take place in a batch, it's just that intermediate ratings are calculated at each step, not just rating changes which are summed at the end.

    It's quite obvious that the best rating change for a particular game will be calculated if the most up to date information is used. The most up to date information should incorporate the information for all games prior.

    Actually it doesnt.

    I looked at this years ago and it makes very little difference for the majority of players. It has an adverse effect on rapidly improving players.
    The only adverse effect it would have is to prevent the calculated rating from overshooting the true rating. This can clearly happen in the current system.


    This isnt correct.
    Great. That's a really helpful comment. Why isn't it correct? Barry Cox's Glicko estimator certainly seems to suggest this is the case.

    Example: Player A 1700! plays a series of games against 1500!! players. For every win , 11 points are gained, for every loss 35 points are lost. This number is the same regardless of the number of games played in the period. After three losses, player A is effectively only 1600, but continues to lose 35 points for every loss rather than the 28 he would lose if his rating was actually 1600. There will be similar effects if RD is not updated until after all the games in the period are processed.

    This is nigh impossible to organise.
    Individual game dating would be difficult, I agree. Tournament dating is surely standard information. Then sequence tournaments by completion date, and games by round number. It's not perfect, but it's ok and not at all hard to implement.

    There is no actual advantage in doing this.
    There is an advantage. Namely, that it results in calculations that has one extra layer of approximation removed. Glickman's formulas have layer upon layer of approximation as it is, and this is one layer that is easily fixed in the calculation process.

    The main downside is that your approach has significant impact on rapidly improving players.
    Like I said, the only impact is to avoid artificial overshoot.

    e.g player B 1300? plays a series against 1700!! players. Each win earns 150 points, and each loss loses 15 points. These numbers stay the same even after a few wins (until the point when rating equals performance rating).

    Note that the problem that causes ratings to potentially increase above the performance rating for a period (or drop below performance rating) does not occur if you calculate game by game. This is the problem that you have admitted has had to be fixed artificially. It is symptomatic of a more general problem with batch processing the ratings.


    Yes, it isnt going to happen.
    Yes, that much is perfectly clear. You don't have to be so supercilious about it though. The suggestion is meant entirely in good spirit.

    Pax

  7. #7
    CC Grandmaster
    Join Date
    Jun 2004
    Posts
    5,672
    Quote Originally Posted by Bill Gletsos
    Glicko's documents make it clear he expects a group of results to be batched and not calculated one game at a time.
    Ok, I found it. From Glickman's web site:

    "To apply the rating algorithm, we treat a collection of games within a ``rating period'' to have occurred simultaneously. A rating period could be as long as several months, or could be as short as one minute. In the former case, players would have ratings and RD's at the beginning of the rating period, game outcomes would be observed, and then updated ratings and RD's would be computed at the end of the rating period (which would then be used as the pre-period ratings and RD's for the subsequent rating period). In the latter case, ratings and RD's would be updated on a game-by-game basis (this is currently the system used by FICS). The Glicko system works best when the number of games in a rating period is moderate, say an average of 5-10 games per player in a rating period. The length of time for a rating period is at the discretion of the administrator."

    Ok, games in a 'period' are treated simultaneously. But he does say, that there's nothing stopping a 'period' from being defined as a single game. Glickman says that the system works best with 'moderate' periods (5-10 games). That would suggest treating each tournament as a period for the purposes of the rating computations. I still believe that the separate game option is the better.

    Note that the last list has one player with 78 rated games, several with over 50 games and many more over 30. Most active players in the period played more than 10 games.

    Pax

  8. #8
    Illuminati Bill Gletsos's Avatar
    Join Date
    Jan 2004
    Location
    Sydney
    Posts
    17,093
    Quote Originally Posted by pax
    Theoretically, it is far better to calculate one game at a time.
    You would have an impossible time supporting this by any statement by Glickman where he says this in his papers.

    Quote Originally Posted by pax
    Can you reference the quote?
    I never said it was a quote.

    Quote Originally Posted by pax
    It certainly is true that the calculations can still take place in a batch, it's just that intermediate ratings are calculated at each step, not just rating changes which are summed at the end.
    So what it makes virtually no difference and certainly does not warrant the extra effort involved in attempting to rate the games individually.

    Quote Originally Posted by pax
    It's quite obvious that the best rating change for a particular game will be calculated if the most up to date information is used. The most up to date information should incorporate the information for all games prior.
    Check out his formula. It all revolves around summing game results in a period where games are from 1...n.


    Quote Originally Posted by pax
    The only adverse effect it would have is to prevent the calculated rating from overshooting the true rating.
    Incorrect. The rating for a rapidly improving player does not increase as dramatically if you rate it game by game.

    Quote Originally Posted by pax
    This can clearly happen in the current system.
    Irrelevant to the point at hand.


    Quote Originally Posted by pax
    Great. That's a really helpful comment. Why isn't it correct? Barry Cox's Glicko estimator certainly seems to suggest this is the case.
    Easy.
    Barry Cox's calculator is wrong under these circumstances because it does not allow for the different c squared value required to age the rating between games if you rate the games one game at a time.


    Quote Originally Posted by pax
    Example: Player A 1700! plays a series of games against 1500!! players. For every win , 11 points are gained, for every loss 35 points are lost. This number is the same regardless of the number of games played in the period.
    If you are going to make statements like this as though they are fact then be damn sure you know what your talking about.
    Unfortunately for you, you dont.

    Your figures are Incorrect.
    You clearly have not done the actual calculations.
    Each win does not gain 11 points. Successive wins give rise to reduced increases per win. The same is true for losses.

    In fact its clear you didnt even use Barry Cox's calculator, or if you did you did so incorrectly.
    Using his web page calculator it is obvious that a 1700! beating a 1500! in just one game gives rise to a change of 13 points. However if the same 1700! beats the 1500!! 5 times the change then averages 11 points per game for a total increase of 55 points. If he had happened to win 10 games the average increase drops to 8.9 points per game for a total increase of 89 points. thats a far cry from you supposed 110 points.


    Quote Originally Posted by pax
    After three losses, player A is effectively only 1600, but continues to lose 35 points for every loss rather than the 28 he would lose if his rating was actually 1600. There will be similar effects if RD is not updated until after all the games in the period are processed.
    Incorrect.

    Quote Originally Posted by pax
    Individual game dating would be difficult, I agree. Tournament dating is surely standard information. Then sequence tournaments by completion date, and games by round number. It's not perfect, but it's ok and not at all hard to implement.
    There is no significant advantage in doing this.


    Quote Originally Posted by pax
    There is an advantage. Namely, that it results in calculations that has one extra layer of approximation removed. Glickman's formulas have layer upon layer of approximation as it is, and this is one layer that is easily fixed in the calculation process.
    It isnt warranted.


    Quote Originally Posted by pax
    Like I said, the only impact is to avoid artificial overshoot.
    Like I said that isnt the impact.

    Quote Originally Posted by pax
    e.g player B 1300? plays a series against 1700!! players. Each win earns 150 points, and each loss loses 15 points. These numbers stay the same even after a few wins (until the point when rating equals performance rating).
    Another apparent statement of fact that is totally false.
    Same reasons as above.

    Quote Originally Posted by pax
    Note that the problem that causes ratings to potentially increase above the performance rating for a period (or drop below performance rating) does not occur if you calculate game by game. This is the problem that you have admitted has had to be fixed artificially. It is symptomatic of a more general problem with batch processing the ratings.
    Incorrect.
    You clearly have not done any actual calculations.
    This problem can occur in game by game or batch pricessing. It even happens in the ELo system irrespectve of whether you do game by game or batch.

    Quote Originally Posted by pax
    Yes, that much is perfectly clear. You don't have to be so supercilious about it though. The suggestion is meant entirely in good spirit.
    Not true.
    If I was being supercilious or even sarcastic I would have not put a at the end of the sentence.

  9. #9
    Illuminati Bill Gletsos's Avatar
    Join Date
    Jan 2004
    Location
    Sydney
    Posts
    17,093
    Quote Originally Posted by pax
    Ok, I found it. From Glickman's web site:

    "To apply the rating algorithm, we treat a collection of games within a ``rating period'' to have occurred simultaneously. A rating period could be as long as several months, or could be as short as one minute. In the former case, players would have ratings and RD's at the beginning of the rating period, game outcomes would be observed, and then updated ratings and RD's would be computed at the end of the rating period (which would then be used as the pre-period ratings and RD's for the subsequent rating period). In the latter case, ratings and RD's would be updated on a game-by-game basis (this is currently the system used by FICS). The Glicko system works best when the number of games in a rating period is moderate, say an average of 5-10 games per player in a rating period. The length of time for a rating period is at the discretion of the administrator."

    Ok, games in a 'period' are treated simultaneously. But he does say, that there's nothing stopping a 'period' from being defined as a single game. Glickman says that the system works best with 'moderate' periods (5-10 games). That would suggest treating each tournament as a period for the purposes of the rating computations.
    Actually it doesnt.
    As Glickman notes with regards the Glicko2 system he states "The Glicko-2 system works best when the number of games in a rating period is moderate to large, say an average of at least 10-15 games per player in a rating period".
    First it was 5-10 now 10-15. Based on your logic that makes it two tournaments.

    Quote Originally Posted by pax
    I still believe that the separate game option is the better.
    Believe what you like it does not make it true.

    Quote Originally Posted by pax
    Note that the last list has one player with 78 rated games, several with over 50 games and many more over 30. Most active players in the period played more than 10 games.
    The number of games that it works best for is dependant on a number of factors. As Glickman said in an email you have to experiment with the various parameters to determine what works best in your circumstances.
    This is especially true with the Glicko2 system.

  10. #10
    Account Permanently Banned PHAT's Avatar
    Join Date
    Jan 2004
    Location
    Wollongong
    Posts
    4,254
    Quote Originally Posted by Bill Gletsos
    As Glickman said in an email you have to experiment with the various parameters to determine what works best in your circumstances.
    FMD. Isn't Glicko capable of "handling" normal circumstances such as pooling.

    BTW, could you name these parameters, and tell us how they effect the "handling" of pooling.

  11. #11
    Illuminati Bill Gletsos's Avatar
    Join Date
    Jan 2004
    Location
    Sydney
    Posts
    17,093
    Quote Originally Posted by Matthew Sweeney
    FMD. Isn't Glicko capable of "handling" normal circumstances such as pooling.
    Who mentioned pooling.

    Quote Originally Posted by Matthew Sweeney
    BTW, could you name these parameters, and tell us how they effect the "handling" of pooling.
    I'm not your nor anyone else's lecturer/research assistant.

    Again I ask, who mentioned pooling.
    If referring to processing a number of games in a rating period, then I'll leave that as an exercise for the interested reader. Of course to anyone who reads Glickmans Glikco and Glicko2 papers and understands them then the answers should be obvious.

  12. #12
    CC Grandmaster
    Join Date
    Jun 2004
    Posts
    5,672
    Quote Originally Posted by Bill Gletsos
    Your figures are Incorrect.
    You clearly have not done the actual calculations.
    Each win does not gain 11 points. Successive wins give rise to reduced increases per win. The same is true for losses.

    In fact its clear you didnt even use Barry Cox's calculator, or if you did you did so incorrectly.
    Using his web page calculator it is obvious that a 1700! beating a 1500! in just one game gives rise to a change of 13 points. However if the same 1700! beats the 1500!! 5 times the change then averages 11 points per game for a total increase of 55 points. If he had happened to win 10 games the average increase drops to 8.9 points per game for a total increase of 89 points. thats a far cry from you supposed 110 points.
    Ok, my example was not accurate, but the fundamental point I'm making is still true.

    The reduced increases per game arise from an effective adjustment in RD, not an adjustment to the rating. Note that changes in both directions are reduced, and the contribution of games at the start of the period is the same as games at the end of the period.

    Here is a better example:

    Player A 1500! plays ten games against players 1500!!. if the sequence is 1010101010, the change is 16 points per game (up or down depending on win and loss). If the sequence is 1111111111, the gain is still 16 points per game.

    What I'm saying, is that after 9 wins against 1500 players you have a pretty good idea that this player is correctly rated higher than 1500, and therefore the gain for the tenth win should be lower than the gain for the first win. If you calculate game by game, then by the time the tenth game is calculated, the information from the first nine games is encoded in the ratings used for the tenth game.

    Pax

  13. #13
    Reader in Slood Dynamics Rincewind's Avatar
    Join Date
    Jan 2004
    Location
    The multiverse
    Posts
    21,575
    Quote Originally Posted by pax
    Ok, my example was not accurate, but the fundamental point I'm making is still true.

    The reduced increases per game arise from an effective adjustment in RD, not an adjustment to the rating. Note that changes in both directions are reduced, and the contribution of games at the start of the period is the same as games at the end of the period.

    Here is a better example:

    Player A 1500! plays ten games against players 1500!!. if the sequence is 1010101010, the change is 16 points per game (up or down depending on win and loss). If the sequence is 1111111111, the gain is still 16 points per game.

    What I'm saying, is that after 9 wins against 1500 players you have a pretty good idea that this player is correctly rated higher than 1500, and therefore the gain for the tenth win should be lower than the gain for the first win. If you calculate game by game, then by the time the tenth game is calculated, the information from the first nine games is encoded in the ratings used for the tenth game.
    When you reduce the batch size down to 1 game you are not guaranteed to get a more accurate rating, just a more volatile one. Your argument would have more weight if it could be shown that the resulting system did lead to greater accuracy in ratings and also if you could suggest a way of overcoming the logisitical problems of getting games rated within 7 days of being played. Otherwise I could be playing against someone who's current rating is 1500 but won 10 games last weekend which were not rated.

    Yes this happens now but to a lesser degree as there is a conscious effort to round up results and include them when ratings are about to be published. If this was just an ongoing chore I could see lots of results falling through the cracks for weeks, months and even forgotten about altogether.
    So einfach wie möglich, aber nicht einfacher - Albert Einstein

  14. #14
    CC Grandmaster
    Join Date
    Jun 2004
    Posts
    5,672
    Quote Originally Posted by Bill Gletsos
    This problem can occur in game by game or batch pricessing. It even happens in the ELo system irrespectve of whether you do game by game or batch.
    Really? Can you give an example where such overshoots happen in either Glicko or ELO when calculated game by game? It is very easy to construct such examples in batch calculations, but it's certainly less obvious in the game by game scenario.

    Pax

  15. #15
    CC Grandmaster
    Join Date
    Jun 2004
    Posts
    5,672
    Quote Originally Posted by Barry Cox
    When you reduce the batch size down to 1 game you are not guaranteed to get a more accurate rating, just a more volatile one. Your argument would have more weight if it could be shown that the resulting system did lead to greater accuracy in ratings and also if you could suggest a way of overcoming the logisitical problems of getting games rated within 7 days of being played. Otherwise I could be playing against someone who's current rating is 1500 but won 10 games last weekend which were not rated.
    You misunderstand my point.

    I'm not suggesting that the ratings be actually calculated in real time. What you do is collect all the results for a period (say three months), you place the games in order and process them one by one. As I say, the ordering will be approximate when tournaments overlap.

    I would have thought my suggestion would make ratings less volatile rather than more. This is because when a rating is going up (or down), the calculations are always based on all of the available information up to that point in time (i.e current calculated ratings).

    Pax

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Underrated Juniors
    By Paul S in forum Ratings Arena
    Replies: 627
    Last Post: 17-02-2008, 08:43 PM
  2. Planned Rating Changes
    By Bill Gletsos in forum Ratings Arena
    Replies: 415
    Last Post: 30-07-2004, 01:00 AM
  3. Best posts of 2003
    By paulb in forum Australian Chess
    Replies: 289
    Last Post: 29-03-2004, 10:54 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •