There was a bug in the update to the Hearts ratings last week which caused the losing players to get more points deducted from their Elo ratings than they should have. My apologies, I know some of you take the ratings very seriously!
I've applied a fix to the problem this morning. Anyone who would like to get their Elo ratings set back to where they were last Wednesday need only write to me and ask (marya@worldofcardgames.com). To do this, just make sure that you send me the email from the address that you registered with, so I know it's you. I will do this for people up until the end of November.
[Edit on Nov 24 2015: Only a small number of requests to reset Elo ratings have come in. After receiving negative feedback on the way this was being done, I decided to just go ahead and do the work to perform a bulk update of Elo ratings. Anyone who played at a ranked table prior to Nov 17 has had their Elo rating reset to the value from Nov 11, before the bug was introduced. To be clear, this was only done if it led to an increase in their ratings - I did not reset values that were lower.]
I've written a page about how the Elo ratings are computed at World of Card Games, so you can always check for yourself to make sure you are getting the correct results.
I've also applied a fix for the "list of tables", which had a bug that caused the list to stop updating. Third times the charm? I hope so!
13 comments:
Since, by tripling the amout of ratings points deducted from losing players, virtually everyone who played during the six days that your program was in error lost points, often triple digits, the default should have been to reset player ratings, not resetting a player's rating only if they requested it.
No matter what I do, I'm sure there will be people who are unhappy. If I had changed the ratings in bulk, I would have gotten complaints from people who did not see a benefit from it.
I have queried a number of players directly, and most people seem to have no interest in getting their Elo ratings reset. It was just a handful who requested it, although most of the people I talked to were aware of the blog post.
I took a look, and the default of resetting player ratings would put some people at 1600+ rating, although they had quit games during the period described. Is it "fair" to reset a player to 1600+ after they've lost heavy points due to quitting a game? The fact is, it is essentially impossible to fix the situation, but I will reset Elo for those who care enough to request it.
There is a reason that I am not too concerned about resetting Elo ratings. They are self-adjusting. If your Elo rating is low, but you are a good player, your Elo rating will increase over time. You should not find that your long-term Elo rating is impacted by a short run of bad luck. The miscalculation over a period of 6 days is similar to a short run of bad luck for some players; anyone who keeps playing and is good should find that they recover - if Elo ratings are meaningful at all.
>> No matter what I do, I'm sure there will be people who are unhappy.<<
Everyone who did not win at least half their games suffered a ratings loss. I don't believe anyone is losing sleep over that, but I also don't believe anyone would be unhappy if you had corrected the error.
>>If I had changed the ratings in bulk, I would have gotten complaints from people who did not see a benefit from it.<<
Who would that be exactly? Do you mean players who kept a high rating by not playing during those six days? Irrelevant complaints can generally be safely ignored.
>>I have queried a number of players directly, and most people seem to have no interest in getting their Elo ratings reset.<<
A "number of players" does not equal "most people" (tell me how many players you queried, how many rated players you have, and I'll calculate the margin of error in your "poll"). And I do wonder how you posed the question, since it seems unlikely that anyone, let alone "most people" would not simply have said "yes" if you had explained the program error, then offered to increase their rating. In any case, as I said, correcting the error should have been the default, simply because it IS an error. If any players actually object to having their rating corrected, fine ... let them email you. As it is, many players seldom read this blog (some speak little or no English) and more than a few, I am sure, are simply unaware of the error (just as you were, for three days).
>>s it "fair" to reset a player to 1600+ after they've lost heavy points due to quitting a game?<<
I don't believe anyone would complain and I also don't believe it actually matters since, if they are in the habit of quitting, their rating will not stay above 1600 for long, since you have Elo set up to punish quitters (a mistake, in my opinion, but that's another subject).
>>The fact is, it is essentially impossible to fix the situation<<
You have player ratings from before and after the six days your program was in error. Switch them. Then adjust them for any ratings change since the program correction. That last step, I assume, would be very time consuming and, perhaps, why you say such a fix is impossible. It had to be done, I guess, at the same time you corrected the program error (something that was suggested to you).
>>There is a reason that I am not too concerned about resetting Elo ratings. They are self-adjusting. If your Elo rating is low, but you are a good player, your Elo rating will increase over time.<<
Of course, but the amount of time may be significant. I know one player who lost approximately 250 ratings points. She did email you for a correction, btw, making her NOT one of those "most people" who play rated games even though they consider ratings irrelevant . Anyway, how long will it take a good player, let alone an average player, to make up a deficit of 250 points?
But it's not a disaster, even though we may have lost some players, and I am perhaps being overly critical, I really do appreciate your new ratings system very much (I doubt that anyone argued harder for that one than I). I still believe that it has the potential to increase both the number of players at your site and the overall level of play, and that is my only interest in this.
-drdammit
Who would that be exactly? Do you mean players who kept a high rating by not playing during those six days? Irrelevant complaints can generally be safely ignored.
There are people who played during that period who gained in Elo ratings.
You have player ratings from before and after the six days your program was in error. Switch them. Then adjust them for any ratings change since the program correction
Adjusting the Elo ratings correctly for games played after Nov 17 is beyond my capacity, and I will not do that.
Anyway, how long will it take a good player, let alone an average player, to make up a deficit of 250 points?
I don't know. I could write a script to figure this out, since I'm curious about this as well, but I don't want to put any more effort into contemplating such questions at this time. I have lots of other things to work on.
I recognize that updating Elo ratings on a case-by-case basis is also not really fair. The benefit received for each player will be different depending on when they ask. So I decided to just go ahead and do the work to perform a bulk update.
Anyone who played at a ranked table prior to Nov 17 has had their Elo rating reset to the value from Nov 11, before the bug was introduced. To be clear, this was only done if it led to an increase in their ratings - I did not reset values that were lower.
This change affected 73 players. The average change was 128 Elo rating points.
I hope people are sufficiently happy with this, but assume I will never know. I don't plan on making more changes that affect ranked Hearts tables any time in the near future.
I think that was good decision, Marya. Thank you for thinking about those who have not read the blog or who have limited english.
I agree. Thank you, Marya.
- drdammit
*sent from Dr. Evil's secret volcano lair
"Bring in the clone!"
I do wish you would make the existence of ranked games more obvious to guests and new members before you abandon the project. Would it be difficult to make rated games visible to guests, then display an invitation to become members if they select it? Is there any reason not to do that?
For new members, allow them to see that there ARE ranked games, then explain to them, if they try to play one, why you won't allow that (your fear that quitters will avoid their 4-hour banishment, I guess...shiver).
Rated games will not attract new members if only current members know that they exist.
- drdammit
Why not display ratings and %abandoned games on the table prior to play? They should disappear once play begins, but play should not begin, even when the table is full, until all four players click "Start" (or whatever). If a player has not clicked "Start" within 30 seconds, plenty of time to check out their opponents, boot 'em.
The idea is to allow players to opt out of a game if they object to another player's numbers. One consequence would be to discourage quitting, by the way, since players with high abandoned rates will, of course, be unpopular (this was how Yahoo dealt with quitters... simple and effective).
Why not?
- drdammit
Many of these items are on my to-do list. I will get to them eventually.
Thank you.
ps
Is the -10JD option still on your list? It's a very interesting game, and my favorite as it happens.
-drdammit
It's still on the list. I don't know when I'll get to it, however. I've done a lot of work on Hearts, and other items have a higher priority now.
grrrr...
Post a Comment