in GR's defense, and for general sanity of discussing this topic, any examples/complaints about wild swings/inaccuracies based on things like solo'ing out of 1 game or drastically improving between games will ALWAYS be a challenge for a rating system, and shouldn't be used as arguments. Unless the system legitimately doesn't have a way to get more accurate the more and more data it acquires for a person, let's not worry about that argument of whether it's "true". But no, no rating system is going to give someone one an accurate "guess of skill" with N=1, especially N=1 where it was a solo.

Now, if someone takes umbrage on the scoring system on a level that it rewards incorrect behaviour, or this current discussion of whether ratings should be retro-active/far-reaching, that's another, legitimate discussion.

@Maher I appreciate your knowledge on this topic but I'm sure GR/ATC are aware of this and I assume they can handle it.

@Oct there's also value in people respecting the rankings because they're correct not because they're legacy. You think the current rankings are particularly correct?

I want to also restate this in case people missed it:

In fact would want to take into account the time delay between the games to decide how much of an effect it has. (So if the games are finishing 1 day apart, it makes a big difference, but if the gap is 2 years, there is no difference at all)

@Maher "I don't think the question here is if we want something better than the old option. I think the real question is whether it's worth it to the administrators of the GR to exponentially increase the difficulty in compiling these rankings. I'm sure we'd all love to have the most finely tuned system in the world. Is it worth that much extra labor to the administrators? "

I would be doing the labour in this project, so this isn't a concern. It is not wholly unrelated to the PhD I am currently doing, and I am supervising an MSc student whose thesis will be on the problem.

Obviously I'm shooting for polynomial time algorithms.

I like the idea. I also think the current GR is a poor descriptor, making it worthwhile.

if it wasn't obvious, I too support a change, and while the current far-reaching through time and space sounds like a good one, I wouldn't be adverse to also hearing about other possible changes.

I think this is an interesting idea, but my gut feeling is that It would have problems with player's skill changing between games as others have pointed out. Ghost's comment that you could have a time based decay for the size future games affect the rating in your old game is a good idea, but I'm not sure how well that would work with webdip- dumb decisions that you made months ago can affect your result today when that game finishes. Especially in a game like diplomacy, where newbies tend to make a lot of mistakes.

I also think that we can make improvements to GR's accuracy without needing to re-adjust ratings after more information about your opponents comes out. I've spoken before about wanting to adjust the way GR treats losses (currently, each loss is worth the same, regardless of who beat you).

Ghost- This is pretty timely- earlier this week, I was discussing improvements to GR with a friend who is looking at writing a paper on rating systems for n-player games where the outcome depends on the other players.

Perhaps there would be some opportunities for collaboration there?

I would not enjoy my rating to be influenced depending on how over or under performing my prior opponents are doing.

@ag7433

Thanks, this is the kind of thing I wanted to know if people cared about.

How do you feel about the following point?

In a sense, your rating is already affected by how well your opponents do in games, we just only limit that to games before yours. In particular, if an opponent loses a game just before your game with him finishes, it could make a significant difference to your rating.

Another element that could be a problem is metagaming. Currently, only the games that you are in will affect your rating - if this extended to games that you're not in, then we could end up with people trying to influence the outcome of other people's games via PM or whatever. I don't think that's an ideal situation.

TGM, I agree with your follow up point and prefer it that way.

To me the game (and results) should be considered as a snapshot in time. In this month, in this moment, with this group of people, at their current level, using their current availability of time to invest into the press, my ratings will be adjusted.

I can tell you that today I am much worse than I was 6 months ago. My time to invest in the press has been cut by 70% due to life circumstances. I will win less that before; I will draw less than before; when I draw the quantity of players included will be more than before. My rating will go down.

Am I a worse player today than 6 months ago? Absolutely. Should my former opponents get a ratings dip because of it? No way. They beat me fair and square and deserve the full credit -- and for it not to be adjusted because of my life circumstances.

Honesty Upfront: I haven't read all the replies.

Many of the rating systems that I have seen do not go back and change ratings as it is simply too complicated. All of the rating systems have a range (whether they show you it or not) so a rating of 100 really means something like 99-105 (if you have played a lot of games/matches. Or 80-120 if you have played less. Some rating systems actually do give (display) the upper and lower ranges to the user. Others use only the upper range, only the lower range or only the midpoint as the users ratings. I think the big problem with webDip ratings is that the games tend to last for a very long time and there are few data points. However, I personally think that if you try to go back and retro actively change everything you are going to open a can of worms. I personally expect that if I have achieved a rating of "X" that tomorrow it will also be "X" unless I finish a game. Having it change all the time I feel would be more annoying. There is one exception to this though. If you base the rating off the lowest point of the range then as time goes on if I don't play in games my rating will slow go down (it will start at 99-101 and display as 99 but will "decrease" to 80-120 and display as 80). Once I start playing again my rating will quickly return to normal in this situation. This has the benefit of slowly removing the inactive players from the top of the list but has the down side of underestimating players for purposes of seeding, etc. Of course if the mid point is used over time the players display rating won't change. I'm not sure if I answered the question or not but I don't think you should retroactively effect ones ratings.

@ In particular, if an opponent loses a game just before your game with him finishes, it could make a significant difference to your rating.

Such is life... the opposite could be true also... in the end it should all even out.

One other point that I remembered though that I think would help in systems like that that have long times between the beginning and the end of a game is to use two "snap shots in time". At the start of the game store everyone's ratings. Then at the end of the game store everyone's ratings. Use the average of these two to determine the players rating (or average rating) over the course of the game and use that to effect scores. This would be my biggest suggestion for improving the accuracy of the ratings!

@ag "Absolutely. Should my former opponents get a ratings dip because of it? No way. They beat me fair and square and deserve the full credit -- and for it not to be adjusted because of my life circumstances"

But then again, when Edi Birsan joined the site, and owned everyone, the people who drew with him early on didn't get the credit they deserved.

@TGM: Fair point. But give me the statistics of an Edi Birsan-type joining as a brand new player vs the majority of everyone else playing through the cycle of normal life

I especially dislike the idea that there could be an incentive for a sitter to play badly deliberately when sitting for someone, in order to give their own rating a boost. Whilst this would undoubtedly be very rare, it's not out of the question.

@ag

Edi is an extreme case, but this happens all the time: your rating lags behind your actual ability. The convergence time of GR is quite slow (improving this is another project), meaning that when you play a game, currently you are rated as if you played against that player's ability from ~5-10 games ago. (This is an estimate for the amount, not established by any numerical work)

The idea behind this would be that the best estimate of the ability of a player in their 10th game does not come from the first 9 games they play, but rather from the games 5-9 and 11-15 (for instance).


@Hellenic Riot This is an interesting point. In principle, if I play against player B, and then account sit for player C, I could help player B win a game, and so improve my rating.

Reducing someone's rating will not directly improve my rating, though.

I don't think this is a new problem. If I am in a game against player B, and sitting for player C, also in a game against B, I could help player B do well, and if the game finishes before mine, I benefit.

Surely you pick a sitter who you trust, and also, surely there are easier ways to cheat, with much bigger payoffs?

"What I want to know is:
Suppose there were a rating system which made these sorts of updates, and was more successful at predicting results than when you remove this. Would you prefer this, or having ratings only change when you've played?

So yeah, I don't want to know if you guys think it will work: I can evaluate that numerically. I want to know how you feel about it *given that is does work* "

First, thanks for all the work you put into compiling the GR.

Yes, a player's rating should attrite with the passage of time, but not by the method you are proposing to use. Firstly, a player's rating is a numerical estimate of the standard of a player's play at the time of the game in question, from which it follows that, if a player's rating changes as a result of that game or as a result of subsequent games, those changes should not be applied at a later date retroactively to the ratings of the other participants in the game.
If one wants to apply to a player's rating a factor which takes account of the time which has elapsed since the last game which contributed to that player's rating, one should simply apply to everyone's rating a factor which attrites the effect of every result depending on the period of time which has elapsed since that result was achieved, causing the rating to regress gradually towards the mean (unless further games are played).
Additionally or alternatively, one could introduce a factor which reduces the impact on a player's rating of a player's earliest games as each new game is played, but I think that may be in there already (??)

Thanks TGM, for a thought-provoking question.

I would agree with ag that, on a personal level, I would simply not enjoy my rating to switch without playing actual games. It detracts from the feeling of having 'earned' it.

On a more philosophical level, you can wonder what the objective of a rating system should be. One goal, that you are following, is to make an accurate predictor of outcomes, which by extension is an accurate measure of skill. Ideally, GR would change very little.

I'd argue, on a gaming site like this, that the goal of a rating should not be accuracy, but an incentive for a certain style of play. I would say that the style should be active, competitive and maximising outcomes (meaning solo-driven). Going on this path leads me to other rating systems, that don't necessarily reflect skill at all. As is well known, a downside the of Elo style systems is sitting on high ratings (like some of us, you know who you are), or not taking too much risk playing low Elo players (because, what if you lose?). I for one, would not mind a rating that decreases if you don't play, and increases when you play a diversity of opponents. Neither of which have to do with skill. Also, echoing HR, ratings should be more or less transparent to all users, and with this time factor included, I think it becomes less transparent.

Having said that, I do see the value of trying to measure skill. It may be worth a thought if we should not have two parallel rating systems on the basis of these two objectives.

This is indeed a very interesting question.

Strictly speaking, those games that I did not participate in still provide information about my skill. The information will probably be quite a bit noisier though due to people learning and natural noise in the game, so that would need to be taken into consideration.

If your goal is to make an accurate predictor, then intelligently including all the information accessible about my skill will be helpful. If your goal is to give people a number so they can be proud of it/use it as a talking point/ect, then I think it would be more of a concern due to the numerous reasons mentioned above.

Unrelated – the skill+uncertainty thing has been brought up many times, but it seems like it would be low hanging fruit in terms of increasing accuracy of a rating system. Probably much simpler to implement and justify mathematically than the current discussion(which I still think is awesome).

I agree with the sentiment generally going around that the goal shouldn't be to alter a players score based on games he was not apart of. If I left the site for a year, or 5 years, during my lapse, there's no guarantee I play games and hone my skill during that time. I could come back flat, but my rating would potentially improve to keep up with current trends.

What I would like to see is a system that effectively removes the small inflation that occur in ELO games that tends to favor people that play hundreds of games compared to those that don't play as frequently. The way GR stands now, the top players on the site have GRs that continue to climb when their scores should remain relatively fixed until someone that has a similar score can challenge them.

Ideally, there should be a "perfect value" that players approach that takes into consideration their recent play (maybe force GR to examine the last 50 games or so). To obtain such a score, a player plays all 50 games against highly qualified players and wins all 50, but for anything else, has a score less than the perfect value.

I used to play Catan on a site that had a system like ELO that caused your score to improve when winning against similar competition, go up more dramatically when winning against players of higher skill, go down only a little for losing against players of equal skill and go down recognizably for playing against those of lower skill. That's ideal to me.

"go down recognizably for playing against those of lower skill"

But would that not incentivize only playing against the same old cadre of high-ranked players? As a community, I would espouse an incentive to play a diversity of opponents. As well as the simple fact that it is simply more enjoyable.

The reverse is also true: a new good player may find himself unable to ever play with any of the established good players, since they would risk their rating playing him. Hence, even if his skill is *actually* high, this may take quite a while to be reflected in the ratings.

Anyone who values their rating so much that they refuse to play people in an effort to preserve it aren't worth playing. It is an excellent filter to keep highly GR'd prats from ruining a game. Not that I think there are many of them.

I agree Oct. But it highlights that an attempt to accurately measure skill of the established players may hamper measuring skill of newcomers.

I don't have the feeling at all that we're the type of community that has this mentality, but I do feel that any system that's attached a value on play should attach a value on the *preferred type* of play.

I think I should mention: This is a small question within a much larger set of changes I am researching, so if you're thinking "there are much easier and less controversial improvements you could make", I agree :)

I think that the rating on any open site is not a predictment of future results but an analisys on past games. You can´t really predict nothing with people coming in an out all the time. Plus, you need a system that can be explained for a normal human being. I don´t think polynomial time algorithms would fit on this.

So, to answer your question, I don´t want a system where other future games where I am not in affect my rating. When I choose to play a game I know what is on stake, but what the other players do after this game ends should not affect me. Of course, if a game takes too long there is risk the rating for the other players can change dramatically, but this could be dealt in the other ways.

To me, the main drawback on GR is that it doesn´t take account on who you played when you lose. It works well when you draw or you win, but when you lose it doesn´t matter if you played the top people on site or a bunch of noobs. So, harvesting GR in low level games is quite easy. And I will not even talk about what I think about apllying it to PPSC games.....

"To me, the main drawback on GR is that it doesn´t take account on who you played when you lose. It works well when you draw or you win, but when you lose it doesn´t matter if you played the top people on site or a bunch of noobs. So, harvesting GR in low level games is quite easy. And I will not even talk about what I think about apllying it to PPSC games....."

This has been mentioned a few times, and it's so, so wrong. I'm working, so don't have the time to explain- hopefully someone else can?