It took me over 5 years to play 100 games...

I'm at 0.64 games completed a day on average.

I haven't read the thread, but i agree wholeheartedly with ssorenn. I have an RR of 90% which is apparently really bad. Most of that is from games I messed up back in 2010 when I just joined the site and didn't yet realize that resigning games once you have 1/2 units is frowned upon.

How about

90% is great! you got an A! you graduate! career! yay!

WFW

+1 to everything ATC said.

@abgemacht Yeah, I'm not sure percentiles is actually what we want - the goal is to be able to tell if a player is likely to be "reliable", i.e., gets their moves in on time, right? So the percentile doesn't matter - it's the straight likelihood of failure that matters.

I think your weighting function on the rolled average is a good idea though - recent performance is presumably a better indicator of likely future performance than old performance.

I also think there should be a separate RR/NMR count for live games, they do seem to be a different category of game for this purpose.

@thorfi

Your point about percentiles is well taken. The issue with the current system, though, is that it has no intrinsic meaning, so people are already just comparing RR against each other to assign value, which is essentially a percentile, just messier.

I personally never liked RR implementation, it made me feel sad about myself

RR in principle is very good. It just needs to be tweaked some.

+1+1+1+1+1

@abgemacht

100% agreed. :-) I think the current system clearly doesn't do the job it should be doing - which is giving some reasonable way to predict if a particular user will "be reliable". If it can be fixed to do that, then it becomes sensible to compare them - whether you then use percentile or straight ranking doesn't matter.

The reason you can't sensibly compare them now is that the current number is a *terrible* predictor of anything.

As I said I'd be in favor of some sort of way to increase GR. I have only 3 CDs in 2014-15, but almost 50 from my earlier years on the site. I did and do primarily play live games, and there's always higher rates of CD in those. Also, I didn't care as much back then, admittedly. My point being that I think a 90% rating is lower than it should be for me, considering the high number of games and GR and such. I don't think CDs from so long ago that in the meantime I've graduated high school and college should matter much.

Having played around with the numbers I think something like this would work. A low value for alpha when a player doesn't CD and a higher value of alpha when they do. This would allow for a slow but steady increase in RR and a sharp plummet when they CD. Here, I used 0.2 and 0.5 as my alphas:

1 0 0.00
2 100 20.00
3 100 36.00
4 100 48.80
5 100 59.04
6 100 67.23
7 100 73.79
8 100 79.03
9 100 83.22
10 100 86.58
11 100 89.26
12 100 91.41
13 100 93.13
14 100 94.50
15 100 95.60
16 100 96.48
17 100 97.19
18 100 97.75
19 100 98.20
20 100 98.56
21 0 49.28
22 100 59.42
23 100 67.54
24 100 74.03
25 100 79.22
26 100 83.38
27 100 86.70
28 100 89.36
29 100 91.49
30 100 93.19
31 100 94.55

I think I like 0.15 and 0.6 a bit better:

1 0 0.00
2 100 15.00
3 100 27.75
4 100 38.59
5 100 47.80
6 100 55.63
7 100 62.29
8 100 67.94
9 100 72.75
10 100 76.84
11 100 80.31
12 100 83.27
13 100 85.78
14 100 87.91
15 100 89.72
16 100 91.26
17 100 92.57
18 100 93.69
19 100 94.64
20 100 95.44
21 0 38.18
22 100 47.45
23 100 55.33
24 100 62.03
25 100 67.73
26 100 72.57
27 100 76.68
28 100 80.18
29 100 83.15
30 100 85.68
31 100 87.83

CDs can ruin games for your opponents. Perhaps one way of rescinding CDs if if you start taking over CDs to help enhance the games of others. Say 5 games taken over = 1 of your own CDs being removed?

What is cd? I thought all music was stored in usb now adays....

@chaqa Yeah - which is why I agree with ATC that some kind of age/count limit and abge that some kind of decay function is necessary. Stuff that happened years ago isn't that relevant to predict anything, but if someone NMRed yesterday, it's probably a lot more likely they're going to do it today and tomorrow.

@abge Shouldn't we be doing "turns processed" rather than "games completed"? I'm not sure what the current code is counting - I would have assumed the former. The math is the same, but just counting something different. :-)

Well, I could understand the sentiments "years ago I didn't know going into civil disorder is bad" and "people can change". However, I don't think it would be a good idea to exclude past CDs entirely as if they have never happened, for several principal reasons.

To start with, it would be a devaluation of the "100\% reliability". To me, "100\%" means just that, "100\%", i.e. perfect reliability. If people who have had (many) CDs in the past can achieve 100\% reliability, then what worth has that 100\%?
Besides, if you exclude CDs which happened in the past, then you would indirectly punish those who did always submit orders long ago. It would be like telling them "great you always submitted moves years ago, but you didn't really had to; you could have gone into CD, completely unpunished".
Furthermore, if you allow to ignore past results such as CDs, then what next? People could start arguing other past results should be ignored as well. For instance: "when I first joined this site, I didn't know how to play properly, therefore it's unfair I lost points" or: "my first games take down my GhostRating, please ignore those".
Ignoring past CDs not only affects those who did CD long ago, but everyone, including those who have never had any CD.

@thorfi

Probably, I'm not arguing that exact algorithm be used. I'm just showing how an EMA could be used fro this situation. If it's actually of interest we could go into more detail.

@Nescio

I'm sorry but that slippery slope argument isn't particularly impressive to me. We are trying to solve a particular issue of RR not presenting useful information. We are not discussing CD amnesty.

Your point about 100% is good though. I'd say if someone ever CD'd their max RR could be 99%. That would allow someone to fix their RR but still reward people who have never CD'd.

I'd say either get all-time reliability which isn't influenced by time as well as time-weighted reliability and simply have those two seperate, or combine the two, with all-time reliability always influencing your time-weighted reliability by, say, 5%. This is much less arbitrary than introducing a max 99% RR and I think it does a good job at representing the truth of how reliable you are. The past matters much less than the present, but it still exists.

If one did all time reliability with a decay function, and rounded down, that would work. Might be a bit database intensive though.

@steephie

That's a very convoluted (and bad) way of doing pretty much the same thing I proposed.

@thorfi

I'm not convinced time should have anything to do with RR, even ignoring the added coding challenges that would bring.

If I CD in a game, but then play 50 games without a CD, does it matter if those 50 games took place in one week or one year? Maybe, but personally I don't think so.

@thorfi

Sorry, I misread your last post. You can ignore my response. The point still stands, it's just not directed at you.

An all time reliability with a decay function is exactly what I've proposed. The reality is that with any decay function, older games will stop mattering at somepoint. The only question is where abouts the threshold will be.

Huh. I like steephie's idea, actually. Have "RR500" or something being the RR of the last 500 turns you submitted which counts for 90% or your "overall RR" then have the current "all time RR" as 10% of your RR. So the all time does count, but for nowhere near as much as the recent stuff. You could even go with RR100 as 50%, RR500 as 40%, RRall as 10%.

It's kind of a decay function, but maybe a bit simpler to understand.

If you do the math, I think you'll find that won't work out the way you want it to.

I suspect the real answer here will be "whatever ATC wants to program", by the way. :-) Unless I get uber keen and decide to do some coding, which depends on how my work is going. Run up to release of a 2.5 year project - so lots of bug fixing, but I suspect we're close to a quiet period just before release. If that happens I might actually do some coding.

@abge Yeah, it does something, but it's not as nice. :-)

The assumption that the current system makes is "past behaviour is an indicator of future performance". A better assumption is "recent past behaviour is an indicator of future performance" - and that's what's made in most of the new proposals.

If black marks stay forever, there's no incentive for users to reform. If they eventually decay, then it's worth reforming. This is another reason I think this decay is really important.

I agree that "never CDed or NMRed" is a nice signal to have - we should definitely at least acknowledge that on the profile somehow.

I'm against having more than one RR indicator - there's no reason to make it any more complex than it already is (and we don't want our game signup page to look like an airplane dashboard).

I 100% agree that we need a more understandable solution than the one we currently have. I think it needs to be easy to understand what "80% RR" means, because users who want to gate games by reliability need to be able to know what they're asking for when they create games with a RR cap. The current system doesn't have that property either.