RR Feedback & Review - have changes achieved their aims?

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:points: :-D :eyeroll: :neutral: :nmr: :razz: :raging: :-) ;) :( :sick: :o :? 8-) :x :shock: :lol: :cry: :evil: :?: :smirk: :!:
View more smilies

BBCode is ON
[img] is ON
[flash] is OFF
[url] is OFF
Smilies are ON

Topic review
   

If you wish to attach one or more files enter the details below.

Expand view Topic review: RR Feedback & Review - have changes achieved their aims?

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Tue Sep 28, 2021 6:43 am

I 100% Agree it should be based on data not "time spent" or convoluted thought processes.

What about just reverting the RR itself to be reflective of actual propensity to CD (the old RR%), but institute a separated penalty for consistent CDs?

The thing is with the old system if you set a minimum RR it had some justification in expected odds of completing the game - this new RR doesn't and I think that is what causes the disconnect between "expected" behaviour and actual behaviour because most people don't look at it that way no matter the intent of the system.

Anyway thanks for thinking about it and potential solutions.

Re: RR Feedback & Review - have changes achieved their aims?

by bo_sox48 » Mon Sep 27, 2021 6:45 pm

I don't think we should just say "the admins and developers have put a lot of thought into this; thus, it's okay" in any instance. I'm happy to consider improvements and jmo is as well. But we have to play the unbiased observer card and qualify every idea, not to mention that we are in the truest sense volunteers and can't just knock something big out at a moment's notice.

I don't know how to change user behavior, but I believe the 80% RR requirement is a default value for users with 100% RR. Assuming that's the case, whatever algorithm we use to populate that field should be changed in my opinion.

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Mon Sep 27, 2021 3:03 am

That's cold Johnny.

Also Bo's premise that the two systems cannot be compared goes against actual behaviour - the preponderance of games requiring 80% + RR has not diminished at all so the entire argument falls over.

The stated aim and premise on which this is based is not reflected in new game creation which is kinda the whole point.

Re: RR Feedback & Review - have changes achieved their aims?

by Johnny Big Horse » Sat Sep 25, 2021 2:44 pm

I am good with Bosox's response. It sounds like they really put a lot of thought into this. I guess, sometimes, when life gets in the way, we have to deal with it.

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Sat Sep 25, 2021 1:01 am

Thanks for replying Bo, and taking the question the way it was intended.

A heavier time-based weighting might be a possible resolution, but the problem isn't a one off effect, it is a cumulative one.

Looking back I probably should've appealed for clearance or forgiveness of the CDs at the time, but I dare say like many people, I personally, have a hard time asking for things like that. My standard response to myself, is that I knew the rules and shouldn't make excuses.

That is the case even if I actually do have a really good excuse - when in those situations it is a pretty natural response to take it on yourself and not seek assistance.

Its only when they all added up forcing it down below the 60% mark that you have an "oh shit" moment, and realise you can't join decent games anymore. Then you re-check and have another "oh shit" moment when you realise hey guess what - this will stick with me for another 6 months.

I also take your point about automation. The problem is that automated systems don't and can't discriminate. Certainly they can't differentiate between the player types you mentioned, without ridiculously complicating the code.

The other part of the question was also about the impact on CDs since the changes and whether you had any data on it, such as total games v total CDs in a given time period. Ultimately, I am sure these changes weren't about punitive measures, but making a better experience for the community, and I was wondering if anyone had gone back to measure it yet.

Re: RR Feedback & Review - have changes achieved their aims?

by bo_sox48 » Fri Sep 24, 2021 10:51 pm

Wusti wrote:
Tue Sep 21, 2021 6:39 am
I confess to being a bit flabbergasted - this was posted in the correct section of the forum and not a single mod or dev has deigned to respond in any fashion at all.

Is there a problem with raising the whole RR issue again?
The moderators are extremely busy and shorthanded. I cleared a months-long backlog the other day and despite that the active moderators still have a lot to keep up with. We don't have the ability to do any major development right now for reasons that are going to be good for the site in the long run. The admins all have busy lives and I know personally the webDip forum isn't my first stop every day anymore believe it or not.

Can you clarify what your question is aside from changing the amount of time it takes to forgive missed turns? Six months is very short. That means that a player who is habitually unreliable and is barred from joining new games by our system for a long period of time due to their unreliability can come back to a nearly or even completely clean slate with perfect reliability again and ruin more high quality games. We aren't going to open that door. It's a shame that it has to be done that way since it can affect experienced players who came up on a rough patch and deserve the benefit of the doubt but there are other experienced players, some that have participated in hundreds or even over a thousand games on this site, that think that quitting when they're behind or joining live games when they only have an hour to kill is okay. The rating is tiered so that the most recent missed turns are weighted most heavily to try and mitigate the issue you face, but an automated system can't ignore those games because it can't differentiate between one-off mistakes and habitual unreliability on that small of a scale.

The last time I responded to a thread of this tune I noted that placing the old system versus the new system is not a cogent comparison. As you pointed out yourself a 96% reliability rating under the old system and a 55% reliability rating under the new system aren't necessarily that dissimilar. The stigma that surrounded a reliability rating of, for example, 70% under the old system is ridiculous under the new one. A player with a number of games under their belt and a 70% reliability rating is generally very reliable. A player with a 55% reliability rating isn't untrustworthy either. They're two completely different systems that prioritize, penalize, and allow one's rating to recover differently - that was the point of the overhaul in the first place.

Re: RR Feedback & Review - have changes achieved their aims?

by Claesar » Wed Sep 22, 2021 6:15 am

Wusti wrote:
Tue Sep 21, 2021 6:40 am
Claesar - I happen to agree with you on that score and I'm not sure anecdotal evidence is worth much in this argument.

Is the success or otherwise of the RR changes something you guys have looked into at all, or are prepared to engage on? If not, why not?
Don't know; I'm not an active mod. The fact that I still have the tag makes me think the team is very busy.

Re: RR Feedback & Review - have changes achieved their aims?

by Chaqa » Tue Sep 21, 2021 7:28 pm

Wusti wrote:
Tue Sep 21, 2021 6:39 am
I confess to being a bit flabbergasted - this was posted in the correct section of the forum and not a single mod or dev has deigned to respond in any fashion at all.

Is there a problem with raising the whole RR issue again?
The mod team was very busy earlier in the year, unclear if that's still the case.

I think shortening the time CD stay on the record to 6 months would go a long way, even if the penalties were slightly increased (say 6% instead of 5% per) or something.

Re: RR Feedback & Review - have changes achieved their aims?

by Johnny Big Horse » Tue Sep 21, 2021 7:13 pm

At least shorten our punishment period

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Tue Sep 21, 2021 6:40 am

Claesar - I happen to agree with you on that score and I'm not sure anecdotal evidence is worth much in this argument.

Is the success or otherwise of the RR changes something you guys have looked into at all, or are prepared to engage on? If not, why not?

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Tue Sep 21, 2021 6:39 am

I confess to being a bit flabbergasted - this was posted in the correct section of the forum and not a single mod or dev has deigned to respond in any fashion at all.

Is there a problem with raising the whole RR issue again?

Re: RR Feedback & Review - have changes achieved their aims?

by Claesar » Tue Sep 21, 2021 6:34 am

I'm fairly sure the mod team catches and bans almost all new players that re-register under a different username. I don't believe this to be a significant thing.

Though you may be correct that many new players *think* their RR doesn't matter because they could create a new account, until they try and run into a mod.

Re: RR Feedback & Review - have changes achieved their aims?

by Aristocrat » Tue Sep 21, 2021 12:33 am

Most of the CDs on this site (if my experience is any guide) come from new or like-new accounts with little history. Given the constant churn of accounts, I am guessing that some significant number of these players simply CD when their game(s) aren't going the way they want and re-register under a different username; the others will quit the site entirely. Regardless, a hefty RR penalty does nothing to deter these types of CDs. In fact, if the penalty is too high it might actually encourage CDs to the extent that it encourages players to relentlessly churn accounts (and although that is against the rules, we all know this happens) rather than simply stick with one account.

I think one year is a little too punitive, although clearly it is having some effect on some individuals if it is resulting in the complaints in this thread. The only problem is that, rather than incentivizing reliability, I suspect it disincentivizes rule-following as stated above.

The fact that forums and other types of community across all diplomacy sites (and the web more broadly) are largely in decline means that people have little incentive to stick around and build up a reputation or brand compared to 10 years ago, which also complicates things. I am guessing if there was some affirmative reason for new players to want to keep only one account they would take reliability more seriously, but without that reason there is near zero cost to just CDing and moving along.

Re: RR Feedback & Review - have changes achieved their aims?

by Johnny Big Horse » Mon Sep 20, 2021 5:40 pm

on the other hand, I like playing with newbies.

Re: RR Feedback & Review - have changes achieved their aims?

by Johnny Big Horse » Mon Sep 20, 2021 5:39 pm

Yep, I agree with Wusti and Chaquito. Sometimes when you are flying to foreign countries, and the experience takes over 24 hours...sometimes,... and you are running through airports between flights...no time for diplomacy. And then you are screwed. My rating dropped too. sometimes life gets demanding.

Re: RR Feedback & Review - have changes achieved their aims?

by Chaqa » Mon Sep 20, 2021 1:59 pm

Peregrine Falcon wrote:
Fri Sep 17, 2021 9:20 pm
Using excused missed turns has minimal penalty and is recovered quickly. What you seem to be complaining about is the length of time you're penalised for Civil Disorders. Yet, those have massive impacts on the game and really should be penalized heavily. Anything less than one year is too short; it's just not enough time to have believably changed in your relation to time management and care for the game.

If anything could change, however, it would be to allow for more than 4 excused missed turns in game creation. People who prefer the safety net and are fine with the delays can join those with more, and those who aren't can avoid it. (That said, it could be worth penalising using excused missed turns more as well, to make using them more of a trade-off.)
Nah, I vehemently disagree that one year isn't long enough. A lot of time civil disorders come because of temporary/short-term life issues. Most of mine come from periods of a few days when work was crazy, or I was sick, or otherwise just not near a computer.

A lot of us have things that wind up taking precedence over Diplomacy sometimes.

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Mon Sep 20, 2021 6:27 am

Hey Mod/Dev team - do you actually have any data regarding behavioural changes from the new RR system?

Re: RR Feedback & Review - have changes achieved their aims?

by Wusti » Sat Sep 18, 2021 12:07 am

Peregrine Falcon wrote:
Fri Sep 17, 2021 9:20 pm
Using excused missed turns has minimal penalty and is recovered quickly. What you seem to be complaining about is the length of time you're penalised for Civil Disorders. Yet, those have massive impacts on the game and really should be penalized heavily. Anything less than one year is too short; it's just not enough time to have believably changed in your relation to time management and care for the game.

If anything could change, however, it would be to allow for more than 4 excused missed turns in game creation. People who prefer the safety net and are fine with the delays can join those with more, and those who aren't can avoid it. (That said, it could be worth penalising using excused missed turns more as well, to make using them more of a trade-off.)
I respectfully disagree, and would lean towards an evidence based model. We have implemented change with no reporting on the impact of that change. I think we really need to see the evidence of exactly what has changed.

I have been playing on this site for 10 years with 480 games and 13 CDs in that time - a 97.3% reliability. It just so happens that half of those have happened within the last 12 months due to extraneous circumstances and a couple of genuine screw-ups.

Inserting more excused turns will just destroy some variants (4 missed turns in chaos or World? holy christ no!) and implies a frequency the site should not encourage.

Frankly, I don't think there is much wrong with the current system of penalties other than the expiry period of the timers, which are debilitating and could encourage poor behaviours (which I listed in my initial post such as secondary accounts to wait out the timer).

Having said that, can the mods/admins please publish the impacts of the new system on behaviour to inform such a discussion.

Re: RR Feedback & Review - have changes achieved their aims?

by Peregrine Falcon » Fri Sep 17, 2021 9:20 pm

Using excused missed turns has minimal penalty and is recovered quickly. What you seem to be complaining about is the length of time you're penalised for Civil Disorders. Yet, those have massive impacts on the game and really should be penalized heavily. Anything less than one year is too short; it's just not enough time to have believably changed in your relation to time management and care for the game.

If anything could change, however, it would be to allow for more than 4 excused missed turns in game creation. People who prefer the safety net and are fine with the delays can join those with more, and those who aren't can avoid it. (That said, it could be worth penalising using excused missed turns more as well, to make using them more of a trade-off.)

Re: RR Feedback & Review - have changes achieved their aims?

by JRoz » Fri Sep 17, 2021 2:29 am

@Jamiet99uk I'm keen to join a game if we can get enough people.

Would be very nice if a mod could reply to us!

Top