Reliability ratings and locking changes
Reliability ratings and locking changes
I've made some changes to reliability ratings; the previous optimizations meant you couldn't really see what was going on with the calcs if they went wrong, but now what's shown on your profile is tied to the actual numbers used in the new calcs.
It seemed like the way it was calculated was built on over time so I tried to simplify the explanation down so it's easier to see where your RR comes from: Let me know if it's not straightforward or if you notice your numbers seem off. It definitely works when doing a global refresh, but whether it'll work as it goes we'll have to wait and see over time.
I also noticed that deadlocks have been coming back, and it was 90+% due to order submission via the new API that the bots and new UI used. It was locking the entire game instead of just the user record so I've reduced that down. Hopefully there won't be anything to notice here, but I'm interested if anyone got any errors during order submission recently.
It seemed like the way it was calculated was built on over time so I tried to simplify the explanation down so it's easier to see where your RR comes from: Let me know if it's not straightforward or if you notice your numbers seem off. It definitely works when doing a global refresh, but whether it'll work as it goes we'll have to wait and see over time.
I also noticed that deadlocks have been coming back, and it was 90+% due to order submission via the new API that the bots and new UI used. It was locking the entire game instead of just the user record so I've reduced that down. Hopefully there won't be anything to notice here, but I'm interested if anyone got any errors during order submission recently.
Re: Reliability ratings and locking changes
I just checked and my reliability rating is 0%. I haven't played a game on webDip in a few years, but that seems a little harsh! Could this be an error with the recent changes?
See my full Profile:
http://www.webdiplomacy.net/profile.php?userID=17421
http://www.webdiplomacy.net/profile.php?userID=17421
Re: Reliability ratings and locking changes
Oops a divide by 0, always forget something.. Thanks that's fixed now:
-
- Posts: 5
- Joined: Wed Sep 28, 2022 9:53 pm
- Contact:
-
- Posts: 383
- Joined: Fri Oct 06, 2017 12:04 am
- Contact:
Re: Reliability ratings and locking changes
In looking at the RR chart, excused misses count against you?
Re: Reliability ratings and locking changes
While we're on the subject of ratings, my ghost rating seems to have gone down in the last two years, despite me not having played a single game. Is there perhaps something wrong with the calculation?
-
- Posts: 5
- Joined: Wed Dec 14, 2022 2:57 pm
- Contact:
Re: Reliability ratings and locking changes
How long does it take for base rate improvements to be reflected in the reliability rating? My base rate has been increasing, and the calculation implies that my reliability rating should be increasing as a result, but it's still stuck at its low point. Is this intentional? Right now the math on my reliability rating page shows this:
"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "
"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "
-
- Gold Donator
- Posts: 577
- Joined: Wed Feb 07, 2018 11:05 am
- Location: Massachusetts, USA
- Contact:
Re: Reliability ratings and locking changes
@ATC -
I had noticed that the GB and Peak GR ratings changed also for some players when the GR calculations were incorporated into the site at the beginning of 2021. I understand that there were some calculation errors in the old ratings that were corrected at that time. I watch the GB GR changes pretty closely and have seen no unusual changes in gunboat ratings since FWIW.
I had noticed that the GB and Peak GR ratings changed also for some players when the GR calculations were incorporated into the site at the beginning of 2021. I understand that there were some calculation errors in the old ratings that were corrected at that time. I watch the GB GR changes pretty closely and have seen no unusual changes in gunboat ratings since FWIW.
Re: Reliability ratings and locking changes
Thanks Doug, but I'm not sure how your reply is related. I'm talking about the stats reported on the site only.
Re: Reliability ratings and locking changes
Hmm, there may well be, I've never looked into how GRs are calculated. Definitely it could be something similar to RRs where they were implemented in a way that was simple and easy to understand but required reading every players entire game history turn by turn for the last year every time a game they were involved in got processed, which contributed to those deadlocking issues that were a big problem for site performance/reliability this time last year.
The new RR calcs are a lot more complex because they put turns into different buckets, and there may still be a bug or two lurking there, but it's O(1) instead of O(N).
I'll take a look asapNavel_Reservist wrote: ↑Fri Dec 30, 2022 5:41 amHow long does it take for base rate improvements to be reflected in the reliability rating? My base rate has been increasing, and the calculation implies that my reliability rating should be increasing as a result, but it's still stuck at its low point. Is this intentional? Right now the math on my reliability rating page shows this:
"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "
Re: Reliability ratings and locking changes
So I did a global RR recalc and you went from 72% to 74%: to Looks like when you last checked the RR should've been 72% (2 missed turns/12 yearly turns = 16.7%), and when I checked before the global recalc the RR should've been 74% (2 missed turns / 14 yearly turns = 14.3%) which it now is.kestasjk wrote: ↑Tue Jan 03, 2023 11:26 amI'll take a look asapNavel_Reservist wrote: ↑Fri Dec 30, 2022 5:41 amHow long does it take for base rate improvements to be reflected in the reliability rating? My base rate has been increasing, and the calculation implies that my reliability rating should be increasing as a result, but it's still stuck at its low point. Is this intentional? Right now the math on my reliability rating page shows this:
"Total:
Reliability Rating: 100% - Base/Total reduction (16.7%) - Non-live penalty (11%) - Live penalty (0%) = Reliability rating (64%) "
Looks like when the yearly turn counter gets incremented it wouldn't trigger a refresh, so it would only update on missed turns. I've fixed that now, thanks for reporting:
https://github.com/kestasjk/webDiplomac ... 6ee7c7f79e
-
- Posts: 206
- Joined: Mon Sep 12, 2022 10:20 pm
- Contact:
Re: Reliability ratings and locking changes
Couldn't you just
Code: Select all
newRR = 99% * oldRR + 1% * current_turn
¶ Keep thy heart with all diligence; for out of it are the issues of life.
-- Proverbs of Solomon, chapter 4, verse 23
-- Proverbs of Solomon, chapter 4, verse 23
Re: Reliability ratings and locking changes
You could do a lot of things.. I wanted to preserve the system jmo put in place as it seemed to be working well. The current system adds some forgiveness for consecutive turns, lets mods forgive turns, differentiates between live games and non-live, etc.
I’m not saying you couldn’t make something like that have similar features.. but you definitely come to learn that any system that tries to sum up any metric like reliability, Diplomacy skill, account-suspiciousness, user-relationship likelihood, etc, it’s always going to be a balance between loads of different trade offs and compromises, and there’ll always be suggestions on how it could be improved in some way or other.
I’m not saying you couldn’t make something like that have similar features.. but you definitely come to learn that any system that tries to sum up any metric like reliability, Diplomacy skill, account-suspiciousness, user-relationship likelihood, etc, it’s always going to be a balance between loads of different trade offs and compromises, and there’ll always be suggestions on how it could be improved in some way or other.
-
- Posts: 5
- Joined: Wed Sep 28, 2022 9:53 pm
- Contact:
Re: Reliability ratings and locking changes
Sorry - how long does it take for non-live penalties to stop contributing to your RR? (I really like the RR system for what it's worth)
Who is online
Users browsing this forum: No registered users