EIDRaS Ratings

Forum rules
This forum is limited to topics relating to the game Diplomacy only. Other posts or topics will be relocated to the correct forum category or deleted. Please be respectful and follow our normal site rules at http://www.webdiplomacy.net/rules.php.

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:points: :-D :eyeroll: :neutral: :nmr: :razz: :raging: :-) ;) :( :sick: :o :? 8-) :x :shock: :lol: :cry: :evil: :?: :smirk: :!:
View more smilies

BBCode is ON
[img] is ON
[flash] is OFF
[url] is OFF
Smilies are ON

Topic review
   

If you wish to attach one or more files enter the details below.

Expand view Topic review: EIDRaS Ratings

Re: EIDRaS Ratings

by Yonni » Wed Jan 24, 2018 1:23 pm

@Bo

7 was just chosen because that's what Judge did. It's in the litany of variables for us to discuss if we adopt the system.

Also, I misspoke in that quote. I adjusted the weighting of games that include provisional players. The reason for this is exactly as you pointed it out. We can get a new strong player (e.g. Village Idiot) that starts with a rating of 1000. That obviously isn't his "true rating" so we discount games played against him for a bit.

The formula that Judge used was:
A = 1 + p/7
;where p is the number of players with provisional ratings (the denominator obviously changes for map types)

I tweaked it to be:
A = 2*p/7

The maximum is the same but I further discounted games against provisional players. But, again, this can all be tweaked as we want.

@Ghug

I suppose you're correct but I'm not sure how to formalize it. In fact, I'm not entirely certain how they decided on the expected score for WTA (other than it giving the curve they like). Open to any ideas you might have.

Perhaps the value of "c" can be adjusted. As a reminder, the formula of expected score, X, is given by:

X = M * [e ^ (R *c)] / sum[e ^ (Ri *c]

Re: EIDRaS Ratings

by bo_sox48 » Wed Jan 24, 2018 5:37 am

Yonni wrote:
Sat Jan 20, 2018 8:46 pm
I made a slight tweak to some of the weightings to give smaller swings for provisional players (i.e. 7 games played or less). I found that it was going a little wild and giving some very new players some very high ratings.
How do you account for this once they play their 8th game? Do those smaller swings get recalculated? Where did 7 games come from as opposed to any other number of games? There is one player at the very high end of the GR list right now that has soloed 8 games, for example, but if you look at his game history none of those 8 games really show what caliber of player he is, so obviously it isn't a magic number.

Does the caliber of those first seven games matter? I can see some Playdippers coming over or something of that sort and playing a number of high quality games, perhaps for a World Cup or something similar. Should those games still not be given the same weight?

Re: EIDRaS Ratings

by bo_sox48 » Wed Jan 24, 2018 5:32 am

Hellenic Riot wrote:
Wed Jan 24, 2018 12:17 am
I really enjoy how Bo is basically the only person to have the exact same rank in both systems. Not even Maths can change Bo being the worst :-D
Ah, the truth finally comes out.
ChippeRock wrote:
Tue Jan 23, 2018 10:29 pm
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.
Both of those games, one of which you listed twice, were pretty terrible. When we talk about a natural solo, that implies that everyone actually plays with some mark of competency, no? A solo in five or six years is very much possible, but even four years seems far fetched without some players literally walking out of their centers to make way for a single unit to move in.

Re: EIDRaS Ratings

by ghug » Wed Jan 24, 2018 4:47 am

Basically, you're never going to get much more than 50% of the pot in PPSC, so expected values have to be lowered. The squares aspect of SoS exacerbates the disparity in results, so expected values have to be spread out accordingly.

Re: EIDRaS Ratings

by Yonni » Wed Jan 24, 2018 12:19 am

ghug wrote:
Tue Jan 23, 2018 11:43 pm
Did you change the expected result calculation for each scoring system? The existing one is tailored to DSS. We expect the best players to do even better in SoS and the worst players to do even worse, and we expect the opposite in PPSC.
The expected score doesn't change with the scoring system but I'm not sure what you mean that it's tailored for DSS. How would you go about changing it for the other systems?

Re: EIDRaS Ratings

by Hellenic Riot » Wed Jan 24, 2018 12:17 am

I really enjoy how Bo is basically the only person to have the exact same rank in both systems. Not even Maths can change Bo being the worst :-D

Re: EIDRaS Ratings

by ChippeRock » Tue Jan 23, 2018 11:55 pm

Yoyoyozo wrote:
Tue Jan 23, 2018 11:32 pm
ChippeRock wrote:
Tue Jan 23, 2018 10:29 pm
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.
I disagree. Players were told that games ending before 1904 would not count towards GR, so I think the same should be upheld for YR. A lot of games that result in CD are drawn in 1903 are done so because of the fact that these games would not could, so the very small number of "legitimate" solos would be heavily outnumbered by the erroneous counting of games that were draw for the explicit reason of it not being counted.
I never said put all games that ended early in this category here. I just said put those specific games in. Also, it's basically impossible to find a game where no one NMRed or CDed in. So I don't think you should exclude games. Perhaps you could make sure that results of a game with a NMR or CD don't matter as much as a game without a NMR or CD but that's about all you should do in my opinion.

You can also find many games with instances of cheating, so I don't think you should remove games that had cheating neither.

Re: EIDRaS Ratings

by ghug » Tue Jan 23, 2018 11:43 pm

Did you change the expected result calculation for each scoring system? The existing one is tailored to DSS. We expect the best players to do even better in SoS and the worst players to do even worse, and we expect the opposite in PPSC.

Re: EIDRaS Ratings

by Yoyoyozo » Tue Jan 23, 2018 11:32 pm

ChippeRock wrote:
Tue Jan 23, 2018 10:29 pm
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.
I disagree. Players were told that games ending before 1904 would not count towards GR, so I think the same should be upheld for YR. A lot of games that result in CD are drawn in 1903 are done so because of the fact that these games would not could, so the very small number of "legitimate" solos would be heavily outnumbered by the erroneous counting of games that were draw for the explicit reason of it not being counted.

Re: EIDRaS Ratings

by ChippeRock » Tue Jan 23, 2018 10:29 pm

Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.

Re: EIDRaS Ratings

by Yonni » Tue Jan 23, 2018 10:07 pm

Here is a side-by-side comparison of Ghost Rating and EIDRaS in a convenient Google Sheet so everyone should be able to access it:
https://docs.google.com/spreadsheets/d/ ... sp=sharing

A few comments:
  • This is for all variant and press types except for 1v1
  • The weightings for each variant are the same as for GR
  • There is a slight discrepancy in the games played for some people and I'm not sure why
  • GR is meant to exclude games that end at 1903 or earlier. However, I believe the code just excludes games that end at turn 3 or earlier (i.e. 1902). I've given EIDRaS the same cutoff for the sake of comparison
  • Some of the biggest risers are newer players (e.g. Kugs). This is a feature of EIDRaS as it allows people to reach their "true rating" quicker. The amount that the system does that can be tweaked later
  • On that note, there are a bunch of variables that can be tweaked in EIDRaS. If people are interested in adopting it as a rating system, we can have discussions about what values people would like.

Re: EIDRaS Ratings

by Yonni » Tue Jan 23, 2018 9:48 pm

After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156

Re: EIDRaS Ratings

by ChippeRock » Tue Jan 23, 2018 9:19 pm

President Eden wrote:
Tue Jan 23, 2018 7:59 pm
I don't think a 1903 solo is possible without Russia's neighbors actively vacating centers for Russia to take. i.e. It won't happen naturally.
Well obviously it's realistically impossible to solo in 3 years with any of the countries, but it is possible to solo in 3 years.

Re: EIDRaS Ratings

by President Eden » Tue Jan 23, 2018 7:59 pm

I don't think a 1903 solo is possible without Russia's neighbors actively vacating centers for Russia to take. i.e. It won't happen naturally.

Re: EIDRaS Ratings

by ghug » Tue Jan 23, 2018 7:56 pm

Not all of your units can take centers every year. It's possible Russia can solo in 1903, but I think it's 1904, and for some it's only 1905.

@Yonni, I'm not sure about that. I know GR just uses the dump.

Re: EIDRaS Ratings

by ChippeRock » Tue Jan 23, 2018 4:59 pm

ghug wrote:
Tue Jan 23, 2018 4:31 pm
It's just before 1904. A solo is theoretically possible in 1904, though practically unachievable.
Isn't it possible for a solo to be achieved in 1903 as well? Let's look at each of the countries path to victory:

Russia's trajectory is something like this:
1901 - 4 SCs to 8 SCs - 4 units to 8 units
1902 - 8 SCs to 16 SCs - 8 units to 12 units
1903 - 16 SCs to 28 SCs - 12 units to 16 units

And for the other countries (except England) something like this:
1901 - 3 SCs to 6 SCs - 3 units to 6 units
1902 - 6 SCs to 12 SCs - 6 units to 9 units
1903 - 12 SCs to 21 SCs - 9 units to 12 units

And England's trajectory something like this?
1901 - 3 SCs to 5 SCs - 3 units to 5 units
1902 - 5 SCs to 10 SCs - 5 units to 8 units
1903 - 10 SCs to 18 SCs - 8 units to 11 units

Re: EIDRaS Ratings

by Yonni » Tue Jan 23, 2018 4:59 pm

Thanks.

Last issue before I can post a proper comparison:

There are some users (e.g. shaun.thompson) who are listed as banned in the csv dump you posted but are included in the Ghost Rating and are not banned on the site so I think the GR list must be grabbing the banned list from another (more correct) table.

Re: EIDRaS Ratings

by ghug » Tue Jan 23, 2018 4:31 pm

It's just before 1904. A solo is theoretically possible in 1904, though practically unachievable.

Re: EIDRaS Ratings

by Claesar » Tue Jan 23, 2018 3:04 pm

GR excludes games that finish in 1904 or before, I believe.

I've been told this cut-off is not arbitrary; one cannot win a solo victory before 1905.

Re: EIDRaS Ratings

by Yonni » Tue Jan 23, 2018 2:24 pm

Are games under a certain length excluded (i.e. 3 turns or less)?

I'm trying to reconcile the games played and am running into issues.

Top