Page 4 of 5

Re: EIDRaS Ratings

Posted: Mon Jan 22, 2018 4:24 am
by Yonni
Cool, I'll tell the script to ignore them for now. You might have missed my edit but I also need the weighting for Known World. I assume 0.25?

Re: EIDRaS Ratings

Posted: Mon Jan 22, 2018 4:17 pm
by ghug
I didn't, I just didn't know yet. HR says it's the same as World. I'll try to get git updated soon.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 2:24 pm
by Yonni
Are games under a certain length excluded (i.e. 3 turns or less)?

I'm trying to reconcile the games played and am running into issues.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 3:04 pm
by Claesar
GR excludes games that finish in 1904 or before, I believe.

I've been told this cut-off is not arbitrary; one cannot win a solo victory before 1905.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 4:31 pm
by ghug
It's just before 1904. A solo is theoretically possible in 1904, though practically unachievable.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 4:59 pm
by Yonni
Thanks.

Last issue before I can post a proper comparison:

There are some users (e.g. shaun.thompson) who are listed as banned in the csv dump you posted but are included in the Ghost Rating and are not banned on the site so I think the GR list must be grabbing the banned list from another (more correct) table.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 4:59 pm
by ChippeRock
ghug wrote:
Tue Jan 23, 2018 4:31 pm
It's just before 1904. A solo is theoretically possible in 1904, though practically unachievable.
Isn't it possible for a solo to be achieved in 1903 as well? Let's look at each of the countries path to victory:

Russia's trajectory is something like this:
1901 - 4 SCs to 8 SCs - 4 units to 8 units
1902 - 8 SCs to 16 SCs - 8 units to 12 units
1903 - 16 SCs to 28 SCs - 12 units to 16 units

And for the other countries (except England) something like this:
1901 - 3 SCs to 6 SCs - 3 units to 6 units
1902 - 6 SCs to 12 SCs - 6 units to 9 units
1903 - 12 SCs to 21 SCs - 9 units to 12 units

And England's trajectory something like this?
1901 - 3 SCs to 5 SCs - 3 units to 5 units
1902 - 5 SCs to 10 SCs - 5 units to 8 units
1903 - 10 SCs to 18 SCs - 8 units to 11 units

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 7:56 pm
by ghug
Not all of your units can take centers every year. It's possible Russia can solo in 1903, but I think it's 1904, and for some it's only 1905.

@Yonni, I'm not sure about that. I know GR just uses the dump.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 7:59 pm
by President Eden
I don't think a 1903 solo is possible without Russia's neighbors actively vacating centers for Russia to take. i.e. It won't happen naturally.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 9:19 pm
by ChippeRock
President Eden wrote:
Tue Jan 23, 2018 7:59 pm
I don't think a 1903 solo is possible without Russia's neighbors actively vacating centers for Russia to take. i.e. It won't happen naturally.
Well obviously it's realistically impossible to solo in 3 years with any of the countries, but it is possible to solo in 3 years.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 9:48 pm
by Yonni
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 10:07 pm
by Yonni
Here is a side-by-side comparison of Ghost Rating and EIDRaS in a convenient Google Sheet so everyone should be able to access it:
https://docs.google.com/spreadsheets/d/ ... sp=sharing

A few comments:
  • This is for all variant and press types except for 1v1
  • The weightings for each variant are the same as for GR
  • There is a slight discrepancy in the games played for some people and I'm not sure why
  • GR is meant to exclude games that end at 1903 or earlier. However, I believe the code just excludes games that end at turn 3 or earlier (i.e. 1902). I've given EIDRaS the same cutoff for the sake of comparison
  • Some of the biggest risers are newer players (e.g. Kugs). This is a feature of EIDRaS as it allows people to reach their "true rating" quicker. The amount that the system does that can be tweaked later
  • On that note, there are a bunch of variables that can be tweaked in EIDRaS. If people are interested in adopting it as a rating system, we can have discussions about what values people would like.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 10:29 pm
by ChippeRock
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 11:32 pm
by Yoyoyozo
ChippeRock wrote:
Tue Jan 23, 2018 10:29 pm
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.
I disagree. Players were told that games ending before 1904 would not count towards GR, so I think the same should be upheld for YR. A lot of games that result in CD are drawn in 1903 are done so because of the fact that these games would not could, so the very small number of "legitimate" solos would be heavily outnumbered by the erroneous counting of games that were draw for the explicit reason of it not being counted.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 11:43 pm
by ghug
Did you change the expected result calculation for each scoring system? The existing one is tailored to DSS. We expect the best players to do even better in SoS and the worst players to do even worse, and we expect the opposite in PPSC.

Re: EIDRaS Ratings

Posted: Tue Jan 23, 2018 11:55 pm
by ChippeRock
Yoyoyozo wrote:
Tue Jan 23, 2018 11:32 pm
ChippeRock wrote:
Tue Jan 23, 2018 10:29 pm
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.
I disagree. Players were told that games ending before 1904 would not count towards GR, so I think the same should be upheld for YR. A lot of games that result in CD are drawn in 1903 are done so because of the fact that these games would not could, so the very small number of "legitimate" solos would be heavily outnumbered by the erroneous counting of games that were draw for the explicit reason of it not being counted.
I never said put all games that ended early in this category here. I just said put those specific games in. Also, it's basically impossible to find a game where no one NMRed or CDed in. So I don't think you should exclude games. Perhaps you could make sure that results of a game with a NMR or CD don't matter as much as a game without a NMR or CD but that's about all you should do in my opinion.

You can also find many games with instances of cheating, so I don't think you should remove games that had cheating neither.

Re: EIDRaS Ratings

Posted: Wed Jan 24, 2018 12:17 am
by Hellenic Riot
I really enjoy how Bo is basically the only person to have the exact same rank in both systems. Not even Maths can change Bo being the worst :-D

Re: EIDRaS Ratings

Posted: Wed Jan 24, 2018 12:19 am
by Yonni
ghug wrote:
Tue Jan 23, 2018 11:43 pm
Did you change the expected result calculation for each scoring system? The existing one is tailored to DSS. We expect the best players to do even better in SoS and the worst players to do even worse, and we expect the opposite in PPSC.
The expected score doesn't change with the scoring system but I'm not sure what you mean that it's tailored for DSS. How would you go about changing it for the other systems?

Re: EIDRaS Ratings

Posted: Wed Jan 24, 2018 4:47 am
by ghug
Basically, you're never going to get much more than 50% of the pot in PPSC, so expected values have to be lowered. The squares aspect of SoS exacerbates the disparity in results, so expected values have to be spread out accordingly.

Re: EIDRaS Ratings

Posted: Wed Jan 24, 2018 5:32 am
by bo_sox48
Hellenic Riot wrote:
Wed Jan 24, 2018 12:17 am
I really enjoy how Bo is basically the only person to have the exact same rank in both systems. Not even Maths can change Bo being the worst :-D
Ah, the truth finally comes out.
ChippeRock wrote:
Tue Jan 23, 2018 10:29 pm
Yonni wrote:
Tue Jan 23, 2018 9:48 pm
After a quick and entirely not rigorous look through the database, the quickest solo I could find that looked somewhat legit was Russia in 1907:

http://www.webdiplomacy.net/board.php?gameID=11156
Taking a look at all of the games that were won in 3 years I see quite a bit of blatantly obvious cheating. However I was able to find the following games which were won in 3 years and were legitimate solos:
http://webdiplomacy.net/board.php?gameID=158353
http://webdiplomacy.net/board.php?gameID=89551
http://webdiplomacy.net/board.php?gameID=158353

Based on this I feel like these games should at least be included in the user's GR; or any other rating system your using.
Both of those games, one of which you listed twice, were pretty terrible. When we talk about a natural solo, that implies that everyone actually plays with some mark of competency, no? A solo in five or six years is very much possible, but even four years seems far fetched without some players literally walking out of their centers to make way for a single unit to move in.