EIDRaS Ratings

Use this forum to discuss Diplomacy strategy.
Forum rules
This forum is limited to topics relating to the game Diplomacy only. Other posts or topics will be relocated to the correct forum category or deleted. Please be respectful and follow our normal site rules at http://www.webdiplomacy.net/rules.php.
Message
Author
Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

EIDRaS Ratings

#1 Post by Yonni » Sat Jan 20, 2018 8:46 pm

Thanks to ghug's dump in the other thread, I have started calculating the EIDRaS ratings for the site. You can access the Excel sheet with the ratings I've calculated so far as well as the MATLAB code I'm working on by clicking here:

https://drive.google.com/drive/folders/ ... sp=sharing.

What is EIDRaS?

EIDRaS is an Elo like system for Diplomacy developed decades ago by some other people. Unfortunately, it's name is way worse than Ghost Rating. Check out these links for more information:
http://www.stabbeurfou.org/docs/article ... ystem.html
and
http://uk.diplom.org/pouch//Email/Ratin ... cribe.html

I made a slight tweak to some of the weightings to give smaller swings for provisional players (i.e. 7 games played or less). I found that it was going a little wild and giving some very new players some very high ratings.

How does it differ from Ghost Rating?

Tons of respect to The Ghost Maker for creating and maintaining a ratings system for us but it had several shortcomings. The part that bothered me most was that losses against expert opponents and beginners are rated equally. That is not the case for EIDRaS which calculates your expected score based on your opponents ratings. There are other differences too of course but that's the big one for me.

Some things I'm still working on:
  • Dealing with non-WTA games
  • Filtering inactive players
  • Filtering banned players
  • Adjust weighting for variants
  • Make the spreadsheet prettier
What am I doing wrong? What would you like added?

Is there anything you guys want to see added?
VI, should I keep tweaking things until you overtake The Wiz and Nikola?
And a last minute warning to coders: please don't have aneurysm wading through my kiddie script.

Aereaux
Posts: 50
Joined: Sun Dec 31, 2017 12:28 am
Karma: 4
Contact:

Re: EIDRaS Ratings

#2 Post by Aereaux » Sat Jan 20, 2018 8:54 pm

Will this take into account users that have taken over a power after it CDed?

Yoyoyozo
Posts: 190
Joined: Thu Mar 23, 2017 1:57 am
Location: Chicago, Illinois, USA
Karma: 81
Contact:

Re: EIDRaS Ratings

#3 Post by Yoyoyozo » Sat Jan 20, 2018 8:55 pm

Oh neat! I think I’ll just call this the Yonni Ratings or YR for short.
Does it filter unranked games as well? If not GG my rank lol

Durga
Posts: 9486
Joined: Fri Sep 29, 2017 6:01 pm
Location: Canada
Karma: 5120
Contact:

Re: EIDRaS Ratings

#4 Post by Durga » Sat Jan 20, 2018 8:59 pm

Did you weigh gunboat and FP equally?

Durga
Posts: 9486
Joined: Fri Sep 29, 2017 6:01 pm
Location: Canada
Karma: 5120
Contact:

Re: EIDRaS Ratings

#5 Post by Durga » Sat Jan 20, 2018 9:01 pm

I find this list to be incredibly inaccurate. You've got some good players reeeallyyy far down

Aereaux
Posts: 50
Joined: Sun Dec 31, 2017 12:28 am
Karma: 4
Contact:

Re: EIDRaS Ratings

#6 Post by Aereaux » Sat Jan 20, 2018 9:12 pm

Also, you should make them avaiable as a csv or something. I don't have anything to read xlsx files.

Durga
Posts: 9486
Joined: Fri Sep 29, 2017 6:01 pm
Location: Canada
Karma: 5120
Contact:

Re: EIDRaS Ratings

#7 Post by Durga » Sat Jan 20, 2018 9:14 pm

But uh, thanks for the work, this is interesting and could be better than GR when the stuff is fixed :D

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: EIDRaS Ratings

#8 Post by Yonni » Sat Jan 20, 2018 9:36 pm

@Aereaux
Will this take into account users that have taken over a power after it CDed?
Unfortunately this is restricted to the same data that is scraped for GR. I believe that means I only have information on who finishes the games, not who starts them.
Also, you should make them avaiable as a csv or something. I don't have anything to read xlsx files.
The .dat files have all the csv that I just copied and pasted into the xlsx document. I'm not sure but you might also be able to open xlsx using google sheets. Let me know if you have any issues accessing them.

@Durgs
Did you weigh gunboat and FP equally?
Nope, I followed the weighting they used at Judge.
FP = 1, PP = 0.8, GB = 0.5, Live = 0.3
I find this list to be incredibly inaccurate. You've got some good players reeeallyyy far down
It's possible that I've fucked things up (it's also possible they've fucked up some of their games). Could you point out a case or two so I can follow through the code and see what happened?

@yoyo
Does it filter unranked games as well?
Sure does. Unranked games excluded from the data.

User avatar
ghug
Bronze Donator
Bronze Donator
Posts: 17580
Joined: Mon Mar 20, 2017 3:51 pm
Location: Seattle
Karma: 11399
Contact:

Re: EIDRaS Ratings

#9 Post by ghug » Sat Jan 20, 2018 9:42 pm

Are non-DSS games simply excluded right now? That could explain some of the weirdness.

I might put all the GR stuff I have on Github in a bit so people can play with it. Can I throw your script in as well?

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: EIDRaS Ratings

#10 Post by Yonni » Sat Jan 20, 2018 9:44 pm

Yeah, all non-WTA is simply excluded at the moment. I'll get to it later this week. Feel free to put the code wherever you like.

Octavious
Posts: 3844
Joined: Fri Sep 29, 2017 4:16 pm
Location: The Five Valleys, Gloucestershire
Karma: 2605
Contact:

Re: EIDRaS Ratings

#11 Post by Octavious » Sat Jan 20, 2018 9:46 pm

Interesting stuff. I think you're on to a hiding for nothing trying to implement a new ratings system, but well done for giving it a go.

So, for my understanding, this is currently a WTA ratings with equal weight given to all WTA variants? That's diplomacy, gunboat, and all the map options?

Very hard to give an opinion on it until the filters for variants and inactive players have been put in place, but nice to see some of the old names again.

It does need a better name :)

Octavious
Posts: 3844
Joined: Fri Sep 29, 2017 4:16 pm
Location: The Five Valleys, Gloucestershire
Karma: 2605
Contact:

Re: EIDRaS Ratings

#12 Post by Octavious » Sat Jan 20, 2018 9:48 pm

Hang on. Why on earth do judge give live games such a low weighting?!?

Aereaux
Posts: 50
Joined: Sun Dec 31, 2017 12:28 am
Karma: 4
Contact:

Re: EIDRaS Ratings

#13 Post by Aereaux » Sat Jan 20, 2018 9:57 pm

Yonni wrote:
Sat Jan 20, 2018 9:36 pm
The .dat files have all the csv that I just copied and pasted into the xlsx document. I'm not sure but you might also be able to open xlsx using google sheets. Let me know if you have any issues accessing them.
Cool, thanks for putting this together.

It would be nice if the GR data dump included bets so that we could exclude players that took over a country or something.

User avatar
ghug
Bronze Donator
Bronze Donator
Posts: 17580
Joined: Mon Mar 20, 2017 3:51 pm
Location: Seattle
Karma: 11399
Contact:

Re: EIDRaS Ratings

#14 Post by ghug » Sat Jan 20, 2018 10:02 pm

Octavious wrote:
Sat Jan 20, 2018 9:46 pm
So, for my understanding, this is currently a WTA ratings with equal weight given to all WTA variants? That's diplomacy, gunboat, and all the map options?
I think he's just using the legacy nomenclature for DSS when he says WTA here. It doesn't appear that SoS is included. And I'm guessing maps are all weighted equally, but press/live are weighted as the judge does.


I also curious why they'd weight live so low.

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: EIDRaS Ratings

#15 Post by Yonni » Sat Jan 20, 2018 10:03 pm

Octavious wrote:
Sat Jan 20, 2018 9:48 pm
Hang on. Why on earth do judge give live games such a low weighting?!?
I guess because people can play so many of them? I'm happy to play around with the ratings which is why I posted this before I was completely done. I just put some in to start.

edit: Yup, using legacy nomeclature because I don't know the new ones (and the data dump uses the old ones)

Durga
Posts: 9486
Joined: Fri Sep 29, 2017 6:01 pm
Location: Canada
Karma: 5120
Contact:

Re: EIDRaS Ratings

#16 Post by Durga » Sat Jan 20, 2018 10:05 pm

Ghug went from like 9 to 330 kek

Octavious
Posts: 3844
Joined: Fri Sep 29, 2017 4:16 pm
Location: The Five Valleys, Gloucestershire
Karma: 2605
Contact:

Re: EIDRaS Ratings

#17 Post by Octavious » Sat Jan 20, 2018 10:13 pm

Durga wrote:
Sat Jan 20, 2018 10:05 pm
Ghug went from like 9 to 330 kek
Yeah, but just because it rates ghug properly doesn't mean it'll do the same for everyone else ;)

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: EIDRaS Ratings

#18 Post by Yonni » Sat Jan 20, 2018 10:18 pm

What I'd really like to add is for the code to track people's ratings over time but it would make the code very cumbersome (because Matlab and me) and I may run into memory issues. That could answer some of these questions.

Octavious
Posts: 3844
Joined: Fri Sep 29, 2017 4:16 pm
Location: The Five Valleys, Gloucestershire
Karma: 2605
Contact:

Re: EIDRaS Ratings

#19 Post by Octavious » Sat Jan 20, 2018 10:21 pm

I think the only way we'll have a chance of getting a ratings system that will be supported is if there's some consensus over weightings from our members. I can't see the Judge weightings having a great deal of support here. It certainly seems a little odd to me.

I look forward to seeing what this Yonder Rating looks like when it's been tweaked to Yonni's satisfaction :)

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: EIDRaS Ratings

#20 Post by Yonni » Sat Jan 20, 2018 10:26 pm

Yup, I look forward to bitter debate over how we should rank no-press ancient med maps against live full press world map games.

Also, I wasn't sure how to deal with rulebook press. I gave it the same rating as full press. For the sake of rankings, I imagine I can just combine rulebook and full press?

Post Reply

Who is online

Users browsing this forum: No registered users and 58 guests