I think a matchmaker is more important there. Like, it would be awesome to have a feature where you're like "I want to play a 36-72 hour FP Classic game with players around my level". And it just did the rest for you.

Having hard gates allows people to game the system a bit - "I want to play with players who are equal to or strictly better than me" means that they'd be in a game where it would be hard for them to lose much rating.

"I never want to play with anyone rated lower than me, because I'm worried about losing my rating" is another situation I am keen to avoid.

It seems like VI said about your new system, you may find that high ranked players WONT play with people a lot lower then them, because of the possible fluctuations in rank.

Ninja'd

Integrated rankings is a great thing. Anything that moves us closer to a post-points system. Points have never made much sense to me. This isn't poker. Why are we putting in an ante?

For that reason I'm personally in favor of a ratings gate as well. Something like when I set up a game I can restrict it to players whose rating is within x% of my own, plus or minus.

Easy to say I'm looking for a top 50 GR game

The old system had a huge disincentive for playing with lower ranked people, too. You may put in the same yourself, but the result you needed to maintain your GR was high.

So I like that I've gotten to a point where my GR is higher, yes. But I like to play with new people so I am willing to play with just about anyone, reguardless of their rank.

First of all I think you guys are doing a great job and I cheer any implementation and integration of any serious scoring system into the site. If it be True skill, another elo variant or elo itself. Diplomacy is a great game and this site makes it possible for me to enjoy the game and play against some top competition from around the world - and even get ranked!

I haven't read the thread, just ATC's post, so I'm sorry if some of my points of discussion have already passed.. A couple of things I'd like to add:

1."In Elo, you're supposed to get bigger ratings boost for beating players who are better than you - not for beating players who are equally rated. It's always impressive to solo over 7 players who are as good as you are - and I don't think it gets more impressive as those players get better."

I do not agree with this statement. I think solo-ing in a top game is much, much more difficult for a top player, than soloing is for a low/medium skilled player in a low/medium skilled game. This has to do with the fact that the game is designed such that actually you should not win. Most games are designed such that the most likely outcome is a win, diplomacy is designed (with all its stalemate lines and to model reality I guess) to not be won if played well by all players. Take the diplomacy 2012 world cup for example. It has to do with the reversibility of the game. Diplomacy positions can always be pushed back to equilibrium if played well and the limit (t-> large) of a well played game should be an immovable draw. Chess for example knows draws as well, but the game is designed to get skewed and end (as much as possible for a logical game) as you can't rebuild pieces. Summarizing, the better the players are, the less likely a win is. Hence I don't think it's strange you get more points for a solo in a high quality game - or getting in a draw even - if you are a top player then a lesser player in a easier game overall. I actually like this feat.

2. "If the top player on the site gets beaten by a bunch of low ranked players, they should take a big rating hit. GR treats that loss as equivalent to being beaten by other top players."

I also don't think this is a problem. Because diplomacy is a seven for all king of the hill that is actually designed to end in a draw, I think top players should not take a huge hit for loosing from a bunch of beginners. Beginners don't recognize all the tactical implications of their actions, hence a game with six beginners and one top player is very likely to end in a solo. If the top player wins he won't win much, as the 6% of the beginners isn't worth much. If a beginner wins, he relatively gets a big pot, as there is 6% of the top player in it. The top player in that game has probably lost because of some beginners making a series of mistakes, should the top player then take a big hit? I don't think so.

I agree with you ATC that the mechanisms that you recognize as faulty are strange if we were dealing with a regular game, say magic or poker or something. But (albeit by accident or genius ghost design) these faults are actually very suitable for this game and this website I feel.

From my own experience: My two stints of diplomacy on this site have been a steady climb to a relatively higher GR. In my earlier games I have solo-ed, I still only solo in games that are of medium skill. The higher my GR got the more games I played against good players, in games with really top ranked players (top 50 or something) it is so incredibly hard to solo, that should be rewarded more. If there is a solo in such a game it usually because there is one very low GR player involved. Most losses in my games against less skilled competition (now and in the past) are usually due to some pretty foolish decision make-ing by the lowest ranked player or lack of dedication (NMR). These wins should not be much more harmful (they are already relatively scaled!) for my GR as I climb the ladder as they are increasingly less my own fault as I get better.

I think if you take out these two 'faulty' mechanisms we will see that top players really won't play against beginners or medium players any more. This is already a very small problem, but I think you'll make it huge if you organize a big GR hit for top players in lesser games. Climbing the ladder I have experienced first hand how much better players get, I find GR remarkably accurate (if GB and variants are taken away or separated). I'm quite sure I won't solo against the current top 6.

I don't know if my contribution is useful, I tried to summarize why I think that GR works so remarkably well (even though it shouldn't theoretically).

I'm just heading out the door, so don't have time for a full post - but:

"these faults are actually very suitable for this game"

I disagree, and so does the data.

If GR is working correctly, we would expect current ratings to be indicative of future performance. Fixing the issues I raised in the initial post produces *substantially* better prediction performance. This indicates that GR is a poor proxy for player skill, and that it's not too difficult to come up with a better proxy by fixing some of the unusual choices that GR makes.

I want to be clear that this isn't a sleight at Ghost - understanding how ratings should be done has come a LONG way since he first created the GR system. GR has been a very good thing for this site - and now we're fortunate in that we can take advantage of several years of research in player ratings to produce an even more accurate system.

" we will see that top players really won't play against beginners or medium players any more."

It is our intention to ensure that this doesn't happen. We have lots of ideas for this, which I'll expand on when I'm back at my desk.

"I don't know if my contribution is useful"

It's absolutely useful! All contributions to this thread are useful - we're looking to gauge community reaction.

@MrcsAurelius--Keep in mind that GR doesn't just inflate solos among top players--it inflates *everything*. Is making it into a 5WD among top players, as a top player, worth 4x as much net gain as making it into a 5WD among average players, as an average player? For that matter, is being defeated worth 4x as much rating loss?

Couple rebuttal points..

The Rating Shield at PlayDip doesn't really negatively effect the rating much. It's just meant as an incentive to pick up surrendered games, but as you're only granted scoring based on the percentage of the game you're in it's not that easy to really use it to pad your rankings. That said, their rating system does have other flaws.

There's a bit of a myth about "solo's rarely happen in top games". The real story is solo's rarely happen in games full of "quite good" players, but when you get into those top games you see a lot more heavier use of extreme hail mary's and very strong kingmaking by great players looking to take everybody out with them as they make their final stand. Solo's happen all the time in top games.

" We will see that top players really won't play against beginners or medium players any more."

Possibly. There is some dissincentive in this sort of system to play lower players because you have more to lose and less to win the higher you go up if you're competing beneath your calculated potential. You may even see people start to "rest on their laurels" once they get to a certain level if the rating swings gets too high. I certainly think you'll see a lot more top players taking a much stronger preference towards anonymous games.

On the heels of Lethologica's point, i'd most certainly endorse any new system that properly incentivizes playing for "best possible outcome".

The biggest flaw we have in my eyes at the moment is the "a draw is a draw is a draw" philosophy. The game just doesn't work properly with black and white attitudes towards a win and end games generally suffer here due to a lot of players wishing to wrap things up early to move onto the next game the moment a solo seems unlikely. This attitude can directly be traced back to the GR where the reward for a 5WD, 4WD, 3WD, 2WD isn't really that wide of a margin.

As long as games gated by rating have a floor somewhat below the rating of the creator (-10%?, -20%?), then the problem of ratings inflation and elitism should be mitigated. There are always webDip points and reliability ratings which are alternate measures of player quality and used as secondary and tertiary gates as they are now.

Besides, players at an extremely rarified ratings level won't be able to use ratings as a gate because there are so few opponents with whom they can be matched!

What they meant to say was that the new ratings system would be solely determined by your record in PPSC games. Only PPSC games are going to be rated moving forward.

What is "PPSC" ??

I think fear that highly rated players won't play with lower rated players is overstated. I think there is a pretty general consensus that the easiest way to boost your GR as a top player is to play live games and low to mid level games (looking at you VI with all your 12 hour games). It's not the case that strong players can get unlucky and get knocked out by weak players. Strong players don't get knocked out by weak players.

I also agree with VI's point about the draw problem actually being a "pretty good" player problem. I think it's because people think setting up stalemates shows their strategic genius. It's really quite the opposite. The best way to go from being pretty good to very good is to get more aggressive.

I don't think we should implement a system that would incentivize eliminating higher rated players. That's not with the nature of Diplomacy.

Anonymous games are the way around it

Trueskill doesn't incentivize eliminating higher-rated players.

Ah. Thanks Letho. I haven't had time to do my own research yet, and got that impression from reading the thread.

In that case, I definitely support integrating TS.

"I think there is a pretty general consensus that the easiest way to boost your GR as a top player is to play live games and low to mid level games (looking at you VI with all your 12 hour games). "

I guess that's true in a sense. You're certainly rise quicker if you churn out more games and sure, playing in general public games probably are easier to solo in then organized top ranked games or tournaments. Top players will still play lower rank just to get those standard games, they just won't play them non-anonymous.

"Trueskill doesn't incentivize eliminating higher-rated players."

This might be accurate, ATC may need to elaborate on this. There's most certainly incentive to try to get into games with higher guys and of course there's always incentive to try to take out the tough competition, but from a scoring perspective I'm actually not sure if you get more points for knocking off the top guys or if points are same as long you're in the draw. I know in GR it was latter.

I'm too tired to check if it's been asked, but will TrueSkill take into account the average results of a given country when calculating results for a particular game?

Should anonymous games be the default?

I wouldn't think so Tr.

@TrPrado
No, it wouldn't take country statistics into account. No major Diplomacy rating systems do.

Sure an integrated system would be nice. My only concern with a rating system including GR is that if you dont play the good players you'll take big losses in rankings when you lose. I don't think that's necessarily fair.

Super generic example.. with 7 people playing you have situations where say as Austria your 3 neighbors decide to gangbang you spring 1901 in gunboat just because. If you lose to low rank players you'll lose huge points even though 3v1 its not necessarily a lack of skill. Even if you survive a poorly thought out gang bang you might be left at 3 centers while everyone else is 6+ centers and depending on the board not have a chance to recover. A lot of losses/wins i have seen are due to other people doing STUPID things. You can say you need diplomatic skills to change their mind. Maybe mine aren't the best but soooo many games end with "well shield you are right he did stab me to get solo. I should have not left myself wide open like you suggested but i really just didnt think it wold happen."

Not that I'm always right but being right is little comfort when everyone just lost to a preventable solo.

so am I just whining? Maybe but In contrast with chess you really know much better that if X beats Y then there's little recourse except to assume 1 player is better. There's no chance that player 4s stupidity or CD gives an undeserved solo.

I usually play random people so these issues are a problem for me as I am better ranked than most. Even if I win the gain is small. I could play only better players but that really limits who I can play.





Plus it seems like half my games have multi/bans/NMRs/CD that ruin the balance of the game.

Quote from the current SC leader on the final table of PlayDip's Elite Tournament after proposing a 6 way draw.

"Does anyone here at this point really think a solo is going to happen?

I don't. So what are we going to continue playing for? a draw whittle? who is for the chop? England? Austria? Turkey? I'm not into that"

I think we can put to bed the idea that the "draw is a draw is a draw" philosophy is a direct result of the GR scoring system. I very much doubt that there are many people who know enough about GR to even make these calls :p