Artificial intelligence in 1v1 diplomacy

[General Subutai]

I am very curious as to whether an AI can be developed for FvA and GvI variants, where the AI teaches itself how to play diplomacy by playing thousands upon millions of games against itself, in a similar manner to the Dota2 OpenAI, Google's AlphaGo and AlphaZero.

I am a complete layman in the field of machine learning.

One may ask, what's the point of developing a superhuman AI that can play better than any human player? I think it may be a fun (if ultimately exhaustive/depressing) experience to play against it in 1v1. It's likely we'll see some novel early to midgame openings and unexpected tactics/gambits on behalf of the AI. It will provide a challenge to players to find unexpected methods to beat the AI, perhaps using unorthodox strategies. It would also be interesting to watch a highly skilled human play against the AI, and what sort of games the AI plays against itself.

[Foxcastle]

Look, if you want Xorxes to give you some pointers, you probably just have to ask...

[General Subutai]

I think it's conceivable for a modern machine learning program to train itself not only to reliably defeat very skilled players like Xorxes, but also to do so in an incredibly short time period

With just 4 hours of training time (and 44 million games of playing against itself) AlphaZero defeated the sophisticated chess engine Stockfish

[CptMike]

I think it's conceivable for a modern machine learning program to train itself not only to reliably defeat very skilled players like Xorxes, but also to do so in an incredibly short time period.

With just 4 hours of training time (and 44 million games of playing against itself) AlphaZero defeated the sophisticated chess engine Stockfish

That's very interesting and worth developing, even more if the IA can learn in simulating games. I am confident that IA will very fast reach a very good level, if not the first rank, but human will come back and win at the end, learning the way the IA plays.

GvI and FvA are at the same time easier and more complex than go. The new element if the fact that movements are simultaneous and odds take a relevant part in the game even if there are no dice.

Very impressed that AlphaGo leart in 4 hours how to beat a chess programm... That seems to prove that machine learning learns strategies and does not calculate all the possibilities !

[y2kjbk]

It would be feasible to do but not as straightforward as a chess program since you’re dealing with incomplete information concerning the exact state change that your move will have on the game, even if the range of necessary board states to consider is smaller. You’d have to incorporate aspects of programs they’ve made that can play poker.

[CCR]

It already exists, in theory.
The wonderful Albert bots can play in any map available on the Daide mapper engine.

[CptMike]

It already exists, in theory.
The wonderful Albert bots can play in any map available on the Daide mapper engine.

Albert is not very strong...

[CCR]

I challenge you - all of you - to play 7 games agaisnt 6 of the latest alberts.

Then post here your results.

References:
Albert bot:
https://sites.google.com/site/diplomacyai/home/daide
DAIDE server and mapper:
http://www.ellought.demon.co.uk/dipai/

[bozotheclown]

I challenge you - all of you - to play 7 games agaisnt 6 of the latest alberts.

Then post here your results.

References:
Albert bot:
https://sites.google.com/site/diplomacyai/home/daide
DAIDE server and mapper:
http://www.ellought.demon.co.uk/dipai/

Has this AI ever been tested in 1 vs. 1?

[ghug]

Is important to remember that Google has gigantic pieces of specialized hardest to put behind AlphaGo. That 4 hour number isn't really meaningful unless someone here wants to shell out a big chunk of change for TPU time.

I don't think a 1v1 AI would need to be nearly as sophisticated though. People correctly note the added complexity of concurrent movement, but both of our variants are very limited in the amount of "reasonable" movesets at any given time, to the point where I think reinforcement learning techniques could very quickly iterate to a point of competence from which to expand.

[CptMike]

[...] reinforcement learning techniques could very quickly iterate to a point of competence from which to expand.

Let's try it...

[rdrivera2005]

Do you really think Xorxes isn't a bot?

[Yigg]

You know, I've already seen this movie. I don't like how it ends.

[xorxes]

Daisy, Daisy,
Give me your answer do...
I'm half crazy,
All for the love of you...

[David E. Cohen]

Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.

[tamilse]

That's right Artificial Intelligence is now implemented every where.

I am very curious as to whether an AI can be developed for FvA and GvI variants, where the AI teaches itself how to play diplomacy by playing thousands upon millions of games against itself, in a similar manner to the Dota2 OpenAI, Google's AlphaGo and AlphaZero.

I am a complete layman in the field of machine learning.

One may ask, what's the point of developing a superhuman AI that can play better than any human player? I think it may be a fun (if ultimately exhaustive/depressing/Artificial Intelligence) experience to play against it in 1v1. It's likely we'll see some novel early to midgame openings and unexpected tactics/gambits on behalf of the AI. It will provide a challenge to players to find unexpected methods to beat the AI, perhaps using unorthodox strategies. It would also be interesting to watch a highly skilled human play against the AI, and what sort of games the AI plays against itself.

[mhsmith0]

I'd actually suggest that, much more than chess, the # of reasonable moves in a given situation is drastically limited.

Based on current meta, there are only six outcomes that are reasonably plausible to be optimal from spring 1901

Austria goes Anti-Apline (Ven/Tyr/Gal), Porcupine (Alb/Tyr/Gal), or Italian (Alb/Tyr/Tri) (some variations going Rum instead of Gal aren't NECESSARILY sub-optimal, but you're flat-out surrendering any chance of getting Germany for the most part in exchange for somewhat limited flexibility with future moves)
France goes Belgian Waffle (Eng/Pic/Bur) or Swiss Split (ENG/Bur/Pie)

Fall 1901 moves come directly from spring 1901 moves

Winter 1901 builds are mainly known (in some cases there's some flexibility for France building F Brest vs A Par, or Austria building F Tri or 2 armies, etc)

Spring 1902 moves often center around Munich (whether to move towards Munich or leave it blank for a turn) with most of the resulting moves being fairly known, and the Munich question is generally a coin flip sort of thing, where attacking or ignoring it is better or worse depending on what the other guy does

After spring 1902, things largely become either fairly obviously optimal moves, or specific tactical coin flips whereby one power or the other needs to "win" some number x of those to pull off the solo.

[Restitution]

I don't believe that it would be especially hard to build a neural net superior to any human being.

[Restitution]

Can't see how to edit the post.

I think the biggest difficulty is that Diplomacy has some WIFOM in it, since moves are simultaneous. So the AI would need to behave probabilistically, since being predictable would make it beatable after repeated games. I don't think that would necessarily be *super* hard, but it's the real distinguishing feature between 1v1 Dip and say Chess or Go.

My guess is that you can solve that by taking the output weights of the NN as a moveset's probability distribution, rather than normalizing to 0 or 1 to determine the action. Although the fact that moveset's components' probabilities shouldn't be measured independently kinda screws with that.

There's also the issue of how you actually gather training data for this thing, unless you're just going to have the AI play itself. My guess is that it would be very difficult to just get the major diplomacy sites to make their game databases accessible.

[mhsmith0]

I think a well designed AI playing itself could probably get reasonably close to an optimal moveset before too long. There just aren't that many meaningful decision points in 1v1 each turn (in particular, unlike chess you can pretty much throw out a LOT of potential moves as obviously bad without too much further analysis, and a substantial # of games have a clear advantage by the end of 1902 or at least 1903, so the number of pieces to track isn't huge either)

Some simple-ish examples...
http://webdiplomacy.net/board.php?gameI ... #gamePanel
Maximus won the coin flip surrounding the Pie/Ven armies in fall 1902, and as a result had a dominant position entering 1903 (and winning the coin flip around Par/Ruhr/Kiel vs Mun/Berlin in spring 1903 only made that edge stronger)

http://webdiplomacy.net/board.php?gameI ... #gamePanel
CptMike won the Bur/Tyr coin flip (Bur/Mun is GREAT for France if it succeeds, and Tyr/Mar isn't godawaful for France if Mar holds), and it got more complicated mainly because i was in an early bad position and needed to run some dice rolls to have a chance
etc