Artificial intelligence in 1v1 diplomacy

Use this forum to discuss Diplomacy strategy.
Forum rules
This forum is limited to topics relating to the game Diplomacy only. Other posts or topics will be relocated to the correct forum category or deleted. Please be respectful and follow our normal site rules at http://www.webdiplomacy.net/rules.php.
Message
Author
General Subutai
Posts: 2
Joined: Wed Jul 04, 2018 12:43 pm
Karma: 6

Artificial intelligence in 1v1 diplomacy

#1 Post by General Subutai » Thu Jul 05, 2018 11:04 pm

I am very curious as to whether an AI can be developed for FvA and GvI variants, where the AI teaches itself how to play diplomacy by playing thousands upon millions of games against itself, in a similar manner to the Dota2 OpenAI, Google's AlphaGo and AlphaZero.

I am a complete layman in the field of machine learning.

One may ask, what's the point of developing a superhuman AI that can play better than any human player? I think it may be a fun (if ultimately exhaustive/depressing) experience to play against it in 1v1. It's likely we'll see some novel early to midgame openings and unexpected tactics/gambits on behalf of the AI. It will provide a challenge to players to find unexpected methods to beat the AI, perhaps using unorthodox strategies. It would also be interesting to watch a highly skilled human play against the AI, and what sort of games the AI plays against itself.
6

Foxcastle
Posts: 1623
Joined: Sat Dec 30, 2017 11:48 pm
Location: Night Vale
Karma: 219

Re: Artificial intelligence in 1v1 diplomacy

#2 Post by Foxcastle » Thu Jul 05, 2018 11:48 pm

Look, if you want Xorxes to give you some pointers, you probably just have to ask...
3

General Subutai
Posts: 2
Joined: Wed Jul 04, 2018 12:43 pm
Karma: 6

Re: Artificial intelligence in 1v1 diplomacy

#3 Post by General Subutai » Fri Jul 06, 2018 3:04 am

I think it's conceivable for a modern machine learning program to train itself not only to reliably defeat very skilled players like Xorxes, but also to do so in an incredibly short time period

With just 4 hours of training time (and 44 million games of playing against itself) AlphaZero defeated the sophisticated chess engine Stockfish

CptMike
Posts: 228
Joined: Sun Dec 31, 2017 8:12 am
Location: Liège, BE
Karma: 24

Re: Artificial intelligence in 1v1 diplomacy

#4 Post by CptMike » Fri Jul 06, 2018 7:09 am

General Subutai wrote:
Fri Jul 06, 2018 3:04 am
I think it's conceivable for a modern machine learning program to train itself not only to reliably defeat very skilled players like Xorxes, but also to do so in an incredibly short time period.

With just 4 hours of training time (and 44 million games of playing against itself) AlphaZero defeated the sophisticated chess engine Stockfish
That's very interesting and worth developing, even more if the IA can learn in simulating games. I am confident that IA will very fast reach a very good level, if not the first rank, but human will come back and win at the end, learning the way the IA plays.

GvI and FvA are at the same time easier and more complex than go. The new element if the fact that movements are simultaneous and odds take a relevant part in the game even if there are no dice.

Very impressed that AlphaGo leart in 4 hours how to beat a chess programm... That seems to prove that machine learning learns strategies and does not calculate all the possibilities !

y2kjbk
Posts: 1197
Joined: Thu Oct 19, 2017 4:23 pm
Karma: 18

Re: Artificial intelligence in 1v1 diplomacy

#5 Post by y2kjbk » Sat Jul 07, 2018 9:39 am

It would be feasible to do but not as straightforward as a chess program since you’re dealing with incomplete information concerning the exact state change that your move will have on the game, even if the range of necessary board states to consider is smaller. You’d have to incorporate aspects of programs they’ve made that can play poker.

CCR
Posts: 41
Joined: Sun Dec 31, 2017 4:51 pm
Karma: 1

Re: Artificial intelligence in 1v1 diplomacy

#6 Post by CCR » Sat Jul 07, 2018 12:08 pm

It already exists, in theory.
The wonderful Albert bots can play in any map available on the Daide mapper engine.
1

CptMike
Posts: 228
Joined: Sun Dec 31, 2017 8:12 am
Location: Liège, BE
Karma: 24

Re: Artificial intelligence in 1v1 diplomacy

#7 Post by CptMike » Sun Jul 08, 2018 7:40 pm

CCR wrote:
Sat Jul 07, 2018 12:08 pm
It already exists, in theory.
The wonderful Albert bots can play in any map available on the Daide mapper engine.
Albert is not very strong...

CCR
Posts: 41
Joined: Sun Dec 31, 2017 4:51 pm
Karma: 1

Re: Artificial intelligence in 1v1 diplomacy

#8 Post by CCR » Sun Jul 08, 2018 11:14 pm

I challenge you - all of you - to play 7 games agaisnt 6 of the latest alberts.

Then post here your results.

References:
Albert bot:
https://sites.google.com/site/diplomacyai/home/daide
DAIDE server and mapper:
http://www.ellought.demon.co.uk/dipai/

bozotheclown
Posts: 2171
Joined: Sun Dec 31, 2017 8:13 am
Karma: 12

Re: Artificial intelligence in 1v1 diplomacy

#9 Post by bozotheclown » Mon Jul 09, 2018 5:21 am

CCR wrote:
Sun Jul 08, 2018 11:14 pm
I challenge you - all of you - to play 7 games agaisnt 6 of the latest alberts.

Then post here your results.

References:
Albert bot:
https://sites.google.com/site/diplomacyai/home/daide
DAIDE server and mapper:
http://www.ellought.demon.co.uk/dipai/
Has this AI ever been tested in 1 vs. 1?

ghug
Posts: 2096
Joined: Mon Mar 20, 2017 3:51 pm
Location: Seattle
Karma: 541

Re: Artificial intelligence in 1v1 diplomacy

#10 Post by ghug » Tue Jul 10, 2018 11:34 pm

Is important to remember that Google has gigantic pieces of specialized hardest to put behind AlphaGo. That 4 hour number isn't really meaningful unless someone here wants to shell out a big chunk of change for TPU time.

I don't think a 1v1 AI would need to be nearly as sophisticated though. People correctly note the added complexity of concurrent movement, but both of our variants are very limited in the amount of "reasonable" movesets at any given time, to the point where I think reinforcement learning techniques could very quickly iterate to a point of competence from which to expand.
1

CptMike
Posts: 228
Joined: Sun Dec 31, 2017 8:12 am
Location: Liège, BE
Karma: 24

Re: Artificial intelligence in 1v1 diplomacy

#11 Post by CptMike » Wed Jul 11, 2018 12:42 am

ghug wrote:
Tue Jul 10, 2018 11:34 pm
[...] reinforcement learning techniques could very quickly iterate to a point of competence from which to expand.
Let's try it...

rdrivera2005
Posts: 1503
Joined: Tue Oct 31, 2017 5:17 pm
Location: Porto Alegre, Brasil
Karma: 64

Re: Artificial intelligence in 1v1 diplomacy

#12 Post by rdrivera2005 » Wed Jul 11, 2018 4:28 am

Do you really think Xorxes isn't a bot?
3

Yigg
Posts: 61
Joined: Sun Dec 31, 2017 3:56 am
Location: Wisconsin, United States
Karma: 79

Re: Artificial intelligence in 1v1 diplomacy

#13 Post by Yigg » Wed Jul 11, 2018 5:12 am

You know, I've already seen this movie. I don't like how it ends.

xorxes
Posts: 1699
Joined: Sun Dec 31, 2017 9:45 am
Karma: 145

Re: Artificial intelligence in 1v1 diplomacy

#14 Post by xorxes » Fri Jul 13, 2018 11:19 pm

Daisy, Daisy,
Give me your answer do...
I'm half crazy,
All for the love of you...
1

David E. Cohen
Posts: 72
Joined: Thu Jan 25, 2018 1:27 am
Location: Treading the Path to Diplo-Shambhala
Karma: 32
Contact:

Re: Artificial intelligence in 1v1 diplomacy

#15 Post by David E. Cohen » Sun Aug 12, 2018 7:04 am

Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.

tamilse
Posts: 1
Joined: Tue Mar 19, 2019 2:31 pm

Re: Artificial intelligence in 1v1 diplomacy

#16 Post by tamilse » Tue Mar 19, 2019 2:33 pm

That's right Artificial Intelligence is now implemented every where.
General Subutai wrote:
Thu Jul 05, 2018 11:04 pm
I am very curious as to whether an AI can be developed for FvA and GvI variants, where the AI teaches itself how to play diplomacy by playing thousands upon millions of games against itself, in a similar manner to the Dota2 OpenAI, Google's AlphaGo and AlphaZero.

I am a complete layman in the field of machine learning.

One may ask, what's the point of developing a superhuman AI that can play better than any human player? I think it may be a fun (if ultimately exhaustive/depressing/Artificial Intelligence) experience to play against it in 1v1. It's likely we'll see some novel early to midgame openings and unexpected tactics/gambits on behalf of the AI. It will provide a challenge to players to find unexpected methods to beat the AI, perhaps using unorthodox strategies. It would also be interesting to watch a highly skilled human play against the AI, and what sort of games the AI plays against itself.

mhsmith0
Posts: 158
Joined: Wed Jan 03, 2018 12:35 am
Karma: 52

Re: Artificial intelligence in 1v1 diplomacy

#17 Post by mhsmith0 » Tue Mar 19, 2019 4:19 pm

I'd actually suggest that, much more than chess, the # of reasonable moves in a given situation is drastically limited.

Based on current meta, there are only six outcomes that are reasonably plausible to be optimal from spring 1901

Austria goes Anti-Apline (Ven/Tyr/Gal), Porcupine (Alb/Tyr/Gal), or Italian (Alb/Tyr/Tri) (some variations going Rum instead of Gal aren't NECESSARILY sub-optimal, but you're flat-out surrendering any chance of getting Germany for the most part in exchange for somewhat limited flexibility with future moves)
France goes Belgian Waffle (Eng/Pic/Bur) or Swiss Split (ENG/Bur/Pie)

Fall 1901 moves come directly from spring 1901 moves

Winter 1901 builds are mainly known (in some cases there's some flexibility for France building F Brest vs A Par, or Austria building F Tri or 2 armies, etc)

Spring 1902 moves often center around Munich (whether to move towards Munich or leave it blank for a turn) with most of the resulting moves being fairly known, and the Munich question is generally a coin flip sort of thing, where attacking or ignoring it is better or worse depending on what the other guy does

After spring 1902, things largely become either fairly obviously optimal moves, or specific tactical coin flips whereby one power or the other needs to "win" some number x of those to pull off the solo.

Restitution
Posts: 47
Joined: Thu Jan 31, 2019 7:00 am
Karma: 16

Re: Artificial intelligence in 1v1 diplomacy

#18 Post by Restitution » Tue Mar 19, 2019 6:13 pm

I don't believe that it would be especially hard to build a neural net superior to any human being.

Restitution
Posts: 47
Joined: Thu Jan 31, 2019 7:00 am
Karma: 16

Re: Artificial intelligence in 1v1 diplomacy

#19 Post by Restitution » Tue Mar 19, 2019 7:51 pm

Can't see how to edit the post.

I think the biggest difficulty is that Diplomacy has some WIFOM in it, since moves are simultaneous. So the AI would need to behave probabilistically, since being predictable would make it beatable after repeated games. I don't think that would necessarily be *super* hard, but it's the real distinguishing feature between 1v1 Dip and say Chess or Go.

My guess is that you can solve that by taking the output weights of the NN as a moveset's probability distribution, rather than normalizing to 0 or 1 to determine the action. Although the fact that moveset's components' probabilities shouldn't be measured independently kinda screws with that.

There's also the issue of how you actually gather training data for this thing, unless you're just going to have the AI play itself. My guess is that it would be very difficult to just get the major diplomacy sites to make their game databases accessible.
Last edited by Restitution on Tue Mar 19, 2019 7:56 pm, edited 3 times in total.

mhsmith0
Posts: 158
Joined: Wed Jan 03, 2018 12:35 am
Karma: 52

Re: Artificial intelligence in 1v1 diplomacy

#20 Post by mhsmith0 » Tue Mar 19, 2019 9:21 pm

I think a well designed AI playing itself could probably get reasonably close to an optimal moveset before too long. There just aren't that many meaningful decision points in 1v1 each turn (in particular, unlike chess you can pretty much throw out a LOT of potential moves as obviously bad without too much further analysis, and a substantial # of games have a clear advantage by the end of 1902 or at least 1903, so the number of pieces to track isn't huge either)

Some simple-ish examples...
http://webdiplomacy.net/board.php?gameI ... #gamePanel
Maximus won the coin flip surrounding the Pie/Ven armies in fall 1902, and as a result had a dominant position entering 1903 (and winning the coin flip around Par/Ruhr/Kiel vs Mun/Berlin in spring 1903 only made that edge stronger)

http://webdiplomacy.net/board.php?gameI ... #gamePanel
CptMike won the Bur/Tyr coin flip (Bur/Mun is GREAT for France if it succeeds, and Tyr/Mar isn't godawaful for France if Mar holds), and it got more complicated mainly because i was in an early bad position and needed to run some dice rolls to have a chance
etc

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest