AI bots on webDiplomacy: Skynet edition!

Site announcements from the moderator team go here.
Message
Author
ppaquette
Posts: 1
Joined: Wed Nov 14, 2018 5:54 pm
Karma: 13

Re: AI bots on webDiplomacy: Skynet edition!

#21 Post by ppaquette » Sat Sep 14, 2019 1:15 am

Thanks very much Kestas. Congrats to you and your team.

Restitution, you can have access to the code on the following repos on Github:

- https://github.com/diplomacy/diplomacy/
- https://github.com/diplomacy/research/

Just the beginning of a new world!
13

Murder Cat
Posts: 1
Joined: Mon Jul 29, 2019 9:32 pm
Karma: 2

Re: AI bots on webDiplomacy: Skynet edition!

#22 Post by Murder Cat » Sat Sep 14, 2019 1:36 am

This is awesome! Hopefully the server starts processing again soon so I can try this out--I'm practically giddy!
2

captainmeme
Posts: 125
Joined: Fri Sep 29, 2017 3:06 pm
Location: Manchester, UK
Karma: 204

Re: AI bots on webDiplomacy: Skynet edition!

#23 Post by captainmeme » Sat Sep 14, 2019 1:37 am

ppaquette wrote:
Sat Sep 14, 2019 1:15 am
Thanks very much Kestas. Congrats to you and your team.

Restitution, you can have access to the code on the following repos on Github:

- https://github.com/diplomacy/diplomacy/
- https://github.com/diplomacy/research/

Just the beginning of a new world!
I've read the paper, and a lot went over my head, but this is incredible and massive thanks for doing it!

I had one question about it - the paper says "orders from the last movementphase are enough to infer the current relationship between the powers". I know it's not easy to tell exactly what it would do in any given situation, but does this generally mean that if you have been supporting a bot for the whole game but then stop for one round, that bot will not consider you to be an ally anymore?
2

peterwiggin
Site Admin
Site Admin
Posts: 167
Joined: Fri Jun 02, 2017 7:55 am
Location: Boston, MA
Karma: 401

Re: AI bots on webDiplomacy: Skynet edition!

#24 Post by peterwiggin » Sat Sep 14, 2019 2:03 am

captainmeme wrote:
Sat Sep 14, 2019 1:37 am

I've read the paper, and a lot went over my head, but this is incredible and massive thanks for doing it!

I had one question about it - the paper says "orders from the last movementphase are enough to infer the current relationship between the powers". I know it's not easy to tell exactly what it would do in any given situation, but does this generally mean that if you have been supporting a bot for the whole game but then stop for one round, that bot will not consider you to be an ally anymore?
That's exactly what that means.
4

peterwiggin
Site Admin
Site Admin
Posts: 167
Joined: Fri Jun 02, 2017 7:55 am
Location: Boston, MA
Karma: 401

Re: AI bots on webDiplomacy: Skynet edition!

#25 Post by peterwiggin » Sat Sep 14, 2019 2:04 am

The bots we have on webDip use a model developed by a group at MILA and described in this paper: https://arxiv.org/abs/1909.02128. Here, I'm going to give a summary of how the model works. For details, go read the paper!

In traditional computing, the programmer writes down what the program should do given specific inputs. In machine learning, the programmer instead defines a model and then uses examples (data) to teach the model how it should behave given certain inputs. For example, if I wanted a program to look at pictures of cats and dogs and tell me which ones were cats and which were dogs, I could attempt to write down a procedure by which my program could make that judgement. If I wanted to do this with machine learning, I would instead program a mathematical function that maps from a picture to a cat/dog prediction, and then use examples of cat and dog pictures to "train" the function's parameters so that it gives accurate predictions.

For a Diplomacy bot, we need to learn a policy that maps from a state (the placement of units on the board and possibly some history) to an action (orders for the bot's units). I'm going to start by describing the model used to map from state to action, and then describe how the model was trained.

We begin with an input that represents the board state and the orders from the previous phase. To represent the board state, for each of the 75 provinces and 6 coasts we encode some dynamic features such as whether there's a unit there, what type of unit, and who owns that unit, as well as some static features, such as whether it's a coast or province or sea space, and whether it's a supply center or not. The orders from the previous state are similarly encoded.

The encodings for the board state and previous-phase orders are then put into a model called a graph convolutional network, which is just a complicated mathematical function that looks at the relationships between the provinces and previous orders to arrive at a numerical representation of each province that incorporates information from the rest of the board state. The most important thing to know about this function is that it also takes the connections between provinces as input in order to take into account the movement rules.

A different function then looks at the output from the first network to produce orders, mapping from the state for that province to a probability for each of the possible orders for the unit in that province. The final order for the unit is then drawn from those probabilities. For powers with more than one unit needing orders, the orders are predicted one at a time, and the orders already chosen are taken into account when predicting the order for each proceeding unit.

The functions that encode the current state and use that encoding to predict orders both contain lots of parameters, which must be learned. Most parameter values would result in very bad orders, so the trick to all this is finding the parameters that lead to strong play. To do this, we get lots of examples of board states from human games as well as the orders that the humans submitted in each of those states. We pick a few board states and feed them into our models to get predicted orders out. We then compare those predictions to what the humans actually did and adjust the parameters to favor the human orders a little bit. By repeating this process, the models eventually output orders similar to what a human would in a given situation.

The drawback of this training method is that it requires lots of human examples. We can also train the models by having them play themselves and adjusting the parameters to make them more likely to capture centers and win. In this case, the two training methods yield bots with pretty similar strengths.
18

dargorygel
Site Moderator
Site Moderator
Posts: 1897
Joined: Mon Dec 18, 2017 1:55 pm
Location: Over the rainbow
Karma: 1326
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#26 Post by dargorygel » Sat Sep 14, 2019 2:33 am

This is incredible.
HEARTY thanks and congrats to the team that PW has written about, and to our Devs, Admins, and Mods for seeing this, imagining this, working for this, building this, checking this, testing this and giving us this.

But supporting awards go to the whole mod team for keeping their mouths (and keyboards) shut about this. I am proud of the sound of silence.

And of course, special something goes to jmo, for giving up his robot-children for this worthy cause.
4

MrAsiago
Posts: 1
Joined: Tue Jul 16, 2019 7:35 am
Karma: 1

Re: AI bots on webDiplomacy: Skynet edition!

#27 Post by MrAsiago » Sat Sep 14, 2019 3:01 am

https://webdiplomacy.net/board.php?gameID=246267

Perhaps this isn't the place for it, but my game just forced a draw against my will. I'm guessing this has to do with the outages that are happening right now, but is there any way I can get the game to restart?
1

jmo1121109
Developer
Developer
Posts: 879
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2108
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#28 Post by jmo1121109 » Sat Sep 14, 2019 3:06 am

It was due to the outage. No, sorry but you can start another one.
1

jmo1121109
Developer
Developer
Posts: 879
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2108
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#29 Post by jmo1121109 » Sat Sep 14, 2019 3:10 am

bo_sox48 wrote:
Fri Sep 13, 2019 11:14 pm
Tom Bombadil wrote:
Fri Sep 13, 2019 11:06 pm
This is pretty amazing. I can only imagine the effort behind the scenes to make it happen
There's six figures worth of work that has gone into this. Maybe more.
This is not a joking estimate either. The developmental cost of this if the site had been required to hire a development team for this project would have been over 100,000 USD hands down. The immense time of the people at the University and the time we put in here was extensive. And we as a site are thrilled that we are able to offer these types of features without cost and without ads to our users.

Thank you to the donators who's donations cover our hosting costs and allow that to be possible as well.
12

ibuprofen
Posts: 3
Joined: Thu Apr 19, 2018 4:33 pm
Location: Georgia
Karma: 6

Re: AI bots on webDiplomacy: Skynet edition!

#30 Post by ibuprofen » Sat Sep 14, 2019 3:25 am

The new AI are causing the server outages! The Singularity is here! Resist the Machines!!!!!
4

bo_sox48
Site Admin
Site Admin
Posts: 1712
Joined: Thu Jun 08, 2017 1:01 am
Karma: 1011
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#31 Post by bo_sox48 » Sat Sep 14, 2019 3:27 am

jmo1121109 wrote:
Sat Sep 14, 2019 3:10 am
bo_sox48 wrote:
Fri Sep 13, 2019 11:14 pm
Tom Bombadil wrote:
Fri Sep 13, 2019 11:06 pm
This is pretty amazing. I can only imagine the effort behind the scenes to make it happen
There's six figures worth of work that has gone into this. Maybe more.
This is not a joking estimate either. The developmental cost of this if the site had been required to hire a development team for this project would have been over 100,000 USD hands down. The immense time of the people at the University and the time we put in here was extensive. And we as a site are thrilled that we are able to offer these types of features without cost and without ads to our users.

Thank you to the donators who's donations cover our hosting costs and allow that to be possible as well.
And a bigger thank you to the developers who do this for free out of love for this platform and its members, and out of a desire to make it the best it can be.

Margaritas all around in Cabo this winter are on me.
8

BobMcBob
Posts: 169
Joined: Mon Feb 11, 2019 6:28 am
Location: Also Not Austalia
Karma: 155

Re: AI bots on webDiplomacy: Skynet edition!

#32 Post by BobMcBob » Sat Sep 14, 2019 3:27 am

jmo1121109 wrote:
Sat Sep 14, 2019 3:10 am
bo_sox48 wrote:
Fri Sep 13, 2019 11:14 pm
Tom Bombadil wrote:
Fri Sep 13, 2019 11:06 pm
This is pretty amazing. I can only imagine the effort behind the scenes to make it happen
There's six figures worth of work that has gone into this. Maybe more.
This is not a joking estimate either. The developmental cost of this if the site had been required to hire a development team for this project would have been over 100,000 USD hands down. The immense time of the people at the University and the time we put in here was extensive. And we as a site are thrilled that we are able to offer these types of features without cost and without ads to our users.

Thank you to the donators who's donations cover our hosting costs and allow that to be possible as well.
I presume we didn't actually get to $100000 of donations though. Is it also partially because it was a university thing that this was possible?
1

jmo1121109
Developer
Developer
Posts: 879
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2108
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#33 Post by jmo1121109 » Sat Sep 14, 2019 3:36 am

BobMcBob wrote:
Sat Sep 14, 2019 3:27 am
jmo1121109 wrote:
Sat Sep 14, 2019 3:10 am
bo_sox48 wrote:
Fri Sep 13, 2019 11:14 pm


There's six figures worth of work that has gone into this. Maybe more.
This is not a joking estimate either. The developmental cost of this if the site had been required to hire a development team for this project would have been over 100,000 USD hands down. The immense time of the people at the University and the time we put in here was extensive. And we as a site are thrilled that we are able to offer these types of features without cost and without ads to our users.

Thank you to the donators who's donations cover our hosting costs and allow that to be possible as well.
I presume we didn't actually get to $100000 of donations though. Is it also partially because it was a university thing that this was possible?
Nope, this project was entirely free. The University gets publications, and I get..uhhh...hmmm, anyway enjoy.
15

bo_sox48
Site Admin
Site Admin
Posts: 1712
Joined: Thu Jun 08, 2017 1:01 am
Karma: 1011
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#34 Post by bo_sox48 » Sat Sep 14, 2019 3:40 am

You get paid in +1s.
6

BobMcBob
Posts: 169
Joined: Mon Feb 11, 2019 6:28 am
Location: Also Not Austalia
Karma: 155

Re: AI bots on webDiplomacy: Skynet edition!

#35 Post by BobMcBob » Sat Sep 14, 2019 3:49 am

So yeah, because it's a publication it's free. Makes sense. And don't worry, I'm sure you'll ge your share of +1's. I can't wait until I can finally actually play the bots though. As you may be able to tell, I'm very excited. Still waiting...
2

jmo1121109
Developer
Developer
Posts: 879
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2108
Contact:

Re: AI bots on webDiplomacy: Skynet edition!

#36 Post by jmo1121109 » Sat Sep 14, 2019 3:53 am

BobMcBob wrote:
Sat Sep 14, 2019 3:49 am
So yeah, because it's a publication it's free. Makes sense. And don't worry, I'm sure you'll ge your share of +1's. I can't wait until I can finally actually play the bots though. As you may be able to tell, I'm very excited. Still waiting...
For what? We turned them back on a little bit ago, can’t swear it won’t go down again but we’re fixing issues as they come up
4

BobMcBob
Posts: 169
Joined: Mon Feb 11, 2019 6:28 am
Location: Also Not Austalia
Karma: 155

Re: AI bots on webDiplomacy: Skynet edition!

#37 Post by BobMcBob » Sat Sep 14, 2019 4:04 am

Yeah, I missed that. I didn't realise you had to go into the game to start them. I'm losing now... Oh well, this change is great. Thanks guys!
2

Temasek22
Posts: 230
Joined: Sat Jan 05, 2019 8:00 am
Location: Not Singapore
Karma: 88

Re: AI bots on webDiplomacy: Skynet edition!

#38 Post by Temasek22 » Sat Sep 14, 2019 4:34 am

I found out that the AI just stalemates itself eternally...

My game was Let's go bots!-2
2

Claesar
Site Moderator
Site Moderator
Posts: 1255
Joined: Tue Oct 03, 2017 10:34 am
Karma: 759

Re: AI bots on webDiplomacy: Skynet edition!

#39 Post by Claesar » Sat Sep 14, 2019 6:27 am

As part of the testing team (i.e. all mods) that got to try these before it went live, I have to say I've been very impressed by the level of play from the bots! Of course they're not perfect, but neither are humans. They should fit right in.
2

MadMarx
Gold Donator
Gold Donator
Posts: 200
Joined: Sun Dec 31, 2017 12:01 am
Karma: 53

Re: AI bots on webDiplomacy: Skynet edition!

#40 Post by MadMarx » Sat Sep 14, 2019 6:31 am

WTF I don’t log on for three months and all hell breaks loose... First thing I did was donate everything I had in PayPal $41.13 not a ton but if more do something similar it will add up...
3

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests