France vs Austria Bot Challenge

Use this forum to discuss Diplomacy strategy.
Forum rules
This forum is limited to topics relating to the game Diplomacy only. Other posts or topics will be relocated to the correct forum category or deleted. Please be respectful and follow our normal site rules at http://www.webdiplomacy.net/rules.php.
Message
Author
amdx
Posts: 2
Joined: Thu Feb 13, 2020 2:50 pm
Karma: 22

France vs Austria Bot Challenge

#1 Post by amdx » Wed Nov 11, 2020 8:53 pm

Hi all,

I’m part of a team at Facebook AI Research working on AI for Diplomacy. We recently published our results on developing a human-level bot for gunboat Diplomacy (thanks to webdiplomacy for the help!), and are now working toward a superhuman bot.

As part of that effort, we’ve developed a bot for the France vs. Austria variant that we believe is very strong, and possibly superhuman. To see just how strong it is, we’ve worked with the webdip team to add the bot to the site so everyone can play against it. When you launch a France vs. Austria bot game, you will now be playing against our bot. We welcome your attempts to poke around for weaknesses and see if you can consistently beat our bot! We would especially appreciate it if any expert France vs. Austria players could try their best to take down the bot.

We are not adding our 7-player bot to the site for now because the computational cost would be too high, but we hope to do that at some point in the future!
18

kestasjk
Co-Owner
Co-Owner
Posts: 53
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 138
Contact:

Re: France vs Austria Bot Challenge

#2 Post by kestasjk » Wed Nov 11, 2020 9:43 pm

Nicely done guys.. I wonder if anyone is up to the challenge?
2

MadMarx
Gold Donator
Gold Donator
Posts: 291
Joined: Sun Dec 31, 2017 12:01 am
Karma: 145

Re: France vs Austria Bot Challenge

#3 Post by MadMarx » Wed Nov 11, 2020 9:53 pm

kestasjk wrote:
Wed Nov 11, 2020 9:43 pm
Nicely done guys.. I wonder if anyone is up to the challenge?
I might have some free time in the near future, would be happy to help out a bit...
4

President Eden
Posts: 548
Joined: Fri Oct 20, 2017 2:11 pm
Karma: 821

Re: France vs Austria Bot Challenge

#4 Post by President Eden » Wed Nov 11, 2020 10:01 pm

Bot leaves no room for error, that's for sure.
2

President Eden
Posts: 548
Joined: Fri Oct 20, 2017 2:11 pm
Karma: 821

Re: France vs Austria Bot Challenge

#5 Post by President Eden » Wed Nov 11, 2020 10:16 pm

Dibs on first win?

For all mankind - http://webdiplomacy.net/board.php?gameID=331847
5

Polynoamial
Posts: 5
Joined: Thu Mar 21, 2019 9:04 pm
Karma: 11

Re: France vs Austria Bot Challenge

#6 Post by Polynoamial » Wed Nov 11, 2020 10:22 pm

Thanks for trying it out Eden! Do you think the bot made any mistakes in the games you played?
3

President Eden
Posts: 548
Joined: Fri Oct 20, 2017 2:11 pm
Karma: 821

Re: France vs Austria Bot Challenge

#7 Post by President Eden » Wed Nov 11, 2020 10:37 pm

Polynoamial wrote:
Wed Nov 11, 2020 10:22 pm
Thanks for trying it out Eden! Do you think the bot made any mistakes in the games you played?
Forgive me for a bit of pedantry; law school is ruining me.

By "mistake" I understand you to mean a tactically suboptimal decision for every realistic moveset the opponent could make, as opposed to simply "guessing wrong" where there are multiple valid movesets for both sides.

For example, in a classic stalemate position in Germany, there are often situations where the defenders need to leave Munich or Berlin unguarded for one turn while the final piece of the stalemate position gets into its proper spot. I don't consider it a mistake to guess the wrong piece to defend, because both Munich and Berlin would be valid targets for attack, and one valid target must be left undefended.
On the other hand, in the game I linked, the bot, on its final turn, submitted NTH -> Nwy -> StP. I think the bot only had two valid movesets there: NTH S Nwy (to defend Nwy from attack) or NTH -> Den (to ensure that I don't take Norway without losing Denmark). I would consider that moveset a mistake.

I only notice the example above as a clear-cut mistake, but I didn't like the bot's decisions to attack Munich from Burgundy in Spring and Autumn 1904. Burgundy is a bottleneck that the bot took just barely too long to clear. If the bot had advanced to Ruhr in Spring 1904 and then advanced Paris to Burgundy with support in Autumn 1904, it would have earned at least one more year to fight. My orders would have had to change in Autumn 1905, which might have allowed it to outguess me in Italy. I would have liked my odds, but the lack of pressure in Germany helped me, and I think it came down to over-prioritizing pressure on Munich instead of advancing the armies.

I'm also not sure about its diversions to pick up Edinburgh and Portugal, as I think the free hand that those decisions gave me in Scandinavia were key to my win. But that may just be a consequence of not using the armies efficiently enough; that is, it's not that picking up the centers was a mistake, but rather a gamble that the increased materiel was worth the tempo loss, and the gamble failed because the new materiel wasn't used effectively.

I'll be honest, it's been two years since I have competitively studied this format, and I would welcome more seasoned FvA players to comment.
5

President Eden
Posts: 548
Joined: Fri Oct 20, 2017 2:11 pm
Karma: 821

Re: France vs Austria Bot Challenge

#8 Post by President Eden » Wed Nov 11, 2020 10:39 pm

All that said, small sample size so far for me, but the bot seems very, very good. I felt the pressure the whole time. Love the addition!
2

goldfinger0303
Site Moderator
Site Moderator
Posts: 1392
Joined: Fri Sep 29, 2017 3:17 pm
Karma: 569

Re: France vs Austria Bot Challenge

#9 Post by goldfinger0303 » Wed Nov 11, 2020 10:39 pm

Woof. Granted I'm not as sharp I used to be, but I'd still place my initial reaction that the bot is as good as a highly skilled player.

http://www.webdiplomacy.net/board.php?gameID=331851

Bot made a few mistakes by my judgement. It should've built a fleet earlier, and the year before last, Apulia should've been force disbanded.

It's definitely beatable, and I will be trying again. But as Eden said, it punishes mistakes.
2

RoganJosh
Silver Donator
Silver Donator
Posts: 468
Joined: Sun Dec 31, 2017 1:02 am
Location: Stockholm
Karma: 327
Contact:

Re: France vs Austria Bot Challenge

#10 Post by RoganJosh » Wed Nov 11, 2020 10:49 pm

President Eden's game:

A Bur > Bel in A1902 is suboptimal. Difficult for France after that.
A Pie > Tus in S1903 is also suboptimal. Almost guaranteed to lose Germany.
F NTH > Edi in A1903 is disastrous. Game was over by then.

The orders [A Bur > Mun, A Par > Bur, A Hol > Kie] in A1904 and S1905 also stands out. There is 0 probability of entering Munich unsupported in this position. You have to order Bur > Ruh, the only question is whether you'd support from Holland or not.
2

jasnah
Posts: 45
Joined: Tue Jun 02, 2020 8:13 pm
Karma: 33

Re: France vs Austria Bot Challenge

#11 Post by jasnah » Wed Nov 11, 2020 10:56 pm

Yeah...bot super bad in Eden's game. Similarly lousy in goldfinger's game..Mun-Ber in A02, wtf? Don't need to see more after that.

"possibly reaching the point of being unbeatable by a human"..not in fva at any rate.
1

President Eden
Posts: 548
Joined: Fri Oct 20, 2017 2:11 pm
Karma: 821

Re: France vs Austria Bot Challenge

#12 Post by President Eden » Wed Nov 11, 2020 11:42 pm

FairBot gave me a +1... is this how it starts???
3

Polynoamial
Posts: 5
Joined: Thu Mar 21, 2019 9:04 pm
Karma: 11

Re: France vs Austria Bot Challenge

#13 Post by Polynoamial » Thu Nov 12, 2020 12:24 am

Thanks for the feedback! We fixed a bug in the bot that probably hurt its performance in the first couple hours it was up. I'm curious to see if it's doing better now.
2

goldfinger0303
Site Moderator
Site Moderator
Posts: 1392
Joined: Fri Sep 29, 2017 3:17 pm
Karma: 569

Re: France vs Austria Bot Challenge

#14 Post by goldfinger0303 » Thu Nov 12, 2020 12:25 am

Maybe I'm just bad as France....lost a second time there.

In hindsight, the decision to not build the fleet in 1902 was mind-numbingly bad. But the bots 1902 moves were so much worse. It really only had a chance of coming back because I played too cautiously and tried to be cute with a convoy up to StP. But I was kinda outplaying myself because I was trying to optimize builds based off what a human would do (hence not moving straight to StP.) But like that was a pretty bad 1901 and 1902 from the bot. And going for Tunis there in 1905 prevented the bot from winning that year.

http://www.webdiplomacy.net/board.php?gameID=331874

However, I did win as Austria pretty handily. Lots of errors by the bot there - strategic and tactical. Opening with the fleet to Spain is, competitively, almost never done. That's because the path to victory in the north there is slim if you commit that early and strong to the south. And always protect Burgundy.

http://www.webdiplomacy.net/board.php?gameID=331879
1

Codeword Iroquois
Posts: 1
Joined: Fri Aug 14, 2020 1:35 am
Karma: 1

Re: France vs Austria Bot Challenge

#15 Post by Codeword Iroquois » Thu Nov 12, 2020 12:26 am

So, here's my game...
Gotta say, I did better than I usually do vs. bots.
http://webdiplomacy.net/board.php?gameI ... ocache=908
I really am not in a position to be critiquing the bot, considering I had just as many, if not more, errors.
1

goldfinger0303
Site Moderator
Site Moderator
Posts: 1392
Joined: Fri Sep 29, 2017 3:17 pm
Karma: 569

Re: France vs Austria Bot Challenge

#16 Post by goldfinger0303 » Thu Nov 12, 2020 12:53 am

Getting into specifics for my France game.

I think the opening to Serbia is suboptimal under most situations, but I'm not going to fault it as a "mistake".

I will say the following though
Autumn 1903 - Moscow should have moved to St. Petersburg.
Spring 1904 - Moscow should have moved to St Petersburg.
Spring 1905 - Ionian should have moved to Naples
Spring 1905 - Tyrolia should have moved to Venice


For my Austria game

I already touched upon the southern fleet open, but it is a valid (if suboptimal) strategy. I'll focus on clear mistakes instead of 50/50s.

Autumn 1901 - I'm going to call it a mistake not to move to Munich. Ceding Munich to Austria in 1901 as France almost always means a certain defeat. ESPECIALLY if you don't build two armies to defend Burgundy
Winter 1902 - the fleet build in Marseilles was a big mistake because it was the direct cause of the missed 50/50 that let me into Burgundy.
Autumn 1903 - moving out of Paris is not something a player would do while dislodging Burgundy. I know it was trying to go for the 500 IQ play of bouncing me in Picardy (the open territory) but still....there are moves that one must play safe because the risk of failure means the game is over.
Autumn 1905 - Venice should have been taken. There was no point in defending Ionian as my fleets already outnumbered his.
2

Ezio
Posts: 1244
Joined: Sat Dec 23, 2017 6:54 am
Karma: 153

Re: France vs Austria Bot Challenge

#17 Post by Ezio » Thu Nov 12, 2020 1:00 am

http://webdiplomacy.net/board.php?gameID=331892
Final turn:
A.I had 100% win if they simply moved Tyrolia into Venice and had Rome hold rather than self bouncing. I could have supported Rome into Venice to punish and Win. I did not.
1

amdx
Posts: 2
Joined: Thu Feb 13, 2020 2:50 pm
Karma: 22

Re: France vs Austria Bot Challenge

#18 Post by amdx » Thu Nov 12, 2020 1:28 am

goldfinger0303 wrote:
Thu Nov 12, 2020 12:25 am
Maybe I'm just bad as France....lost a second time there.
Well, the bot doesn't like playing France either :) In a sense that it loses to itself playing France more often than the other way. It seems to be the case in human games too.
goldfinger0303 wrote:
Thu Nov 12, 2020 12:25 am
Opening with the fleet to Spain is, competitively, almost never done. That's because the path to victory in the north there is slim if you commit that early and strong to the south. And always protect Burgundy.
Yeah, this kind of long-term implications is hard for bot to grasp. We'll try to get them in, thanks for poking!
4

tsvi14
Posts: 3
Joined: Sat Mar 21, 2020 1:15 pm
Karma: 4

Re: France vs Austria Bot Challenge

#19 Post by tsvi14 » Thu Nov 12, 2020 1:33 am

Awesome!!

I have one question - a tactic some diplomacy players use, when the games aren't anonymous, is looking into a player's past games on the site and trying to divine some of their past tactical behavior in order to better predict that of the future. Is the bot doing this?
2

Ezio
Posts: 1244
Joined: Sat Dec 23, 2017 6:54 am
Karma: 153

Re: France vs Austria Bot Challenge

#20 Post by Ezio » Thu Nov 12, 2020 3:50 am

http://webdiplomacy.net/board.php?gameI ... ocache=547
Another final move closing out mistake. They could have used Silesia to support Galicia into Warsaw rather than support holding Munich. This would prevent the move sequence Livonia-Warsaw and Gulf of Bothnia-STP from denying the win.
1

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests