France vs Austria Bot Challenge

Forum rules
This forum is limited to topics relating to the game Diplomacy only. Other posts or topics will be relocated to the correct forum category or deleted. Please be respectful and follow our normal site rules at http://www.webdiplomacy.net/rules.php.

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:points: :-D :eyeroll: :neutral: :nmr: :razz: :raging: :-) ;) :( :sick: :o :? 8-) :x :shock: :lol: :cry: :evil: :?: :smirk: :!:
View more smilies

BBCode is ON
[img] is ON
[flash] is OFF
[url] is OFF
Smilies are ON

Topic review
   

If you wish to attach one or more files enter the details below.

Expand view Topic review: France vs Austria Bot Challenge

Re: France vs Austria Bot Challenge

by MerryBBuck » Wed Feb 02, 2022 8:38 pm

I think Fairbot (FvA) is broken. It's not moving, and is starting to resign games.
https://webdiplomacy.net/userprofile.php?userID=128943

Re: France vs Austria Bot Challenge

by Puddle » Fri Dec 17, 2021 8:45 pm

My second victory against FairBot today!

https://webdiplomacy.net/board.php?gameID=387564

Re: France vs Austria Bot Challenge

by MerryBBuck » Thu Oct 14, 2021 12:33 am

Jamiet99uk wrote:
Wed Oct 13, 2021 6:21 pm

How does it "rely" on it?
I just meant that the bot is coded/trained that that order results in the army moving. If the bot played on a different website, it might lose games that it would win here.

Re: France vs Austria Bot Challenge

by Jamiet99uk » Wed Oct 13, 2021 6:21 pm

MerryBBuck wrote:
Tue Apr 20, 2021 7:07 pm
Today I learned a DATC rule interpretation on which Fairbot occasionally relies.
Example: Autumn 1901, Fairbot as France https://webdiplomacy.net/board.php?gameID=349632
  • The army at Picardy move to Belgium via convoy.
There's no fleet for this convoy. But per DATC 6.G.8, "Explicit Convoy that Isn't There" http://web.inter.nl.net/users/L.B.Kruijswijk/#6.G.8, the "via convoy" is ignored, and the army moves by land.
How does it "rely" on it?

Re: France vs Austria Bot Challenge

by MerryBBuck » Thu Oct 07, 2021 11:15 pm

Re: France vs Austria Bot Challenge

by MerryBBuck » Sat Sep 04, 2021 2:23 am

I figured out enough about Python and Keras to scrape game data and create models to predict moves. I trained one set of models to predict Fairbot's moves as France. I played a game where my "bot" was France, versus the real Fairbot as Austria: https://webdiplomacy.net/board.php?gameID=373463

I know my efforts are like a caveman throwing rocks at a starship.

Re: France vs Austria Bot Challenge

by Peregrine Falcon » Thu Apr 22, 2021 5:44 am

That is an odd quirk of the bot! Good find!

And to add to what RoganJosh said, the DATC is also just a series of things to decide upon based on multiple conflicting iterations of the rules. (Although conforming to the most recent rules, published in 2000 [the 2015 rulebook doesn't contain any wording changes], is typically what people mean when they say DATC compliance.)

Re: France vs Austria Bot Challenge

by RoganJosh » Tue Apr 20, 2021 8:58 pm

DATC is not the rules. On Conspiracy, that move order would fail.

(Though, by the official rules, the move should succeed.)

Re: France vs Austria Bot Challenge

by MerryBBuck » Tue Apr 20, 2021 7:07 pm

Today I learned a DATC rule interpretation on which Fairbot occasionally relies.
Example: Autumn 1901, Fairbot as France https://webdiplomacy.net/board.php?gameID=349632
  • The army at Picardy move to Belgium via convoy.
There's no fleet for this convoy. But per DATC 6.G.8, "Explicit Convoy that Isn't There" http://web.inter.nl.net/users/L.B.Kruijswijk/#6.G.8, the "via convoy" is ignored, and the army moves by land.

Re: France vs Austria Bot Challenge

by Tjappator » Fri Apr 16, 2021 12:03 pm

MerryBBuck wrote:
Sun Mar 28, 2021 4:17 pm
Congrats to Myth1202 for the victory as France against Fairbot: http://webdiplomacy.net/board.php?gameI ... #gamePanel
I believe that the Austria bot might have made a mistake in Fall 1902, when it disbanded the army in Ven instead of retreating it to Tus or Apu (probably Apu would have been the best)? I'm not sure what the benefit is of building a new army in Bud instead of a retreat.
Especially since the Bot then selfbounced in Spring 1903 instead of taking an SC in Ser/Rum.

Re: France vs Austria Bot Challenge

by MerryBBuck » Thu Apr 15, 2021 5:02 pm

MerryBBuck wrote:
Fri Mar 05, 2021 4:13 pm
I wonder if anyone who has the "webdiplomacy dataset" has shared the historical FvA win/lose/draw split.
Answering my own question for posterity. In ~19400 FvA games, Austria won 53.7% of the time, France won 39.7% of the time, and 6.7% were draws.

Re: France vs Austria Bot Challenge

by jasnah » Sun Mar 28, 2021 4:51 pm

MerryBBuck wrote:
Sun Mar 28, 2021 4:17 pm
Congrats to Myth1202 for the victory as France against Fairbot: http://webdiplomacy.net/board.php?gameI ... #gamePanel
A remarkable and innovative victory, don't think I've ever seen anything quite like it.

Re: France vs Austria Bot Challenge

by MerryBBuck » Sun Mar 28, 2021 4:17 pm

Congrats to Myth1202 for the victory as France against Fairbot: http://webdiplomacy.net/board.php?gameI ... #gamePanel

Re: France vs Austria Bot Challenge

by RoganJosh » Fri Mar 05, 2021 6:40 pm

Yeah, I should've written something like "net 3 guesses to win" in stead of .875... Also, not all guesses are 50/50.

.75 is "net 2 guesses to win", and .50 is "net 1 guess to win".

Re: France vs Austria Bot Challenge

by jasnah » Fri Mar 05, 2021 5:26 pm

People don't like to play messy positions with lots of uncertainty. The bot doesn't give a shit, so it doesn't ever do the usual hedging moves that people are prone to when playing Austria, which somewhat reduce tactical efficiency while simplifying the position and make it easier for France to catch up. No way around it, to beat the bot as France requires getting every guess right.

And so yeah, 87.2% is oddly specific, but like somewhere around that to 15%-ish France without a simplification bias sounds about right.

Re: France vs Austria Bot Challenge

by RoganJosh » Fri Mar 05, 2021 5:08 pm

I think it's generally agreed upon that Austria has an advantage. That's why most tournaments has France progressing if the tiebreaker game ends in a draw. Nothing new there.

In addition, Austria is much easier to play. France has a narrower path to victory. As France, you must win all three fronts (Tunis, Germany, and Scandinavia) to win the game. Against a good Austrian player, the only way to win all three fronts is to win each front narrowly. So you have to constantly react to what choices Austria is making, and there are few good options, easy to misstep. Austria has so many different options that it can be difficult to blunder as Austria.

The bot is very good at closing out the game early as Austria. That's where the bot is better than humans, I'd say. And it has made me think that the map is more unbalanced than what I originally though. Maybe more like 87.2% A than 75% A. Maybe even worse.

Re: France vs Austria Bot Challenge

by MerryBBuck » Fri Mar 05, 2021 4:13 pm

I defer to RJ. I lack the perspective to give a useful answer. Apart from 3 aborted attempts, I've never played FvA against a human. I fear that my "expertise" is not actually in FvA. Instead, I think fairbot itself has simply trained me in ways to exploit fairbot.

At the moment, I find it lots easier to beat fairbot when I play as Austria. My intuition does says Austria has a moderate advantage in FvA. Googling, I've seen 2 datapoints that might support this: a 2018 tournament result (53% Austria win vs. 47% France), and RJ's Tableau database (a 57-to-43 "advantage" for Austria after 1 year). I'd love to hear others answer this question. I wonder if anyone who has the "webdiplomacy dataset" has shared the historical FvA win/lose/draw split.
https://www.playdiplomacy.com/forum/vie ... 45&t=58794
https://public.tableau.com/profile/jens ... ublish=yes

Re: France vs Austria Bot Challenge

by Doug7878 » Fri Mar 05, 2021 12:21 pm

A little update - MBB and RJ have established themselves as the top 2 AvF players against Fairbot ("FB").

RJ has played as France many times with a win rate of approximately 20%.
MBB, who has a similar win rate as France, has just played as Austria a number of times, and has beaten FB in 4 of the last five games (80%).

So I ask RJ and MBB (who have proven some level of expertise here) ...
Given equal opponents, does France win only 20%, the remainder being maybe 70-75% Austria wins, and 5-10% draws ?

Re: France vs Austria Bot Challenge

by MerryBBuck » Mon Mar 01, 2021 3:27 am

I have noticed ~ 4 points in time when fairbot's mix of opening moves changed. E.G. as Austria, fairbot used to love opening to RUM, but lately only opens to GAL. But maybe fairbot's opening turn(s) are just based on a tree of the "best" moves, with a human occasionally tweaking the odds of taking each branch.

Re: France vs Austria Bot Challenge

by Doug7878 » Mon Mar 01, 2021 2:46 am

There's no doubt that the bot's play as Austria was improved several times.

At the time of gameIDs in the 333378-334651 range, the bot lost 23 of 98 games as Austria. We humans were doing pretty well.

There must have been an improvement at this point. Subsequently, there was a period of very few losses, followed by period of scattered losses only to very good players. (11 losses out of 189 games)

When we get to game IDs 336239-339549 there begin to be more losses - 46 out of 320 games. I would guess that this is due to human players adjusting to the bot's new and improved style of play.

At some point soon after game 339549, the bot must have been upgraded again - it loses infrequently after this - only 20 times of the next 332 games.

So, it appears from the patterns of losses, that there were upgrades at certain points, each following increased numbers of losses by Fairbot. This may be wrong (appearances can be deceiving) but this is what I see FWIW.

Top