webdiplomacy.net

Where can we play against true AI Bots? I have read that DeepMind and DipNet have advanced AI diplomacy skills, and can win more than 90% of the time against master level Gunboat players. The bots here are truly awful, they are worse than basic rank beginning humans. I see the most atrocious moves imaginable against our bots. It's child's play to beat them, they are not real AI, of course. So where do the real AI diplomacy players play?

You can play FvA any time against Fairbot, Meta's advanced AI. If you choose Games/Play Against AI/Bot, and select Classic-FvA, you will automatically be playing against Fairbot. Fairbot is not easy to beat. I consider myself an above-average FvA player and my win percentage against Fairbot is about 7%.

WebDip has had some test runs of an advanced Meta AI in 7-player Classic, but I don't think the site offers that on demand at this time.

Our bots do not even understand the objectives of the game, they are that bad. The objective is a) to solo, b) if unable to solo, then to share in the final draw, which means an AI player has to recognize the long term threat of another player solo, and set up stalemate lines to stop it. Our bots are totally unable to do this.

Example from one of my games. I was Italy, France was dead, Russia was overrunning Turkey and Austria, England had lost all of his home SC, but held Brest. I, as Italy. held most of France. I can see the arrival long term of a Russian solo, so I set up a rock solid perfect stalemate line at Venice and Ionian Sea. It was absolutely impenetrable, until England poached Portugal from me, //even though he had ZERO ability to build a new unit, because he had no home SC left// This forced a disband and I was unable to hold the line after that. So, apparently, the bots are programed to grab any SC in view, regardless of the consequences. England was directly helping Russia solo.

That is pretty sad.

I have seen hundreds of idiotic bot moves. Like the Austrian player, opening fleet to Trieste... an experienced human player would never ever open gunboat like that.

I've also seen, on a spring move, the German take Warsaw, and then on the fall turn, a Russian army in Wales, sitting next to German owned London, totally undefended. And he lets Germany take Warsaw, but fails to walk into London, then he disbands Wales... ???? It's like they decide who they like on turn one, then /never/ change their minds, no matter what.

I have even seen lots of misorders, where a bot supports the wrong directional move of one of his own units. Actual basic misorders, I mean come on...

Those France vs Austria style games are not the real game obviously. Bots should do well there, it's like a different version of checkers or chess. Where bots need vast improvement is at analyzing the long term effects of their actions. If as Italy, I weaken Austria or draw his ire towards me, do I hand massive power to Turkey or Russia? Good humans players consider these sorts of things all the time, AI needs to as well.

Tugster wrote: ↑

Tue Aug 09, 2022 8:42 pm

Those France vs Austria style games are not the real game obviously. Bots should do well there, it's like a different version of checkers or chess.

I agree, although the right way to describe FvA might be as a far, far more complicated version of chess. Orders of magnitude more possibilities, so many that traditional computer programs can see maybe 2-3 moves ahead, compared with maybe 6-8 in chess.

Where bots need vast improvement is at analyzing the long term effects of their actions. If as Italy, I weaken Austria or draw his ire towards me, do I hand massive power to Turkey or Russia? Good humans players consider these sorts of things all the time, AI needs to as well.

My understanding is, even the strongest AI can't really do this yet in Diplomacy. So the AI teams have to come up with substitute ways to achieve the desired effects of making the bot a strong player. It's like in the movie Imitation Game, Alan Turing says, "Machines are different; hence, they think differently."

And the advanced AI bots are having some success. In addition to Fairbot, in the last year or so, Meta has been developing advanced AI that can hold their own, or better, in 7-player, Classic gunboat games against good human players. Webdip quietly gave a couple of them player IDs last year and let them play. One was "yolo5"; another was "Fairplay". You can look up their results. They scored quite a bit better than average human players. And yes, they also made some foolish moves in the endgame occasionally, attacking the player holding the defensive line against the 15-center power.

There was also the Meta speedboat tournament earlier this year. Classic gunboat, 5-minute turns, with a hard game-end at a variable season decided on in advance, but not announced. There were maybe 100 of these games, each with 6 human WebDippers and the bot. The bot was called "Jon15". All of its games are on Webdip, too. I think it may have had the highest score of all players before they tweaked its programming.

In the first phase of the Meta speedboat tournament, the bot played under its "yolo5" username. One participant kept a meticulous log of results and the bot outperformed every human in the tournament. It scored ~27% of all points available in the 100 game series compared to the top human's ~23%. There were no ghost ranking requirements or other barriers of entry for human players in the tournament. It was an impressive achievement by the bot, but not chess bot levels of impressive.

In the second phase of the tournament, different bot(s) from different developer(s) were swapped in, presumably as a control group, now playing under the "jon15" username. Recordkeeping was not as thorough at this point. In my opinion, "jon15" couldn't hold a candle to "yolo5". For starters, it had a weird affinity for opening to Armenia and Silesia as Russia.

90kicvesb wrote: ↑

Wed Aug 10, 2022 5:49 am

In the first phase of the Meta speedboat tournament, the bot played under its "yolo5" username. One participant kept a meticulous log of results and the bot outperformed every human in the tournament. It scored ~27% of all points available in the 100 game series compared to the top human's ~23%. There were no ghost ranking requirements or other barriers of entry for human players in the tournament. It was an impressive achievement by the bot, but not chess bot levels of impressive.

Did yolo5 play each power an equal number of times during the 100 games? Were those results published somewhere (either here on the forum or elsewhere)?

yolo5 is a registered webdip user. You can do a user search and see all of its games. The Meta speedboat games all have the name "Meta Speedboat Tournament-" and the game number. I'm not sure how many times the bot played each power.

Trigfea63 wrote: ↑

Wed Aug 10, 2022 11:26 pm

yolo5 is a registered webdip user. You can do a user search and see all of its games. The Meta speedboat games all have the name "Meta Speedboat Tournament-" and the game number. I'm not sure how many times the bot played each power.

True, but I was trying to take the lazy way out.

Bot games really fun but developers forgot to program bots to realize 18 centers = a win (stop ‘any’ country from getting) and developers didnt know enough about faulty game play for italy to attack austria 80% of the time

In my latest bot game, where I was England, Germany was down to just one SC (Kiel), but disbanded the Kiel army, kept its North Sea fleet, and decided to run around stealing my home SCs while France and I were desperately trying to stop an Austrian solo. Very frustrating.

https://webdiplomacy.net/beta/?gameID=705843#gamePanel

Bladerunners wrote: ↑

Wed Apr 26, 2023 12:11 am

Bot games really fun but developers forgot to program bots to realize 18 centers = a win (stop ‘any’ country from getting) and developers didnt know enough about faulty game play for italy to attack austria 80% of the time

I feel like there are certain player-nations that the bots tend to attack far more often than when those nations are other bots. I see Russia moving Mos-St.P nearly every time I play England, and whenever I play France it seems Italy always wants to move to the Piedmont. I hardly ever see bots play those openings when England or France are also bots.

hewwo gamers wrote: ↑

Tue Oct 03, 2023 8:35 pm

Bladerunners wrote: ↑

Wed Apr 26, 2023 12:11 am

Bot games really fun but developers forgot to program bots to realize 18 centers = a win (stop ‘any’ country from getting) and developers didnt know enough about faulty game play for italy to attack austria 80% of the time

I feel like there are certain player-nations that the bots tend to attack far more often than when those nations are other bots. I see Russia moving Mos-St.P nearly every time I play England, and whenever I play France it seems Italy always wants to move to the Piedmont. I hardly ever see bots play those openings when England or France are also bots.

How many bot games have you played as England? I have several hundred games worth of Russia S01 bot move analysis if you'd like to see it, I just have to update it and from the last time I ran it about 100 games ago...

I'd be interested in seeing your statistics.
I had exactly the same impression about the Mos-StP opening, which seemed to amount up to 50% when I draw England (out of a very small sample of max 10 games as England, which I used to play on play.webdiplomacy.org, i.e. the games haven't been archived).
Too small of a sample, I know, but that opening rarely happened when I drew the other 5 powers.

GuineaPig wrote: ↑

Fri Oct 06, 2023 10:35 am

I'd be interested in seeing your statistics.
I had exactly the same impression about the Mos-StP opening, which seemed to amount up to 50% when I draw England (out of a very small sample of max 10 games as England, which I used to play on play.webdiplomacy.org, i.e. the games haven't been archived).
Too small of a sample, I know, but that opening rarely happened when I drew the other 5 powers.

I just finished bot game #455 today (65 as each power). I will try to regenerate the the opening move stats sometime in the next couple of days.

Continuing to love playing Bot games but wish the developers 1) had some deep diplomacy history and 2) knew that 18 centers wins a game My #1 adv9ce for playing bots is to always immediately cancel your game if italy attacks Austria outright as this rarely happens in real games - live or online or bots ... although I very much doubt that the bots are programmed to learn anything...

Bladerunners wrote: ↑

Sun Nov 12, 2023 11:25 am

Continuing to love playing Bot games but wish the developers 1) had some deep diplomacy history and 2) knew that 18 centers wins a game My #1 adv9ce for playing bots is to always immediately cancel your game if italy attacks Austria outright as this rarely happens in real games - live or online or bots ... although I very much doubt that the bots are programmed to learn anything...

Yes, most of the time playing as Austria and having bot Italy attack you is curtains, because R or T does too, but, every now and then, something like this happens...

https://webdiplomacy.net/board.php?gameID=801488