ChatGPT and Diplomacy

[echidna]

I do not recommend relying upon ChatGTP's understanding of Diplomacy. I asked "in the board game diplomacy can an army move from Norway to Sweden?"

ChatGPT said:
In the board game Diplomacy, an army can move from Norway to Sweden.

Why it's allowed:
Norway and Sweden do share a land border.

It’s only one-way for armies, though — this is a quirk of the map design.

Summary of movement:
Direction Army Move Legal? Why?
Sweden → Norway No No land border from Sweden into Norway
Norway → Sweden Yes Norway has a land border into Sweden

This may seem asymmetric — and it is! It’s an intentional feature of the game map.

How this works:
Norway has a land border with Sweden, so armies in Norway can move into Sweden.

Sweden, however, does not have a land route into Norway — only a coastal connection (used by fleets).

This is one of a few “weird” map quirks in Diplomacy — others include Bulgaria’s coast split, or Spain’s dual coasts.

[montalvo]

One thing about ChatGPT and other large language models that most people seem to miss is that there is no legitimate 'intelligence' or reasoning behind the response the model gives you. Large language models work by converting your prompt into a mathematical representation and then calculating the most likely response you may wish to see to that prompt. Words are chosen based on their likelihood to appear next in the current string of words the model is printing out.

[Jamiet99uk]

Damn those Norwegian border guards are tough.

[Jamiet99uk]

Strange thing is, I just posted the same question into ChatGPT as a follow-up question *after* first asking "Do you know the rules of the board game, Diplomacy?".

ChatGPT answered that it did know the rules, and proceeded to list off some of the basic concepts (correctly).

I then asked echidna's question, to which the answer I received was:

"Yes, an army can move from Norway to Sweden—they are adjacent land territories, so it's a legal move for an army.

"Important Notes:
- Norway and Sweden are both land provinces.

- The move does not require a convoy—armies can walk directly from Norway to Sweden (or vice versa).

- If another unit is moving into Sweden, or a unit is already there, the usual support and conflict rules apply.

"Let me know if you want to explore how this interacts with support, fleets in adjacent seas (like Skagerrak or Baltic), or similar situations."

I wonder whether asking the basic "do you know the rules?" question first caused Chat GPT to be more likely to answer the second question correctly...

[Jamiet99uk]

I then tried adjudicator Test Case 6.A.5 (from: https://webdiplomacy.net/doc/DATC_v3_0.html#6)

The question (exactly as I gave it to Chat GPT) is:

What happens if the following orders are entered?

England:
F North Sea Convoys A Yorkshire - Yorkshire
A Yorkshire - Yorkshire
A Liverpool Supports A Yorkshire - Yorkshire

Germany:
F London - Yorkshire
A Wales Supports F London - Yorkshire

Chat GPT answered:

"Great question—this involves a nuanced interaction of illegal orders, convoy mechanics, and conflicts. Let’s break it down carefully:

1. English Orders Are Invalid:
- A Yorkshire → Yorkshire: Units cannot move to their own space. Invalid.
- Convoying a unit to its own location: Also invalid.
- Supporting an invalid move (Liverpool) is itself invalid.

So all of England's orders fail or are void.

2. Germany’s Orders Are Valid:
- F London → Yorkshire: Legal.
- Support from A Wales: Legal and adds support strength to the move.

Result: Germany dislodges the English army in Yorkshire."

This is the exactly correct answer to adjudication test case 6.A.5.

So Chat GPT is at least capable of getting the right answer in quite a complex case.

[miminena]

Strange thing is, I just posted the same question into ChatGPT as a follow-up question *after* first asking "Do you know the rules of the board game, Diplomacy?".

ChatGPT answered that it did know the rules, and proceeded to list off some of the basic concepts (correctly).

I then asked echidna's question, to which the answer I received was:

"Yes, an army can move from Norway to Sweden—they are adjacent land territories, so it's a legal move for an army.

"Important Notes:
- Norway and Sweden are both land provinces.

- The move does not require a convoy—armies can walk directly from Norway to Sweden (or vice versa).

- If another unit is moving into Sweden, or a unit is already there, the usual support and conflict rules apply.

"Let me know if you want to explore how this interacts with support, fleets in adjacent seas (like Skagerrak or Baltic), or similar situations."

I wonder whether asking the basic "do you know the rules?" question first caused Chat GPT to be more likely to answer the second question correctly...

did you try asking it before asking if it knew the rules? i'll ask it rq about movement, then if it knows the rules, then about movement again

also there should be a gpt diplomacy variant that's just regular diplomacy but an army in sweden can't move to norway but a fleet can

[miminena]

without asking whether it knows the rules:

In the board game **Diplomacy**, **an army cannot move directly from Norway to Sweden**.

Here’s why:

* **Norway (Nor)** and **Sweden (Swe)** are **not directly connected by land** on the standard map.
* **Norway** is only connected by land to **Finland** (via the St. Petersburg north coast), and otherwise bordered by sea.
* **Sweden** is connected by land to **Finland** and **Denmark** (via Copenhagen over the Denmark-Sweden strait), but **not** directly to Norway.

### How could an army get from Norway to Sweden?

To move an army from Norway to Sweden, you would need to:

1. **Use a fleet to convoy** the army:

* Have a fleet in the **Norwegian Sea** or **North Sea**, for example.
* Convoy the army from Norway to Sweden (assuming the fleet can reach both coasts).

2. **Move indirectly via Finland**:

* First move the army from **Norway to Finland (St. Petersburg north coast)**.
* Then move from **Finland to Sweden**.

So, **direct land movement from Norway to Sweden is not allowed**, but you can still reach Sweden through **a convoy** or **indirect land movement via Finland**.

Let me know if you’d like a visual of the map or help planning a move!


%%%%%%%%


after asking:

Yes, **an army can move from Norway to Sweden** in *Diplomacy*.

### Here's why:

* **Norway and Sweden are adjacent provinces on land**, and
* Both can be occupied by **armies** (as opposed to just fleets or coastal territories).

So, if you have an **army in Norway**, you can issue a standard **Move** order like:

> **Norway → Sweden**

---

### Reminder:

* If another unit is trying to enter Sweden at the same time, or if there's a unit in Sweden **holding or supported**, the move could **fail due to a standoff** or be **defeated** depending on supports.
* Fleets can also move between these areas, but if you're using an **army**, land adjacency is all that matters here.

Let me know if you want help with move orders or potential conflicts in that region.

which corroborates jamie's theory. i wonder why that happens?

[echidna]

montalvo - thanks for describing that key characteristic of large language models.

Jamiet99uk - thanks for posting the same question to ChatGTP as a follow-up. Indeed, the question I posted was also a follow-up to a path question I had asked that got an incorrect answer, so I was digging in to try to find out why.
I now tried to recreate the path question I had asked previously (probably not word for word), this time getting different (but still problematic) path results:

My question to ChatGTP:
In the board game Diplomacy what is the longest path an army can take
without convoys and without going through the same province twice?

ChatGTP's result (omitting some introductory text, and omitting blank lines):
Known Result:
The longest known land-only path an army can make on the standard Diplomacy map is:
38 moves (i.e., 39 provinces visited, no repeats).
Example of such a path (abbreviated, one known variant):
St Petersburg
Moscow
Warsaw
Galicia
Vienna
Bohemia
Silesia
Prussia
Berlin
Munich
Tyrolia
Trieste
Albania
Serbia
Rumania
Bulgaria
Constantinople
Smyrna
Ankara
Armenia
Sevastopol
Ukraine
Livonia
Finland
Norway
Sweden
Denmark
Kiel
Ruhr
Holland
Belgium
Burgundy
Paris
Gascony
Spain
Portugal
Marseilles
Piedmont
Tuscany
This list visits 39 provinces in 38 moves.

-----------------
There are at least two problems with this route:
(1)
Why not go from Constantinople to Ankara to Smyrna to Syria to Armenia so as to also include Syria?
I asked ChatGTP this, to which it confirmed my correction, saying in part:
"Excellent observation — and yes, you're absolutely right to flag Syria as a potential extension point in the path. Let's dig into this carefully."

(2)
How was it possible to go from Portugal directly to Marseilles?
I asked ChatGTP this, to which it confirmed my correction, saying in part:
"Great catch — and you're absolutely right to question that. Let's examine it."
reat catch — and you're absolutely right to question that. Let's examine it."

I have to say that ChatGTP is very polite in its updates, much like some of my most feared opponents when they explain why it was necessary to revise their previous statements and instead enter several of my supply centers.

[Esquire Bertissimmo]

I wonder whether asking the basic "do you know the rules?" question first caused Chat GPT to be more likely to answer the second question correctly...

Yes, ChatGPT is much more likely to produce accurate answers if doesn't come to an issue cold. Getting into a conversation about the game, asking it to web search rules, feeding it a PDF of the rulebook, etc. will greatly improve its accuracy in my experience.

Below is a snippet of ChatGPT's own response to seeing this thread included the following advice, which corresponds with my experience:

The Core Issue: Context Sensitivity and Probabilistic Responses
LLMs don’t "know" facts in a traditional sense. Instead, they generate text based on probabilities learned from massive datasets. That makes them:

Context-sensitive: Earlier parts of the conversation shape later answers.

Phrasing-dependent: How you ask affects what kind of completion you get.

Vulnerable to plausible-sounding errors: Especially when trained on inconsistent or conflicting source material.

How to Get Better Results with LLMs for Games Like Diplomacy
Ask setup questions first (“Do you know the rules of Diplomacy?” or “What are the land-adjacent provinces for Sweden?”).

Use structured questions — asking about specific map relationships or rule clauses rather than general movement.

Cross-verify with external sources (like the DATC tests or official rulebooks).

Use specialized adjudicators for rule-heavy games — ChatGPT is best used for planning, narrative, or strategy advice, not adjudication.

[CocoHM2011]

I asked ChatGPT to rank the nations best to worst to play as.

This is how ChatGPT ranked them:

1: France
Why? It has many early centres and expansion routes. Allying Italy is beneficial and it is easy to put England and Germany against each other.

2: Turkey.
Why? Advantage of the corner, higher chance of Black Sea than Russia, and very difficult to invade.

3: Russia.
Why? The only one to start with four centres, and can invade Germany and Austria easily. Sometimes will get teamed on early.

4: England.
Why? Very slow start, but has control of the sea. Expansion is possible if you have allies.

5: Germany.
Why? Lots of places to expand into and supply centres, but you also face a lot of enemies. High risk high reward.

6: Austria.
Why? At the centre of everything and has barely any naval access. Has lots of centres to expand to, but will die quickly if not diplomatic.

7: Italy
Why? Very difficult to expand, centre of the Mediterranean, and although it has good defence, Turkey will invade eventually and you won’t win.

[Axm Donrac]

I asked GPT on a different website and it said this:
1. France

Strengths: Secure starting position with good access to both land and sea territories. Strong potential for growth in Western Europe.
Challenges: Needs to manage potential conflicts with England and Germany.
2. Germany

Strengths: Central position allows for flexible alliances and expansion opportunities. Strong initial military presence.
Challenges: Vulnerable to being attacked from multiple sides; requires careful diplomacy.
3. Russia

Strengths: Starts with four units, more than any other country. Large territory provides strategic depth.
Challenges: Needs to manage both European and Asian fronts; vulnerable to early-game alliances against it.
4. England

Strengths: Insular position makes it difficult to invade. Strong naval potential.
Challenges: Limited initial expansion options require effective diplomacy with France and Germany.
5. Turkey

Strengths: Secure home territory with defensible position. Potential for strong alliances with Russia or Austria.
Challenges: Limited initial expansion options; needs to break out of its corner position.
6. Austria

Strengths: Central position allows for various strategic options. Potential for strong alliances with Italy or Germany.
Challenges: Vulnerable to early-game elimination due to central location and multiple neighboring threats.
7. Italy

Strengths: Potential for growth in the Mediterranean. Can capitalize on conflicts between other powers.
Challenges: Limited initial options for expansion; often overlooked in alliances.