Page 1 of 1

Is there a way to export or scrape the chat archive?

Posted: Fri Feb 05, 2021 11:32 pm
by ShadowSpy
Hello,

I just finished a game of Diplomacy with some online friends, and was hoping to export our chat archive and compile all the messages together for posterity. Unfortunately, it seems like there's no easy way to do this--all the chat archive messages are on separate pages.

Is there any existing scripts or an export function that I am not aware of?

Thanks.

Re: Is there a way to export or scrape the chat archive?

Posted: Sat Feb 06, 2021 12:00 am
by Yonni
Nope but if you click on "messages" at the bottom of the page or should condense it to fewer pages and allow you to scroll back further.

Re: Is there a way to export or scrape the chat archive?

Posted: Mon Feb 08, 2021 4:04 am
by Guike
You could definitely scrape the messages archives using the message archive page.

The url of the page looks like this: http://www.webdiplomacy.net/board.php?g ... GE#results

You could very well code a javascript script that parses the page, extracts the messages, fetches the next page by incrementing MESSAGE_PAGE in the url, etc.

You can determine by the contents of the page how many pages you have to crawl, there's a button to see the last page, with the page number as content.

Re: Is there a way to export or scrape the chat archive?

Posted: Mon Feb 08, 2021 12:48 pm
by A_Tin_Can
To compile it nicely, you'd want to include the maps, and probably thread the conversations. If someone wrote a script to do this, it would be very welcome, I'm sure.