Is there a way to export or scrape the chat archive?

Developers and contributors can find a link to our github page and engage in development project planning here.
Post Reply
Message
Author
ShadowSpy
Posts: 1
Joined: Fri Jan 08, 2021 9:24 am
Karma: 1
Contact:

Is there a way to export or scrape the chat archive?

#1 Post by ShadowSpy » Fri Feb 05, 2021 11:32 pm

Hello,

I just finished a game of Diplomacy with some online friends, and was hoping to export our chat archive and compile all the messages together for posterity. Unfortunately, it seems like there's no easy way to do this--all the chat archive messages are on separate pages.

Is there any existing scripts or an export function that I am not aware of?

Thanks.
1

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: Is there a way to export or scrape the chat archive?

#2 Post by Yonni » Sat Feb 06, 2021 12:00 am

Nope but if you click on "messages" at the bottom of the page or should condense it to fewer pages and allow you to scroll back further.
1

Guike
Posts: 1
Joined: Fri Jan 15, 2021 4:44 am
Karma: 1
Contact:

Re: Is there a way to export or scrape the chat archive?

#3 Post by Guike » Mon Feb 08, 2021 4:04 am

You could definitely scrape the messages archives using the message archive page.

The url of the page looks like this: http://www.webdiplomacy.net/board.php?g ... GE#results

You could very well code a javascript script that parses the page, extracts the messages, fetches the next page by incrementing MESSAGE_PAGE in the url, etc.

You can determine by the contents of the page how many pages you have to crawl, there's a button to see the last page, with the page number as content.
1

A_Tin_Can
Lifetime Site Contributor
Posts: 283
Joined: Fri Sep 29, 2017 9:18 pm
Karma: 451
Contact:

Re: Is there a way to export or scrape the chat archive?

#4 Post by A_Tin_Can » Mon Feb 08, 2021 12:48 pm

To compile it nicely, you'd want to include the maps, and probably thread the conversations. If someone wrote a script to do this, it would be very welcome, I'm sure.
2

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests