webDiplomacy.net new server

Site announcements from the moderator team go here.
Message
Author
User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

webDiplomacy.net new server

#1 Post by kestasjk » Fri Jun 11, 2021 8:08 am

Hi all,

We have moved to a new server; the last one was several years old and was struggling with new features and the database size. We also are hoping to be able to add exciting features which require more server resources, make use of modern container-based deployment to encourage new developers, and to integrate the new AI bots into the system.

But as with any major change there are bound to be a few problems, so if anything seems to have broken please let us know.


A good time to recognise those who have contributed to webDiplomacy.net; whether donating to keep the server up and free of obnoxious ads, making improvements to the software, or patiently keeping cheaters out and answering support requests: 15 years is a long time in web-years and I think we’ve exposed a lot of new people to the hobby.

Regards,
Kestas

Edit: Looks like there was an issue with map rendering but no other errors. I've resolved that and turned game processing back on.
42

jmo1121109
Lifetime Site Contributor
Posts: 1099
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2944
Contact:

Re: webDiplomacy.net new server

#2 Post by jmo1121109 » Fri Jun 11, 2021 6:01 pm

Thanks for the upgrade Kestas!
8

cdngooner
Bronze Donator
Bronze Donator
Posts: 865
Joined: Sun Mar 15, 2020 11:52 pm
Karma: 313
Contact:

Re: webDiplomacy.net new server

#3 Post by cdngooner » Sat Jun 12, 2021 12:56 pm

Seems extra slow loading lately, compared to its old self and to other websites. Something with the new system?
4

User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

Re: webDiplomacy.net new server

#4 Post by kestasjk » Sat Jun 12, 2021 1:35 pm

Thanks for the info. When you say lately how lately are you talking about? Is there anything in particular that is repeatably slow? Does it vary throughout the day or is it fairly consistently slow?

I've found it's a fairly common issue people have had that queries that used to run fast on older MySQL versions have become slower after upgrades. A query might work fine, then there's an optimizer change and it performs terribly.


We have identified a few issues with variant maps and some other bits and pieces; will post here as I go.
2

Gowron
Posts: 14
Joined: Tue Apr 13, 2021 4:45 pm
Karma: 6
Contact:

Re: webDiplomacy.net new server

#5 Post by Gowron » Sat Jun 12, 2021 6:59 pm

No pages would load about 20 or 30 minutes ago.
2

cdngooner
Bronze Donator
Bronze Donator
Posts: 865
Joined: Sun Mar 15, 2020 11:52 pm
Karma: 313
Contact:

Re: webDiplomacy.net new server

#6 Post by cdngooner » Sat Jun 12, 2021 7:02 pm

Its been hit or miss for about 2 days now.
2

User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

Re: webDiplomacy.net new server

#7 Post by kestasjk » Sat Jun 12, 2021 7:31 pm

A while ago while tuning the database for the new system I did accidentally take the database down for a few minutes, so I think that's all that was.

There are logs whenever there is a failure, and I have also enabled logging for slow queries. I made some config changes to make the system use the increased memory space and disk speed, plus a few optimizer recommendations.

Let me know if things remain sluggish. It would be especially helpful if you could record the time, the page you viewed, and copy and paste the data at the bottom of the page that looks like this:

Code: Select all

Rendered in: 0.01sec - Data retrievals: 112 - Data insertions: 9 - Memory used: 0.545MB

Logged on: 77 - Playing: 1292 - Registered: 133889 - Pages served: 310854973
Starting games: 38 - Joinable games: 0 - Active games: 514 - Finished games: 168188

webDiplomacy version 1.66
Enable Desktop Mode
(Note you don't see this at the bottom of the forum, only non-forum pages)

The render times etc and the query numbers, plus the time, will let me look up whether there were any slow queries going on at the time.

We are still experiencing a few deadlocks every now and then (every time an error occurs it is logged, so we should at least know if things break), but a lot less regularly than on the old server.
2

Gowron
Posts: 14
Joined: Tue Apr 13, 2021 4:45 pm
Karma: 6
Contact:

Re: webDiplomacy.net new server

#8 Post by Gowron » Sat Jun 12, 2021 7:47 pm

All pages would not load at all 12 minutes ago.
2

User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

Re: webDiplomacy.net new server

#9 Post by kestasjk » Sat Jun 12, 2021 10:15 pm

I just sorted the last known issue, and had to wipe the map/order cache as maps/orders were being generated with an inconsistent database state causing javascript errors. Might be a bit slow while it regenerates.

Keep letting me know when you experience issues; looking at the logs I had rebooted the database to apply a config change at that time. Let's wait and see how we go from here forward; I'm collecting any errors or slow queries that occur, and all the causes of previous issues are resolved.


I expect we'll need to do some more tuning of the database. It looks like MySQL's internals have changed quite a lot in the latest couple of versions, so that queries that used to function well might now be locking huge sections of the database and causing deadlocks etc.
3

Wusti
Posts: 399
Joined: Thu Oct 19, 2017 10:12 pm
Karma: 232
Contact:

Re: webDiplomacy.net new server

#10 Post by Wusti » Sun Jun 13, 2021 1:13 am

Can you do us a favour? Can you give a longer recovery time for order placement when you put it back up? Some games got hit by missed turns when it came back up. Cheers.
1

User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

Re: webDiplomacy.net new server

#11 Post by kestasjk » Sun Jun 13, 2021 2:34 am

Yep fair point; the downtime was 3-4 hours and I think I added 12? Should have done 24.
3

teccles
Posts: 237
Joined: Mon Mar 25, 2019 8:10 pm
Karma: 285
Contact:

Re: webDiplomacy.net new server

#12 Post by teccles » Wed Jun 16, 2021 6:52 pm

You may well already be aware, but just in case: the website seems to intermittently very slow. Maybe once or twice a day since the server move, I've been experiencing very long load times (maybe a minute?), on an otherwise good internet connection.
9

User avatar
Chaqa
Bronze Donator
Bronze Donator
Posts: 13227
Joined: Fri Sep 29, 2017 7:33 pm
Location: Allentown, PA, USA
Karma: 7983
Contact:

Re: webDiplomacy.net new server

#13 Post by Chaqa » Wed Jun 16, 2021 7:06 pm

teccles wrote:
Wed Jun 16, 2021 6:52 pm
You may well already be aware, but just in case: the website seems to intermittently very slow. Maybe once or twice a day since the server move, I've been experiencing very long load times (maybe a minute?), on an otherwise good internet connection.
Same issue. I know Eden has it as wwell, site seems to go down sometimes
4

jmo1121109
Lifetime Site Contributor
Posts: 1099
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2944
Contact:

Re: webDiplomacy.net new server

#14 Post by jmo1121109 » Wed Jun 16, 2021 7:50 pm

Chaqa wrote:
Wed Jun 16, 2021 7:06 pm
teccles wrote:
Wed Jun 16, 2021 6:52 pm
You may well already be aware, but just in case: the website seems to intermittently very slow. Maybe once or twice a day since the server move, I've been experiencing very long load times (maybe a minute?), on an otherwise good internet connection.
Same issue. I know Eden has it as wwell, site seems to go down sometimes
Yes, I am also having this issue and not seeing any direct errors tied to it when I've experienced it.
5

Changez
Posts: 2
Joined: Sat Jun 19, 2021 10:34 pm
Karma: 4
Contact:

Re: webDiplomacy.net new server

#15 Post by Changez » Sat Jun 19, 2021 10:36 pm

Hey Mods, my game against bots is totally stuck and gives me this message at the top of the screen:

"The last process time was over 12 minutes ago (at Saturday, 19-Jun-21 16:39:06 UTC UTC+0); the server is not processing games until the cause is found and games are given extra time."

Don't know if this is related or if it's an update of some kind.

Thanks
2

User avatar
kestasjk
Developer
Developer
Posts: 518
Joined: Tue Mar 14, 2017 8:13 pm
Location: Perth, Australia
Karma: 775
Contact:

Re: webDiplomacy.net new server

#16 Post by kestasjk » Sun Jun 20, 2021 1:14 am

Apologies I've restarted processing. Clearly need to dedicate some time to this locking issue
2

jmo1121109
Lifetime Site Contributor
Posts: 1099
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2944
Contact:

Re: webDiplomacy.net new server

#17 Post by jmo1121109 » Sun Jun 20, 2021 5:56 pm

Has anyone ran into the issue of the site being slow to load who is NOT on dark mode? I noticed that part of the delay was a missing css file in the KnownWorld variant and created a blank file, and I immediately stopped seeing the delayed load times. If you run into this again please let me know here.
2

teccles
Posts: 237
Joined: Mon Mar 25, 2019 8:10 pm
Karma: 285
Contact:

Re: webDiplomacy.net new server

#18 Post by teccles » Sun Jun 20, 2021 7:06 pm

I'm afraid I just had another slow load time (several minutes).

I am using dark mode.
2

jmo1121109
Lifetime Site Contributor
Posts: 1099
Joined: Fri Sep 29, 2017 4:20 pm
Karma: 2944
Contact:

Re: webDiplomacy.net new server

#19 Post by jmo1121109 » Sun Jun 20, 2021 7:27 pm

teccles wrote:
Sun Jun 20, 2021 7:06 pm
I'm afraid I just had another slow load time (several minutes).

I am using dark mode.
Was it the home page or something else on the site?
2

Yonni
Silver Donator
Silver Donator
Posts: 408
Joined: Thu Oct 19, 2017 6:55 pm
Karma: 368
Contact:

Re: webDiplomacy.net new server

#20 Post by Yonni » Sun Jun 20, 2021 7:31 pm

I believe it was a game and the home page for me but definitely the home page. Not using dark mode.
2

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests