[luzz]

Hello everyone. We hope you're all enjoying your Halloween weekend.

We are scheduled to fix a few bugs and reboot the server this Tuesday, November 1st, at 11PM (23:00) server time.

This is to fix a few bugs. During this time, we will be conducting some database management tasks as well. Due to this, the downtime will be a little but longer than usual.

Downtime will be about 15-30 minutes.

A patch will be released after this maintenance. All users MUST patch in order to reconnect to the server. A new Small Installer will also be released for those of you who have difficulty patching.


Thank you for playing Reign Online!
_________________
Support Reign Online by voting!

[luzz]

Hello. Maintenance has been postponed to THURSDAY, November 3rd at 11PM (23:00) server time.

We did not have enough time to prep for this maintenance schedule.
_________________
Support Reign Online by voting!

[luzz]

Server maintenance is completed, with some minor issues.

There are some bugs we still need to work on. Some of the bugs include item restrictions errors, map load display messages, some minor sprite issues, and possible some new quest bugs (Anthell, Nydhoggur's Dungeon).

These are being worked on and will be fixed in a day or so.


Website maintenance is fully completed.

We'll probably do another quick restart tomorrow night to fix some of these new bugs.


Although this was primarily maintenance, we did add the new Kagerou and Oboro sprites (with and without their frog mounts!) .

There are some new items, some new NPCs, a few new quests, and some changes to items.





Also to note is that items that heal 6% or more HP have a use delay of 3 seconds. We'll see how this pans out.


Enjoy, and let me know if you find any bugs.
_________________
Support Reign Online by voting!

[luzz]

Server was restarted to fix all the item restriction bugs (and other bugs).

Healing item delay applies to all items that heal 4% or greater. The delay is once per 3 seconds.

The server 'lag' issue was identified and corrected.



New Full and Small Installer version 2.12 is available for download for those who have issues patching.


I'll be working on all other minor bugs over the next days to ensure everything is resolved. Again, if you notice anything odd or out of the ordinary, please bring it to my attention.

Thank you for your patience in us getting all the bugs ironed out with this maintenance/update.
_________________
Support Reign Online by voting!

[luzz]

Hi everyone - the server is under maintenance for a few minutes.

We need this time to gather more data on what the cause of the issue may be. Server will be back up in a few minutes - hopefully with the issue resolved.

Will update shortly.
_________________
Support Reign Online by voting!

[luzz]

The issue was identified, and fixed.

Thank you all for you patience. This was a tough bug to fix, but mission accomplished!


_________________
Support Reign Online by voting!

[luzz]

As a show of thanks for everyone's patience and to celebrate this bug-squishing, I have enabled Double Experience and Double Pet/Homunculus Feeding for 3 days! (Ending 11:59PM Nov-10th)

Donation Girl trade-in timer also reset!


_________________
Support Reign Online by voting!

[luzz]

A few people asked me what the cause of the lag was.

Well.. here's the short of it:
The update caused more database queries to be run every 5 minutes. This caused a lot of disk writes. All the disk writes queued up for about 1 or 2 seconds. This caused the lag.


Here's the technical information (important for all you future server owners and administrators out there!):

- Every 5 minutes, the map server process updates the mapreg database table. Depending on how much information is in that table, this could amount to quite a few queries. Update queries are not very fast. We have 225 items in our mapreg table. This caused 225 UPDATE SQL queries every 5 minutes.

- Normally, this isn't an issue at all.

- However, us being the great admins that we are, we have live database backup set up in a replication manner. The live game database has a slave database at another datacenter. This means that there are two live copies of the game data at any given moment in time. All maintenance processing is done at the slave database (such as half-hourly snapshot backups) so that it would not slow down the live master database. (This is in addition to 3 offline copies of the database backups. We have over 68GB of COMPRESSED backups dating back to 2008! =O)

- In order for database (MySQL) replication to work well, we set a setting called sync-binlog. This setting states how often queries are written to the binary logs needed for database replication. For 100% fail-proof replication, this should be set to 1. This means that for every database update, don't delay, and write it to the binary log. The slave database gets it right away. BOOM. Instant sync.

- That's all nice and lovely... except that every 5 minutes, the map server updates mapreg (and possibly some other tables). With sync-binlog set to 1 write per query, the system was attempting to write about 300 blocks almost simultaneously. Since disk IO have highest system priority, this caused the map server to 'lag' for the during of this write.

- The solution was to set sync-binlog to 0. This queues up queries before writing them to the binary log for replication. This is more efficient because many queries can be written in a single disk IO operation, as opposed to 1 block disk IO per query. Essentially, we went from 300 disk IO operations in this 5 minute cycle to about 1 or 2 every minute.

- This increased disk IO efficiency is great, but it does make replication less reliable. That means should the main database ever go down unexpectedly, I have to do database maintenance on the slave server to make sure it's fully sycn-ed to the master database server.



All this technical mumbo jumbo!
*head explodes*
_________________
Support Reign Online by voting!