This update was authored by Mod Cyphus and was released on 24 May 2011.
You wait. You pass the hours with a raid here, a capture there. Maybe some small-scale Settlement upgrades. Still it builds.
Finally the time comes to unveil your new Town Hall. You'd been told it would be ready on time, but to your horror you find that your workers have not yet finished. Only one thing stands between you and the latest jewel in your Settlement's crown:
'00:00:00 / Soon': We have unfortunately all seen it, but what causes this most annoying of bugs? Well, in order to answer that, I will have to tell you a little bit about how things work behind the scenes.
The servers that run 8Realms spend most of their time doing one of two things:
1 — Answering page requests. Every time someone clicks on a Settlement building or moves around the World Map, a request is sent to 8Realms. The server, on receiving the request, performs some processing and returns the result to your web browser. In order for the game to not appear laggy, this all has to happen as fast as possible.
When lots of people all try to play 8Realms at the same time, the server needs to do more work, and so it takes longer to send a response back to your web browser.
2 — Processing queued tasks. Whenever the server needs to schedule an action to happen in the future, it adds an item to a queue, along with instructions about when that item should be processed. Queued items include the completion of a building, training a troop, transferring resources, and moving a marching army to its next tile.
If that list of items looks familiar, it's because these are actions that can suffer from the dreaded 00:00:00 / Soon problem. You see, like page requests, the processing of items from the queue slows down as the number of people simultaneously playing 8Realms grows. If the processing of queue items gets too slow, then a backlog of items builds up, and queue items end up getting processed late.
Late queue items = 00:00:00 / Soon problem.
So how do we solve it? As we've seen, it's not one bug that a few lines of code can solve, but a question of server performance and scalability. These are some of the thornier issues when developing a game such as 8Realms, and for us developers, such issues are a big part of what we want to test during Closed Beta. Rest assured that every time the server is slow or crashes, we get a lot of debug info which directly feeds back into our development.
For example, one area that we found to be very slow was the Activity Stream. While trawling through the data from the Closed Beta, Mod Jenesis discovered a number of bits of code that were retrieving data from the database every request. This was quite slow, as the 8Realms server needed to talk to the database server and wait for the reply before it can continue. A few hours later, he had a version which produced the same result without needing to talk to the database at all. This made the Activity Stream ten times faster than before.
Similarly, we found that some queue items were taking longer than others to complete. At that time, we were only processing one queue item at once, and a small backlog of items would form when a relatively slow item was being processed. In order to avoid this happening, we have increased the number of queue items that can be processed at the same time. In addition, we have moved some of the slower queue item types into their own queue, so that they no longer cause a backlog in the main queue.
So, to summarise, we're very aware of the performance problems and we're working hard to address them. This is, in part, what the Closed Beta is for, and I'd like to thank you all for helping us give this game a proper workout.
Mod Cyphus
8Realms Lead Developer