- Joined
- Apr 21, 2015
- Messages
- 484
UPDATE: Thank you for your patience while we continue to work on resolving the issue of your village's progress reverting. Currently we are restoring bases to a more current state. Within the next 12 hours, you should receive a message in your inbox stating that we have resolved your issue, as well as compensation for your situation. If you do not receive a message after 12 hours, are still having issues, or this message is not relevant to your issue, please resubmit a ticket.
--------
LATEST - 21:45 UTC Update
The restoration scripts passed testing and are rolling out to players now. We will be messaging those affected and fixed players through the in-game inbox shortly.
--------
20:45 UTC Update
We're performing tests on the restoration scripts. If the scripts pass testing, we will be rolling out the fix to restore those affected players.
--------
18:00 UTC Update
We identified each of the affected accounts and discovered that less than 5% of players have been impacted by today's rollback. Most players are unaffected. For impacted players, they could see lost progress. We are working on a solution to restore those players to our most recent backup available. Further details below.
Around 10:00 UTC we had a single server encounter a hardware issue and fail. We began our standard progress of promoting that server's backup. Unfortunately that backup server also failed. When the backup server came online around 10:25 UTC, players on that server started reporting lost progress. Fortunately, we save and backup individual player data in a separate system that we are utilizing to restore these impacted players. Since these individual backups were not intended to mass restore accounts, we are developing a solution that will automate the restoration and hope to deploy this fix as soon as it is ready.
--------
14:00 UTC Update
Some users may have had their game progress rolled back. We are currently investigating the scope and severity of this issue and working on a solution.
More info will be posted here as it emerges.
Server hardware and backup failures are at the root cause of the rollback. We are currently working on a solution to restore users' progress.
--------
LATEST - 21:45 UTC Update
The restoration scripts passed testing and are rolling out to players now. We will be messaging those affected and fixed players through the in-game inbox shortly.
--------
20:45 UTC Update
We're performing tests on the restoration scripts. If the scripts pass testing, we will be rolling out the fix to restore those affected players.
--------
18:00 UTC Update
We identified each of the affected accounts and discovered that less than 5% of players have been impacted by today's rollback. Most players are unaffected. For impacted players, they could see lost progress. We are working on a solution to restore those players to our most recent backup available. Further details below.
Around 10:00 UTC we had a single server encounter a hardware issue and fail. We began our standard progress of promoting that server's backup. Unfortunately that backup server also failed. When the backup server came online around 10:25 UTC, players on that server started reporting lost progress. Fortunately, we save and backup individual player data in a separate system that we are utilizing to restore these impacted players. Since these individual backups were not intended to mass restore accounts, we are developing a solution that will automate the restoration and hope to deploy this fix as soon as it is ready.
--------
14:00 UTC Update
Some users may have had their game progress rolled back. We are currently investigating the scope and severity of this issue and working on a solution.
More info will be posted here as it emerges.
Server hardware and backup failures are at the root cause of the rollback. We are currently working on a solution to restore users' progress.
Last edited by a moderator: