Christopher-Outlaws!!
Approved user
- Joined
- Apr 15, 2016
- Messages
- 296
Hello,
I'm not writing this as an attempt to step on toes, or to moan about things that don't exist. However, I do know for a fact that there have been many disconnect errors regarding gameplay preventing people from getting on the game for any consistent basis. That and I don't often have lots of time to geek out or share it here. But here goes.
Now understand there is a raft amount of speculative information herein, and the recommendations are based on this speculation, so I may be way off base, however will do my best to speak from experience in online game development/deployment/analasys.
The game communicates via http to the host prod2.dominationsgame.com
according to my discovery, this resolves to Amazon servers, I can only assume cloud based architecture.
so, the problem regarding disconnects becomes difficult on the failure to reproduce the problem on a consistent basis. However a simple tcpdump could be run on one of the effected customers in order to iscolate the problem and garner more details for dev to investigate, or a network architect, I'm sure Amazon has people to help with that.
So here's the Speculation/Assumption
The communication protocol in use is http 1.1 it's gzip and identity encoded traffic.
I have to assume on the production or server side of the problem they would have apache or tomcat running which would listen / parse the traffic in order to accomplish what was required. not a big deal, though I'm positive if it is apache it could potentially be tuned or even compiled differently in order to better utilize the system resources.
I'm going to assume Apache here, because it's easier... that and the traffic told me so.
Apache uses sessions to organize traffic, and when not configured they aren't handled in the best manner. My suggestion regarding sessions, move them to the Database, that way at a minimum you would have the ability to trace a connection with a time, further to this with the traffic dump it would be excellent supporting evidence.
I would suggest as well enabling apache statistics alongside even customizing the logging on apache to garner information such as timouts.
Moving on from there, the system, and here's where we really assume a lot.
I'm assuming the system is linux based, which is assumption 1.
I also assume based on the fact that I love the game and assume others do as well, that the servers are extremely busy. it would be worthwhile to check the logs for nf_conntrack. This is, the number of connections to the server based on the allowable system limitations. If it were a problem it would be very evident within the logs, ie. "nf_conntrack table full, dropping packet". That would help significantly and allow the dev or sys admins to modify the table limits, alongside the tcp stream timeouts and stabalize connections.
The enabling of SAR would be extremely beneficial, allowing a historical view of specific system information, disk i/o memory performance etc etc. couldn't hurt to have eyes on.
last but certainly not least, the number of allowable connections to the DB, I'm assuming it's a hardened environment, I'm not testing this as I don't really want to get banned from the game and don't feel it's my right to check such things. but the connections from apache to the db should be checked, I'm willing to garner that the allowable connections could be increased, alongside the base memory and cache to improve the db performance.
Anyways. If you made it this far in the read, thanks for letting me geek out. If you're with Nexon, I hope you can take this and go get yourself a raise as I feel these are pretty good recommendations from an outside set of eyes and no insight on the actual architecture past what is publicly available.
I guess the point of doing this, I like this game, and want to see it improve by any means possible. I do this for a living so yes, this is donated time(about an hour). but I hope ultimately it helps to improve the game we all love.
Sincerely,
Christopher
Keep your sticks on the ice, and have a good time!!
Outlaws!!
I'm not writing this as an attempt to step on toes, or to moan about things that don't exist. However, I do know for a fact that there have been many disconnect errors regarding gameplay preventing people from getting on the game for any consistent basis. That and I don't often have lots of time to geek out or share it here. But here goes.
Now understand there is a raft amount of speculative information herein, and the recommendations are based on this speculation, so I may be way off base, however will do my best to speak from experience in online game development/deployment/analasys.
The game communicates via http to the host prod2.dominationsgame.com
according to my discovery, this resolves to Amazon servers, I can only assume cloud based architecture.
so, the problem regarding disconnects becomes difficult on the failure to reproduce the problem on a consistent basis. However a simple tcpdump could be run on one of the effected customers in order to iscolate the problem and garner more details for dev to investigate, or a network architect, I'm sure Amazon has people to help with that.
So here's the Speculation/Assumption
The communication protocol in use is http 1.1 it's gzip and identity encoded traffic.
I have to assume on the production or server side of the problem they would have apache or tomcat running which would listen / parse the traffic in order to accomplish what was required. not a big deal, though I'm positive if it is apache it could potentially be tuned or even compiled differently in order to better utilize the system resources.
I'm going to assume Apache here, because it's easier... that and the traffic told me so.
Apache uses sessions to organize traffic, and when not configured they aren't handled in the best manner. My suggestion regarding sessions, move them to the Database, that way at a minimum you would have the ability to trace a connection with a time, further to this with the traffic dump it would be excellent supporting evidence.
I would suggest as well enabling apache statistics alongside even customizing the logging on apache to garner information such as timouts.
Moving on from there, the system, and here's where we really assume a lot.
I'm assuming the system is linux based, which is assumption 1.
I also assume based on the fact that I love the game and assume others do as well, that the servers are extremely busy. it would be worthwhile to check the logs for nf_conntrack. This is, the number of connections to the server based on the allowable system limitations. If it were a problem it would be very evident within the logs, ie. "nf_conntrack table full, dropping packet". That would help significantly and allow the dev or sys admins to modify the table limits, alongside the tcp stream timeouts and stabalize connections.
The enabling of SAR would be extremely beneficial, allowing a historical view of specific system information, disk i/o memory performance etc etc. couldn't hurt to have eyes on.
last but certainly not least, the number of allowable connections to the DB, I'm assuming it's a hardened environment, I'm not testing this as I don't really want to get banned from the game and don't feel it's my right to check such things. but the connections from apache to the db should be checked, I'm willing to garner that the allowable connections could be increased, alongside the base memory and cache to improve the db performance.
Anyways. If you made it this far in the read, thanks for letting me geek out. If you're with Nexon, I hope you can take this and go get yourself a raise as I feel these are pretty good recommendations from an outside set of eyes and no insight on the actual architecture past what is publicly available.
I guess the point of doing this, I like this game, and want to see it improve by any means possible. I do this for a living so yes, this is donated time(about an hour). but I hope ultimately it helps to improve the game we all love.
Sincerely,
Christopher
Keep your sticks on the ice, and have a good time!!
Outlaws!!
