They could of spent that much time of making an announcement, replying to people etc when they could just switch off from Aws.I don't necessarily think that d3l3t3d is a bad SysAdmin - he just doesn't understand how affected the site is. Maybe he doesn't know how to fix it, who knows? But still - Mick should persuade him to update the site and/or transfer the data to a new platform. It's all in his power. I've offered many times to help him out.
Yeah - again Mick - I'm very glad to help out with the transfer and reliability engineering.They could of spent that much time of making an announcement, replying to people etc when they could just switch off from Aws.
Yes, AWS went down for 3 hours - that's correct. This does not shadow the fact that McM had downtime before and still has random outages these days too. There are CloudFront configurations that they aren't utilising & networking SRE tools they aren't using. This downtime CAN be fixed, especially by transferring to GCE & reconfigurating a few things.[DOUBLEPOST=1488920773][/DOUBLEPOST]
I'd agree that a dedi would work and be easy to manage, BUT easy is never the way to go - especially in today's generation of websites and services, everything needs elastic scalability and reliability-designed services. If they're going to go with AWS, they need to do it right - and they aren't at the moment. So I'd suggest moving to GCE and learning how to scale using it.Personally if I was MCM's system admin, I would got for a dedicated server running Nginx behind cloudflare. Relatively cheap, somewhat lacks future expandability however is quite reliable and easy to manage. Not as reliable as something like AWS should be though. Mick and d3l3t3d are going down a more expensive path, where there have been a few problems to say the least. There has to be good reason behind them doing this. I cant see Mick blindly letting MCM go though all these problems.
Several suggestions and posts have been made about this. Nothing we say or do is going to change it. All we can do is wait until things get sorted out.
What does GCE have to do with anything?Yes, AWS went down for 3 hours - that's correct. This does not shadow the fact that McM had downtime before and still has random outages these days too. There are CloudFront configurations that they aren't utilising & networking SRE tools they aren't using. This downtime CAN be fixed, especially by transferring to GCE & reconfigurating a few things.
1mil requests/15min isn't a DDoS (d3l3t3d his numbers). In fact. MCM gets roughly 1.1k requests per second(avg during my time. Addons/settings may have changed). If you do the math, that's 990k per 15 minutes. The previous MCM setup was capable of handling a DDoS attack which resulted in 90k requests per second. Not a big number, highest I was able to record though. Forum stayed online! 70k requests per day is way too little. Imagine each browser checking alerts, sb messages and PM's every 500ms. Those are all considered "requests" which add up to resource usage.In some cases we absorbed over a million requests in 15 minutes. For comparison, our daily average is about 70K.
I see your point - my main reason behind mentioning GCE is due to the simple setup & the amazing network speeds. Of course, it could run on an OVH box - but GCE is a scalable host and doesn't require that much skill to scale like AWS - it's a lot more user friendly and IMO is the best place to host a website like this. (also, pointing out - Linode is technically a cloud host, it's just a very simple one like DigitalOcean)What does GCE have to do with anything?
You guys are seriously overthinking it. MC-Market doesn't require anything complex or sophisticated like a cloud setup. Spigotmc.org runs on a regular dedi and has been doing that ever since it started. They have x6 times more members and there's no doubt that their resource section transfers more data in a single hour than MCM does in a whole month.
Let's do this: https://theadminzone.com/forums/servers-and-control-panels.435/
This is the server category of TAZ, the biggest admin forum. Nobody there talks about AWS simply because it's unnecessary and overly complicated. You can obviously see everyone refering to nginx and Apache configurations simply because that's the standard and it works just fine. Besides that, XenForo isn't really ready out of the box to run on a cloud platform. This simply isn't needed because it's extremely lightweight.
I've done a lot of research on XenForo before getting the setup and infrastructure I eventually ended up using. Not because I didn't know what to use but simply because I was curious. Here's some stats:
https://smashboards.com/forums/ - a large gaming forum. Has 1.4k users on at this time - runs on an OVH box.
http://www.redcafe.net/ - Manchester United forum. Has 2k users on at this time - runs on a regular coretx.com box
https://www.urban75.net/forums/ - banter forum. 500 users on at this time (what MCM gets during peaks) - runs on a cheap exonetric.com box
https://forums.spacebattles.com - Spacebattle forum has 3.6k users online right now, owner is on taz and talked to him - runs it on a 4gb linode VPS
https://www.spigotmc.org/ - runs on an OVH box
These are just a few examples that I know of that operate on extremely cheap solutions because they don't have the financial backing MCM has. The point I'm trying to make is that MC-Market is tiny resource usage wise.
1mil requests/15min isn't a DDoS (d3l3t3d his numbers). In fact. MCM gets roughly 1.1k requests per second(avg during my time. Addons/settings may have changed). If you do the math, that's 990k per 15 minutes. The previous MCM setup was capable of handling a DDoS attack which resulted in 90k requests per second. Not a big number, highest I was able to record though. Forum stayed online! 70k requests per day is way too little. Imagine each browser checking alerts, sb messages and PM's every 500ms. Those are all considered "requests" which add up to resource usage.
I'm not trying to sound rude, offensive or degrade d3l3t3d in any way. I'm just giving my opinion from my point of view. This issue has been ongoing for way too long and everyone has been left in the dust. Nonetheless, best of luck fixing this.
GCE literally does the same thing as AWS. AWS just has more services.I see your point - my main reason behind mentioning GCE is due to the simple setup & the amazing network speeds. Of course, it could run on an OVH box - but GCE is a scalable host and doesn't require that much skill to scale like AWS - it's a lot more user friendly and IMO is the best place to host a website like this.
I know that McM will probably never *NEED* to durastically scale; but, on GCE it's very easy to do so and doesn't have any side effects like downtime - for example, across the week, McM could be running on a lower-cost plan other than the weekends automatically which is already more efficient - and GCE/AWS have the same idea, I agree, but I have worked with AWS for 2 years and GCE for the past year and I know from experience that it's easier to perform scaling and network stability tasks. And back to your point about Xenforo - it doesn't matter that it isn't "built for the cloud" - most things aren't, but it doesn't mean it can't utilise it well.[DOUBLEPOST=1488923896][/DOUBLEPOST]GCE literally does the same thing as AWS. AWS just has more services.
The whole concept is the same. Both are Cloud platforms and have different scalable services which make a scalable website or platform work. An application or website can't perform optimally on a Cloud platform if the code and structure isn't made for it (XenForo's case). MC-Market doesn't need to worry about scalability unless it starts hitting over 10000 simultaneous visitors which isn't going to happen within the next few years(or ever imo).
Secondly, it would NOT be cost efficient to run McM on an OVH box with 16GB RAM for example (lowest you can go) - dedis aren't really optimised for websites and network services and more for stuff like Minecraft which take up quite a lot of memory and computing power, whether with the cloud, you have a lot more pre-optimisation already done for you. Not mentioning that McM won't even take up a GB of a dedi's compute power usually.GCE literally does the same thing as AWS. AWS just has more services.
The whole concept is the same. Both are Cloud platforms and have different scalable services which make a scalable website or platform work. An application or website can't perform optimally on a Cloud platform if the code and structure isn't made for it (XenForo's case). MC-Market doesn't need to worry about scalability unless it starts hitting over 10000 simultaneous visitors which isn't going to happen within the next few years(or ever imo).
There's a thing called scalable coding. I'd know because my company does it. Please inform yourself before telling others to put their service on a cloud and calling it a day.And back to your point about Xenforo - it doesn't matter that it isn't "built for the cloud" - most things aren't, but it doesn't mean it can't utilise it well.
I'm aware about scalable coding - that doesn't mean the system side can't scale as well. There are many different types of "scaling operations" - and cloud service scaling is one. Scalability is a VERY broad topic these days - read more here:There's a thing called scalable coding. I'd know because my company does it. Please inform yourself before telling others to put their service on a cloud and calling it a day.
Basic article on scalable code: https://www.mullie.eu/why-your-code-doesnt-scale/
I'm done replying to you, I'm not a teacher and this is the suggestion section.
Spigot runs on Linode. They used to use Intreppid, however dropped them as a sponsor and left their services.https://www.spigotmc.org/ - runs on an OVH box
I know for sure that they were running on OVH at one point. That's what a fellow staff member told me at least. Nonetheless, thanks for letting me know.Spigot runs on Linode. They used to use Intreppid, however dropped them as a sponsor and left their services.
They certainly don't use OVH.
"some point" - them leaving OVH is a big example of why hosting a website on an OVH box is efficient.I know for sure that they were running on OVH at one point. That's what a fellow staff member told me at least. Nonetheless, thanks for letting me know.
You're not competent enough to be the system administrator for MCM, nor would Mick employ you anyway since he's already said hiring someone from the community will put the site's security at risk. I would leave this sort of stuff down to people who know what they're talking about. I'm sure Mick appreciates your enthusiasm to help nonetheless.I'm aware about scalable coding - that doesn't mean the system side can't scale as well. There are many different types of "scaling operations" - and cloud service scaling is one.
