Portal Home > Knowledgebase > Articles Database > Staminus Packet Loss and Downtime
Staminus Packet Loss and Downtime
|Posted by Jcink, 05-24-2012, 05:31 PM|
I'm not sure if this is the right place but over the last week I've been having serious problems with Staminus|secureport proxy/cloud service. Is anyone else noticing a LOT of packet loss with them lately or it is just me? It's very frustrating, I get anywhere between 25% - 75% packet loss and sometimes the service just drops out.
Before anyone asks, no, my attack logs show no indication of attacks at the time of the packet loss.
This hasn't happened once or twice, it's becoming a regular occurrence for me. I've noticed their website is down sometimes at the time these things happen and I've checked downforeveryoneorjustme.com as well.
I've complained to support and was given some credit for the downtime so far, but still, it's getting out of hand and I just wonder if I'm alone here.
Last edited by Jcink; 05-24-2012 at 05:36 PM.
|Posted by Jcink, 06-04-2012, 06:21 PM|
No idea why anyone would EVER use them now. The problems have only gotten worse.
My DDoS proxy with them is constantly down or loads extremely slow, and whenever it dies so does their main site. I checked with downforeveryoneorjustme.net and they're down there as well. Staminus is just terrible.
|Posted by ddosguru, 06-04-2012, 10:13 PM|
Do you have any specific evidence that you could share? My general impression was that Staminus had been improving overall service quality.
|Posted by toro, 06-08-2012, 11:54 PM|
I'm sorry you're experiencing issues. Please open a ticket in our help desk so we can assist you. Please note that we protect tens of thousands of websites. We are not experiencing any widespread network issues. We would be happy to help you explore the issues you are facing.
Jeff, thanks for the positive insight. It's difficult being the leader in ddos mitigation for so long. We have maintained this title because of customers like Dan who share their experiences with us. Pioneering new technologies consistently year after year takes determination and time.
Again, much appreciation to you for reaffirming this
|Posted by Jcink, 06-15-2012, 11:53 AM|
I've opened a lot of tickets. I've had a lot of trouble with the service really from day 1 but most of the issues got resolved, the random packet loss remains an extremely pressing issue.
I guess you'll know who I am if you look up my history since I'll reveal my IP address here.
Here's a rundown, I'm posting it since today once again my service was dead in the water from packet loss for over 30 minutes.
May 18th, filed a ticket "Packet Loss"
Gave a traceroute, the issue remained for quite a while. Packet loss was around 20%.
2 hours later and the issue was fixed. Wasn't sure what it was at this point.
May 19th, filed a ticket "service unavailable"
I don't have any traceroutes, but I noted in this ticket that packet loss was 90%. It could have been isolated to just my route, but I had several of my users complaining to me about it so I doubt it, unless they were being routed the same as me. My back-end was up, and they said they couldn't find any problems. I think it was only 15 minutes of downtime so I let it go.
May 23rd, filed a ticket "service not loading again"
This was an hour of downtime, and it included the staminus.net website for me. They said it was an issue with my IP only but I didn't believe it because downforeveryoneorjustme showed their site and other IPs within my subnet were down as well. I was credited for this downtime so that is a plus.
May 24th, filed a ticket "Service down again"
This was about 15 minutes of downtime that I noted in a comment on the ticket and I closed it. Had mild packet loss at this time but did not post a trace.
May 24th again, filed a ticket "Massive amounts of packet loss"
This went on for 2 hours, an issue was identified with my proxy or something and that the issue was fixed, but the packet loss remained:
I think the whole ordeal was around 3-4 hours of packet loss. They ended up changing my routing from Cogent communications to "above.net" and I was told it was due to an attack coming in over Cogent. Problem was solved.
June 3rd, filed a ticket "Service down once again"
The staminus.net website was down according to downforeveryoneorjustme and so was my service. For 2 hours! Support said it wasn't down just a "bit slow" but that was such an understatement. At this point I was just getting a little irate in the comments in the support ticket, saying I was going to cancel, being a little sarcastic, etc. As usual the issue sorted itself.
June 3rd, filed a ticket "You guys are hilariously bad at this point"
Main website at staminus.net was down for me again was down for 10 minutes before I could file the ticket as well as my proxy again. They say they have no complaints of their site being down, so I don't know. But I always check downforeveryoneorjustme, and it said they were. I think this was about 20 minutes of downtime for me again. Once again the issue just fixed itself.
Jun 6 00:06:45 PDT 2012, left a comment in ticket "You guys are hilariously bad at this point".
Service was down before I had posted in the ticket I think for 15 minutes according to some of the complaint times in my email. 5 minutes after filing it, it was back up again. Frustrating...
June 15th, 502 bad gateway, service EXTREMELY SLOW
This went on for 45 minutes TODAY:
This is just a constant problem with them, it doesn't happen 1 or 2 times, and it's usually severe. I'm sorry to post this but I feel this is never ending issue. "Fixed" is never really fixed. Can I go a week without packet loss?
Last edited by Jcink; 06-15-2012 at 12:00 PM.
|Posted by mcaynoz, 06-15-2012, 01:24 PM|
is the whole network having issues or is it just a certain amount.
|Posted by Jcink, 06-15-2012, 01:32 PM|
I don't have enough information to be able to answer that. They host tons of people. Maybe I've just gotten unlucky and have been given a bad section.
|Posted by toro, 06-16-2012, 02:59 PM|
It seems that most of your tests are via ICMP, which is severely to SecurePort IPs. If any of the above are TCP and are at 100%, it's suggestive of a block, not of packet loss. Your tests are showing that we have on average 50% loss from EVERYWHERE in the world. If we had ~1% loss, it would immediately show up in severe performance degradation.
I can't provide any assistance here. We obviously host many thousands of people who are happy. I would like to help you with your situation. However, I will need you to communicate everything via tickets. I am simply not able to provide you assistance via a public forum.
|Posted by Jcink, 06-16-2012, 03:27 PM|
I'm not really asking for a solution in this forum, I know the proper place is to file tickets, which I have done many times. Thanks though. But people wanted to see some support for my claims so I laid out my experience and what I knew.
I just shared my experience. And I started this thread to see if anyone else was having as much trouble as I am. No one else has responded. Be that as it may, whether I am an isolated incident or not I know that my IP appears to be having a lot of problems. And I swear that your website has been down as well before at the same time my own IP has gone down; whether those incidents are related I don't know.
I used just-ping because support staff linked that result one time during a ticket; that's what they were using to check packet loss. Maybe the numbers are not spot-on but I believe that there was certainly a lot of packet loss.
All I can continue to do is file tickets whenever the service goes down, but it's getting really old really fast. Surely you can understand my frustration when the same problem continues to happen again and again.
I am going to look into a more advanced monitoring service to watch my IP address more closely. As I continue to have problems with the service, I will file tickets and depending on the resolution will post my findings here. Maybe if others are having the same issue a common denominator can be found to fix it and I can mention it.
Just to add to all of this - service was disrupted again last night while I was sleeping. I got a notification about a site down check from siteuptime, backend was up, proxy was down. This was at June 15, 2012 23:08:21. A member of mine who I have watching this with me tested it with just ping and these were the results. http://imgur.com/5u9kn
|Posted by toro, 06-16-2012, 03:44 PM|
I'm not disputing that you have a problem. I'm simply saying that 50% loss is not "real". It's a side effect of possibly some other issue. Perhaps it's our ICMP limit.
We have monitors that test our network every few seconds. We have 0% loss consistently at every node. Any node that experiences >0% loss immediately pages our engineers. Any issues you face are isolated. That doesn't make them less of an issue by any means. It simply makes them harder to track. We will be happy to track these for you and resolve them.
If your goal is to share your experience, I commend you. If your goal is to get feedback from others with similar experiences, you can see that our track record is very positive. On the other hand if you are simply 'venting' and still believe there is a relationship to salvage, then I request that you return to our ticket system so we can work to resolve the issue.
There really is nothing more I can share regarding this thread. Thank you.
|Posted by Jcink, 06-16-2012, 04:03 PM|
Gladly. I'll let some time go by and continue to work with support, I wasn't planning on posting here again for another week or so anyway.
|Posted by Jcink, 06-17-2012, 01:37 AM|
I just want to say that Staminus kindly offered to refund the ENTIRE month because of how unhappy I was in my review here.
But I declined. Why?
The fact is that despite my venting I do like everything the service offers and it's been effective against a lot of attacks. My attack log is fantastic, they even filtered a 13Gbps attack and didn't null me.
These downtimes I keep having are just extremely painful for me and my users. I really hope my issues can be resolved and I will just keep working with support until they are.
|Posted by markspeed, 06-17-2012, 01:37 AM|
Just a few general observations since I've read your thread.
1. Any DDoS mitigation provider is going to result in some downtime for your own site, even if you aren't being attacked. This is because by the nature of their service, they are dealing with hundreds/thousands of attacks often on a daily basis. Unless your site fits into this category, you may want to reconsider your hosting strategy, or use a primary + backup strategy where you can fall back to your DDoS provider in the event that you are experiencing a large scale attack. To decide on this you should look at how often you're attacked, how willing you are to accept the consequences (cost/implementation) of failover, etc.
2. DDoS protection varies as well as how invasive it is. For example, mitigation at the network level only vs. the application level can have large repercussions for your site. If you are behind a mitigation device at the application level (say http) and it's filtering all requests, the potential for something to interact badly with your application may be higher, depending on the type of data/response being submitted via your web application. Additionally, being on a shared platform with other customers leaves room for more to go wrong.
That said, this type of protection can be really necessary with todays basic slow http/service attacks, or attacks specifically designed to cripple your application (suppose you expose an API for example.)
3. On the other hand, network level filtering will often result in blocks, rate-limiting, and possibly strange behavior unless tuned correctly. It would be important to understand what is impacting you and be able to create custom rules that work for your services. At the same time, you should understand that simply pinging, or providing trace results may not have any impact on your actual http services due to the mitigation procedures put in place. In fact, most networks globally deprioritize icmp responses from routers. What that means is when a router is busy, it continues to pass traffic without issue, but itself will appear to be showing packet loss because it's simply not responding to your packets based on policy. Use a tool like mtr, or tcptrace to check performance to your services instead of simple ping/traceroute.
I think what would also be really interesting is for you to setup an account over at pingdom.com. Next setup ping checks for your site at 1 minute intervals, and also setup checks for different services that you run - http, https, etc. Compare that to what you're seeing yourself. Since pingdom checks from multiple locations, you'll also get a feeling for the worldwide performance of your service.
Note: I don't know anything about Staminus and am not a customer. From what I have heard indirectly, they are more then competent with a well run operation.
I think that arming yourself with more knowledge on what could be happening, and test results that can help set your expectation and help you work to fine tune your implementation would be great, and that's why I'm chiming in. It sounds like you're trying/willing to work on the issue and in that case I would go beyond the simple ping/traces from your location.
|Posted by Jcink, 06-17-2012, 02:02 AM|
Thanks for the insightful reply. A lot of good points. I completely understand how using any DDoS protected service will result in some downtime compared to perhaps just a normal setup. I'm fine with occasional downtime but the more that this happens, the less it makes it worth paying for.
I haven't gotten there yet I don't think due to my attack frequency but it's still disheartening to get DDoS protection and end up with a lot of downtime not from DDoS but from either technical issues or because there are others in my IP pool taking huge hits ruining the service for everyone else.
You can see the general nature of my business if you visit my IP address - it's just a free forum hosting provider. Unfortunately the larger its grown, the bigger of a target its become, and because of the proliferation of "booters" where you can easily whack a site/service for a few minutes at time has made our life miserable. I don't really see us not having some sort of DDoS protection always up, sadly.
I've gone ahead and took your advice and set up an account at pingdom and I'm watching my Staminus IP address at 1 minute intervals. I'll pass any information I get from it on to staminus support and hopefully get to the bottom of what's going on here. Thanks for the suggestion.
Last edited by Jcink; 06-17-2012 at 02:09 AM.
|Posted by markspeed, 06-17-2012, 02:26 AM|
Cheers, I think some of the interesting things to look for would be:
1. Are there alerts, and if so are they from specific geo/regions. ( If so, could indicate a networking/routing issue.)
2. Do you get alerts for your ping check, but not for your service checks, ie http/https etc. ( Could indicate normal network level filtering, while your actual site is working well.)
3. Do your services alert, but not the network. (Could indicate an issue at the application level filtering vs. network)
One other thing to consider, is that it's possible for the monitoring devices themselves to be identified as an attack source and blocked. A service like pingdom which checks from multiple locations will check from another on failure, but it's not the be-all end-all. If you get an alert it'd still be interesting to check yourself if you're at the console to see what's going on.
Still, it should provide interesting data, especially over time and will help you objectivelly narrow down what's going on
Last edited by markspeed; 06-17-2012 at 02:34 AM.
|Posted by netmultiple, 06-17-2012, 02:31 AM|
I have seen few staminus thread for packet loss now. Hope they can fix it finally.
|Posted by white_tiger, 06-20-2012, 06:19 AM|
are staminus down again now?
|Posted by markspeed, 06-20-2012, 06:21 AM|
Well, I guess it would depend on what you define as down.
Their network isn't down, ie:
But going to their http://www.staminus.net site results in an HTTP/400 error for me.
Last edited by markspeed; 06-20-2012 at 06:22 AM.
|Posted by white_tiger, 06-20-2012, 06:28 AM|
website is not loaded...
|Posted by Brave, 06-20-2012, 06:42 AM|
Please wait patiently
|Posted by mrstevenl, 06-20-2012, 10:30 PM|
Maybe you should find out who staminus really is and who they protect. They are a heaven for the very "ddos" kids who are attacking many sites. Just do a simply google search about Staminus its know relationships with members of many different "hacker" groups.
Feel free to do some google searches on the ip ranges of staminus also and find all the scam sites and spam hosted there that is allowed because it is hosted by the same people above.
Heres some simple information there is tons of it out there if you search for it.
|Posted by Jcink, 06-20-2012, 10:46 PM|
That's a very harsh claim you're making there. I highly doubt they have known relations with hacker groups unless you have real proof of this.
I researched Staminus before I chose them and I didn't find anything of the sort. I'm sure you might have found some toxic and even controversial content that was hosted on staminus or something but they're like any other big host, you're always going to have some bad apples in the pool.
|Posted by mrstevenl, 06-20-2012, 10:50 PM|
Just do a simple google search of "Staminus kelly" should be the second link down is one site, and from there just find those keywords and search the web. You will find plenty more. From different people and sources. Do your own research and dont just take one posters opinion.
You can really judge someone by who they associate with, and since the early 2000s when staminus was falcon-networks it was always the same.
Add to Favourites Print this Article