Current Time: 15:26:04 EST
 

iis517 – Emergency maintenance- Resolved

Posted In: Maintenance — Feb 06th, 2013 at 4:56 am EST by IX: Admin

Incident Description:

Our systems administrators have identified an issue in the RAID for iis517.  The server is currently under a raid rebuild. We will keep you updated on the status as often as possible.

Which Customers are Impacted?

All clients hosting websites using iis517.

How are Customers Impacted?

All websites/FTP/control panel will not be reachable.

How often will we be updated?

6 hours.

Time to Resolution (ETA)

2/09/2013 11:00 PM - ETA is 30 hours 2/10/2013 7:41 PM - ETA is 40 hours 2/11/2013 3:15 PM - ETA is 30 hours 2/13/2013 12:42 PM - ETA is 35 hours 2/13/2013 8:12 PM - ETA is 30 hours 2/14/2013 5:59 AM - ETA is 24 hours 2/14/2013 11:24 AM - ETA 24 hours. 2/14/2013 3:50 PM - ETA 20 hours. 2/14/2013 9:10 PM - ETA 16 hours. 2/15/2013 10:04 AM - ETA is 10 hours 2/15/2013 3:10 PM - ETA is 6 hours 2/15/2013 8:10 PM - ETA is 2 hours

Incident Updates

  • 2/06/2013 9:52 AM - The operating system has be reinstalled and our admins are currently restoring the disk data.
  • 2/06/2013 1:32 PM -We are finishing installing server applications. ETA currently 16 hours for total server recreation.
  • 2/07/2013 7:30 AM -Our administrators are preparing to install the control panel. Current ETA is an additional 10 hours.
  • 2/07/2013 5:08 PM - Our administrators are working on seting up the IPs and then into the recreation of IIS site structure. Current ETA is 10 hours.
  • 2/08/2013 02:56 AM - Recreation slower than expected. Current ETA is 12 hours.
  • 2/08/2013 01:36 PM - Recreation is still ongoing. Current ETA is 10-24 hours.
  • 2/09/2013 1:446 PM - Recreation of user accounts via control panel has resulted in errors that are currently being investigated.  Once resolved, account recreation can resume.  ETA is unfortunately increased to 48 hours.
  • 2/09/2013 11:00 PM - Recreation is still ongoing. Our current ETA is 30 hours.
  • 2/10/2013 9:52 AM - Physical recreation is on the way. 400 domains have been recreated. Around 900 domains left. Once all domains are recreated our admins will start working on recovering permissions. ETA 32 hours domains recreation + recovering permissions. We will have an ETA on recovering permissions once it starts.
  • 2/10/2013 7:41 PM - Recreation of all the domains is still underway. The recreation of domains is set for about another 22 hours then there will be some time to correct all permissions on the server. Estimated ETA for everything is 40 hours.
  • 2/11/2013 12:56 PM - Recreation of all the domains is still underway. The recreation of domains is set for about another 22 hours then there will be some time to correct all permissions on the server. Estimated ETA for everything is still around 40 hours.
  • 2/11/2013 3:15 PM - Currently we are working on copying all user content back to there respective home directories for all customers on the server. ETA is 30 hours for all data to be restored completely.
  • 2/12/2013 7:33 AM - We are restoring content from a local copy. We have 60 accounts done and 120 more to go and then our administrators will have to restore permissions.  ETA for account restoration is 16 ho0urs plus the time it will take to restore permissions.
  • 2/12/2013 2:20 PM - Content has been restored. We are still installing other control panel features and restoring permissions. ETA 30 hr.
  • 2/13/2013 06:16 AM Restoring permissions on accounts.  ETA for this stage is 40 hours.
  • 2/13/2013 12:42 PM 7 out of 155 accounts have had permissions restored. After this there are 2 more steps in the server recreation before it will be back online. Current ETA is 35 hours.
  • 2/13/2013 8:12 PM Current ETA is down to 30 hours. Our administrators are working diligently to restore permissions.
  • 2/14/2013 05:59 AM Permission restoration continues.  Current ETA is about 24 hours.
  • 2/14/2013 11:24 AM Permission restoration continues.  75 accounts have been restored with 80 still remaining. ETA 24 hours.
  • 2/14/2013 3:50 PM Permission restoration continues.  90 out of 155 accounts have had permissions restored. ETA 20 hours.
  • 2/14/2013 9:10 PM Permission restoration continues. 130 out of 155 accounts have had permissions restored. ETA 16 hours. Once this is complete complete server will be started and site functionality verified. If all is well and no further troubleshooting is needed, accounts will be made accessible at that time.
  • 2/15/2013 12:25 AM Permission restoration complete.  Sites should be available, however they may experience slowness or interruptions.  We are troubleshooting and verifying all server details.  Current ETA is about 12 hours.
  • 2/15/2013 10:04 AM - Troubleshooting and verification of all server details continue.  Our administrators are focusing on bringing sites online. Secondary services like FTP will be brought online as soon as possible.  Current ETA 10 hours.
  • 2/15/2013 3:10 PM - Troubleshooting and verification continue.  Current concern is an issue with network connectivity stability.  In order to isolate the issue, some services (including web service) will need to be toggled.  This will impact site availability.  Current ETA 6 hours.
  • 2/15/2013 8:10 PM - All services are up and running. Some fixes need to be aplied to PHP but these should be completed before the end of the previous ETA.  Current ETA less than 2 hours.
  • 2/16/2013 7:15 AM - The server is completely back online. There may be some issues with SSL configuration within individual accounts. If you are having any issues please call, chat or put in a ticket with us 24/7 and we would be happy to get that resolved.

Resolution Description

The server is back online. If you are experiencing any issues please call, chat or put in a ticket with us.

posted in Maintenance by IX: Admin

104 Comments to "iis517 – Emergency maintenance- Resolved"

Wed, February 6th, 2013

Karen says:

Is mail effected?

Wed, February 6th, 2013

IX: Robyn M. says:

Hi Karen,

Email should not be effected. Your email is stored on a different server.

Thanks!

Wed, February 6th, 2013

jdsqc says:

when maintenance finished? urgent …………..

plz quick!

Wed, February 6th, 2013

Derek Martin says:

This outage is not being handled very well. Where is the redundancy/fail-over options. Considering that the total time of unavailability is now estimated at more than 24 hours, there should be some communication indicating this will not be how this type of outage will be handled in the future.

Wed, February 6th, 2013

IX: Jared says:

Hello Derek,

Unfortunately with an issue such as a RAID error, the drive has to be rebuilt in order to prevent much more serious problems. There isn’t really any way of speeding this up. These issues are rare on any given server, but when they occur, we have to take quick action to protect the accounts on that server.

I certainly apologize for the inconvenience this has caused you. If you have any questions, feel free to contact us.

Thu, February 7th, 2013

Derek Martin says:

16+ hours since last update…no accessibility and no update.

Thu, February 7th, 2013

IX: Robyn M. says:

Hello Derek,

Sorry for the delay. I am currently getting an update and will update the blogs here shortly.

Thanks!

Thu, February 7th, 2013

Derek Martin says:

Would it be possible to list all of the remaining steps, and their eta, necessary to restoring access? It appears that iis517 is going to suffer close to 48 hours of downtime.

Thu, February 7th, 2013

IX: Robyn M. says:

Hello,

Unfortunately I do not have the exact steps in the recreation. We will continue to update this blog on a regular basis with any additional information we have.

Thank you!

Thu, February 7th, 2013

Derek Martin says:

Is there a step after installing the control panel?

Thu, February 7th, 2013

Brian T. says:

I find this outage to be handled terribly, and has gone on for far too long.
I feel IX-Hosting now owes all the users of this server 1 month free service for the inconvenience

Thu, February 7th, 2013

IX: Robyn M. says:

Hello Brian,

We are extremely sorry for the down time. I am updating the ticket in your account to discuss matters further. Please update us there with any questions.

Thanks!

Thu, February 7th, 2013

Karen says:

This is ridiculous. I hope you are working around the clock until this is sorted! Please give an updated ETA. N/A is not acceptable.

I agree with Brian T, charging us for your incompetence would be like a slap in the face. My clients could be and probably are losing custom!! So much for your “Tier 3 n +1 redundant data center”. You may want to rephrase your promotional text as the following statement is clearly false advertising.

“The success of a hosting company largely depends on whether they are able to keep their customers’ websites online and accessible to the web. But 99.9% website uptime is really just one one of those phrases you hear hosting companies throw around, but not define in real terms. We have a Tier 3 n +1 redundant data center that keeps your site up the maximum amount of time.”

Thu, February 7th, 2013

Derek says:

The level of service offered is in need of much improvement. The amount time to restore is horrible. the frequency of updates is horrible. Horrible…horrible…horrible!

Thu, February 7th, 2013

IX: Chris B says:

Hello Karen,

We do apologize greatly for this downtime. It was definitely not planned and once everything is resolved we are more than happy to work with everyone on what happened. We are working as fast as we can and as much as we possibly can to get this back up and running as soon as we can. We just provided an update that was given to us with an ETA of 10 hours. Once we have more information we will make sure to let you know as this progresses.

Thank You.

Thu, February 7th, 2013

IX: Chris B says:

Hello Derek,

I am very sorry that this is and has taken so long to resolve. This was definitely not something that was planned. We are working as fast as we can but since this is not just a restore but a complete recreation of the server it is taking quite some time. Once we have more information we will make sure to update you.

Thank You.

Fri, February 8th, 2013

jenn says:

This is so beyond unacceptable that I don’t even know where to start. You obviously had a single point of failure, though your advertising claims otherwise. Worse, you can’t even get your single failed point back up after 48 hours. I agree with Karen. You committed fraud when you stated the capabilities of your data center. I’m thinking now that it’s a server in some dude’s basement. I just hope now that you can restore the data so I can move the site asap.

Fri, February 8th, 2013

Derek says:

I agree with Jenn, Chris and Karen. I left my last provider for the same nonsense. However, it took close to a week before they restored service after a similar outage, an you appear to be tracking the same. I am very much under the impression, sadly, that you have overstated your company’s hosting service reliability and support. Once, you have actually restored service, as a matter of customer service, IX should be VERY proactive and generous to the customers they have adversely affected.

Fri, February 8th, 2013

Brian T. says:

Unbelievable.
This is still down
This has been well over 48 hours!

I’m in full agreement with what the others are saying here.

Quote Dereck
“Once, you have actually restored service, as a matter of customer service, IX should be VERY proactive and generous to the customers they have adversely affected”

I couldn’t agree more.

Fri, February 8th, 2013

IX: Robyn M. says:

Hello Brian and Derek,

We are extremely sorry for the downtime and are working as quickly as we can to get the server back online. I will be posting a new update here in about an hour once we have further information from our administrators.

Thank you.

Fri, February 8th, 2013

Will K. says:

I’d like to add my voice to those who have expressed GREAT disappointment that our web and ftp sites have been down for such an extended period. We were told we would have exceptional reliability here — being down this long does NOT show a very good design to your system. We’re not getting what we pay for.

Fri, February 8th, 2013

jenn says:

I think at this point we need some reassurances that the data is ok. I can’t think of any scenario where a rebuild from backup would take this long unless you’re having problems with the data.

Fri, February 8th, 2013

Derek says:

There should be another update posted to indicate if you are nearing the end of the last 12 hour ETA.

Fri, February 8th, 2013

Mike says:

I am very surprised that there is no data backup that can be restored to backup server or virtual server. I would have assumed that a hosting service the size of this one would have data and hardware redundancy.

Isn’t there a guarantee of service like 99.999% up-time? It is time to look for a new host.

Fri, February 8th, 2013

Derek says:

10-24 hours?!?!?! what kind of estimate is that?

Fri, February 8th, 2013

fadi kadi says:

Dear IX Team,

I would appreciate if you could send us emails when the issue started rather than getting to know the issue from our clients. Also to keep us informed when the server is back online for us to inform our clients….
you effort is appreciated….

Regards,

Fri, February 8th, 2013

Karen says:

For anybody thinking of moving. I hear Dreamhost is very good!

Fri, February 8th, 2013

Brian T. says:

A very wide open estimate indeed.

I’m not allowed to give such estimates to my boss let a lone the customers.

Fri, February 8th, 2013

Derek says:

Post an update.

Sat, February 9th, 2013

Derek says:

Hey, IX! You need to post an update!

Sat, February 9th, 2013

Brian T. says:

Unbelievable! Are you going for a 96 hour outage or what?

UPDATE REQUIRED

Sat, February 9th, 2013

Will K. says:

Now there’s ANOTHER drive problem on a different server (IIS1001)!!! What’s happening, IX — are you going under? No one’s posting updates to this incident– what does that say to your customers? We’re getting pretty nervous…

Sat, February 9th, 2013

Brian T. says:

Can you tell it’s the weekend.
Are they even working on this still.
No Updates still

Do yourselves a favour and remove the 4 hour updates notice. It’s just more lies.

Sat, February 9th, 2013

Derek says:

Please update or resolve.

Sat, February 9th, 2013

Carol says:

Although this is a real pain,
I have had three accounts with IX for at least 6 years
and their service has always been very responsive.
no I do not know anyone over there, nor am I connected with anyone over there. Fair is Fair though,
and although I have 40 sites down over this, when you average it out, the service has been good.

However, get the darn thing fixed please….

Sat, February 9th, 2013

Mike says:

3 days and counting.
1 day since an update posting.
No email to those affected

I’m looking forward to getting a full refund and finding a new hosting service.

Sat, February 9th, 2013

wael says:

3 days and counting.
1 day since an update posting.
No email to those affected

I’m looking forward to getting a full refund and finding a new hosting service.

Sat, February 9th, 2013

Andres says:

An alternative solution has to be found…
I even gave the possibility of re-mounting all from scratch on the site and even that was not a possible solution. I need the server working, in worst case scenario by monday 07:00 GMT-4… in case it is not, I also might want refund and out…

Sat, February 9th, 2013

Derek says:

48 hours is a joke. That sounds like “we’ll work on it again on Monday.” As a professional company, there is no excuse for this terrible level of service. IX has exhibited no level of urgency for resolution of any kind. Besides the false representation of capabilities, the blather offered as technical updates is worthless. I regret that you were selected as our host provider.

Sat, February 9th, 2013

IX: Jared says:

Hello Derek,

I greatly apologize for the inconvenience. Our administrators are working on this issue diligently, and they will get it resolved as soon as possible. This is not them putting off the issue until Monday. Running even something as small as a File system check can take 12-24 hours on a large server such as this one. I assure you, we will work on this issue, and expedite it in any way that we safely can.

Sat, February 9th, 2013

Karen says:

Do you guarantee that our sites will be up as they were when the server is re-created? No data lost? I am really starting to stress over this.

Sun, February 10th, 2013

Ahmed says:

where is your high avalibilty for the servers, where is your redundancy, where is your SLA for up time, all of this not implemented, pls stop lies to the customers

if you have a proper backup strategies , this problems not happened any more, now your company reputation and confidence is dead to me and most of the people that used your services.
iam afraid now that my website not working and my data will lost.

Sun, February 10th, 2013

jenn says:

You still haven’t said anything to reassure us regarding the data. The server is back up. We’re all getting iis errors now. If there wasn’t a data problem, we’d all be up by now. I want somebody from IX to tell us that the data is ok. Because my guess is that they had a single point of failure, no backup, and they’re now trying to recover data from a bad drive.

Sun, February 10th, 2013

IX: Omari J. says:

Hello Jenn,

At this time all we can say is the server is going through recreation, so knowing if the data has been lost is not possible. All backups are left on our backup servers which are separate from the web servers. Once the server is back online and fully functional we will be able to give more information about data. Also please note there is no guarantee that we have backups of all your content. We do advise our customers to keep frequent backups of their own content stored on their local machines. You can find this information on our Terms of Service page here: http://www.ixwebhosting.com/terms-of-service. We will provide another update once we have more information.

Sun, February 10th, 2013

Karen says:

Oh my g**, so you’re saying all our data may be lost????

Sun, February 10th, 2013

IX: Omari J. says:

Hello Karen,

We cannot say if this is the case or not. When the server comes back online and you notice there has been changes, we can restore your domains with any disaster recovery backs that we have, but again I suggest you keep and maintain your own backups of your content just in case.

Sun, February 10th, 2013

Brian T. says:

Well. from the sound of that last update, we’ll be lucky to have our service back by the end of next week.

I guess then we’ll see just how much data was lost. I’m lucky to have a complete copy of my sites localy and data backups for them. So I’m not overly concernd. but others may very well be.

I getting a lot of complaints from my customers, and I’m sure the others are as well.

This has been so far the worst downtime I’ve ever had with a hosting company. I’m now starting the search for a new host, as I’m sure many of your clients are.

Sun, February 10th, 2013

Chirag says:

This is terrible. 1 week is too much… Who’s responsible for our loss of business?

Mon, February 11th, 2013

Lee D says:

What type of RAID is this? Why isn’t it Hot Swappable? The server should be able to stay up and running even with a failed RAID Drive!!!

Mon, February 11th, 2013

Derek says:

You need to post another update on your progress.

Mon, February 11th, 2013

jenn says:

We are waiting for an update.

Mon, February 11th, 2013

Davorin says:

We will appreciate an update.

Mon, February 11th, 2013

John says:

After 40 hours, it will be a full week since the service crash down. It took so long for this company to restore back their service. It means they really do not have any backup and data protection, they just keep recruit new clients and get money, but do not really care about existing customers. If you see how them keep high price for old customers and lower the price to attract new customers, you will understand this.

We really doubt this company’s professionalism, ability and caring mind to handle their customers.

We keep losing business and creating so bad image about our company because so long website shut down like this. Who takes this responsibility? Service up time and service quality are not provided as original contract promised. What we can do and who will be responsible for all these? We, all people who are hosted by IX, got think about this and should took serious action on this.

Mon, February 11th, 2013

IX: Chris B says:

Hello John,

I tried to pull up your account so that we could discuss this more with you personally in a ticket and possibly give you a call. If you are able to make a ticket or at least give us any information to pull up your account we can discuss this more as well.

We do appreciate our current customers and do everything that we can to make sure your sites are up and running. One of the main reasons this is taking so long is that we have made sure to maintain your data and restoring all of that data along with rebuilding the server takes quite a bit of time. And as mentioned this is not your fault and the responsibility does fall on us. Since the responsibility does fall on us we do want to try and reach out to any of those customers that are having a tough time with this downtime. We are more than happy to discuss any options with you, we just simply need to have your contact info/ticket so we can try to reach out to you and discuss this more.

Once again we do apologize for all issues caused by this downtime but we are working as fast as we can to make sure the data you had is maintained and properly put back in place with the correct permissions, etc. so that your sites do not have a bunch of errors once they are restored. Once we have more information about this and an update we will make sure to update this blog to let everyone know.

Thank You.

Mon, February 11th, 2013

Derek says:

Same update posted 2 days in a row. has any real progress been made? Ifo so, please post THAT update.

Mon, February 11th, 2013

IX: Chris B says:

Hello Derek,

Sorry for any confusion. We have just updated the progress for you.

Thank You.

Tue, February 12th, 2013

Kaveshan says:

I have been an IX client for many years now… Over the last year i have lost 50% of my business due to IX service issues… It is 8:27am in South Africa. My current clients have come to work AGAIN and not have emails. I have since lost a further 20% of my clients… IX wont help me get them back..

I have since decided to shut my business down and head for the unemployment line… Thank you IX for screwing me and my business over…

Tue, February 12th, 2013

duha-bridget says:

Of people are angry – but I am sorry you are getting so much flak. You don’t want this failure any more than customers do. Best wishes to all who are workig on the problem. It will be a salutary lesson if as i fear all our lovely site is lost. lets hope and pray. Duha

Tue, February 12th, 2013

Brian T. says:

So from what I understand of the last update we are still over 24hrs from the service being back up and running..

Thats just so sad.

Tue, February 12th, 2013

wael says:

We keep losing our business, Who takes this responsibility?

Tue, February 12th, 2013

jenn says:

It says that some of the sites have been restored. Are any of them accessible at this point? Because I would want to see a link to one as proof that all of our data has not been lost and that progress is actually being made.

Tue, February 12th, 2013

IX: Robyn M. says:

Hello Jenn,

Accounts are being restored but we will then have to restore permissions. The data is being restored from a separate backup server so it will not be lost.

Thank you.

Tue, February 12th, 2013

Derek says:

Please post another update with an updated ETA.

Tue, February 12th, 2013

Davorin says:

How can we see what content is restored? I can not connect to server via FTP.

Tue, February 12th, 2013

IX: Robyn M. says:

Hi Davorin,

Unfortunately you will not be able to see the restored content until the server is put back online once the entire process is finished.

I am sorry for the time this has taken.

Tue, February 12th, 2013

Derek says:

It has been a full week of inaccessibility. There has to be some base level of service that could be restored at this point, in order to allow your affected customers some measure to interact with their own customers. Your updates are incomplete an vague, bordering on techno-babble. You should be able to perform the remaining maintenance off-hours, and at least restore the accessibility to our domain pages to our customers.

Tue, February 12th, 2013

IX: Jared says:

Hello Derek,

Permissions cannot be restored until the data is restored. We have to make sure this process is completed in the most efficient and secure manner possible. In order to do that, we have to follow that rule. I greatly apologize for any inconvenience this has caused you. I can certainly understand the distress you can have from an issue like this one, and with your website being down so long. We will get this resolved as soon as possible. If there is any way to securely put the data up early, that is generally done with issues such as these, and our administrators will look into any and all possibilities.

Tue, February 12th, 2013

Derek says:

Your last update states ‘content’ is restored. If that is not data, then what is the content? Additionally, please stop posting ‘as soon as possible’ as part of the update.It not a reassuring comment at this point. I would think that after a week that would be a given, but more importantly details are what is really needed by your affected customers not generalities until service is restored.

Tue, February 12th, 2013

Elizabeth says:

WOW. This is unbelievable.

Wed, February 13th, 2013

Arafa says:

when try visit my website ask me username and password, please more clear.

Wed, February 13th, 2013

Brian T. says:

So we have lost our service for basically two business weeks now. Meaning we’ll be lucky if you have this back up by Friday.

Clearly 1 month free doesn’t even come close to repaying us for this extended down time.

Wed, February 13th, 2013

IX: Robyn M. says:

Hello Brian,

Please contact us via phone or ticket regarding compensation once the server is back online and we can look into the matter further for you.

Thank you.

Wed, February 13th, 2013

Karen says:

My clients are threatening to leave me – their web designer. Your service is reflecting so badly on me. Not only are my clients losing face, but I could face the reality of losing at least one very, very good client. I’m so upset by this, you guys have no idea!

Wed, February 13th, 2013

John says:

This is our client (a business partner) sent to us as they keep trying to visit our websites:


Dear xxx,

Hope you all well

Please check your web site as its not opened

We just start a connection with a new company to work with you and they want to see the web

thanx

This is business partner sent. But some direct customers who want to apply for our program even cannot connect to us because only we they can reach us is web form before they know our email.

Keep losing business everyday, we have an average number of signs on customers (a big $$$ of contract), not zero customers / day for 7 days and then same situation seems still going on.

This loss is a huge number of money and then plus company reputation!

Wed, February 13th, 2013

IX: Jared says:

@John,

Please create a ticket in your account if you would like to discuss this further. We would be happy to hear your feedback.

@Karen,

I am very sorry about the inconvenience this has caused you. Please believe this is very rare issue, and that we will do everything in our power to prevent similar issues in the future. Unfortunately, in this case the issue became a very large one.

Thanks!

Wed, February 13th, 2013

Karen says:

What are your plans for financial compensation for all of your customers effected by this?

Wed, February 13th, 2013

IX: Jared says:

Hello Karen,

Please create a ticket or give us a call once the server is back online, and we will discuss compensation.

Thanks!

Thu, February 14th, 2013

John says:

Yes, we just created a ticket, attached the messages we left on this page, and explained the issue.

the ticket number is
ID# 2195661

Look forward to the solution.

John

Thu, February 14th, 2013

duha-bridget says:

reading the blog it looks like some of your more needy cusstomers are likely to get ‘compensation’ what about those of us who have the luxury of just being able to wait??

Thu, February 14th, 2013

IX: Robyn M. says:

Hi duha-bridget,

As always we are extremely sorry it has taken so long. Once the server is back online please but in a ticket or give us a call and we can definitely talk about providing you with compensation.

Thank you.

Thu, February 14th, 2013

duha-bridget says:

Thank you. Don’t worry – i don’t want to sink the company.
D

Thu, February 14th, 2013

jenn says:

I presume at the very least that you will be offering full refunds for the remaining months of our contracts for those of us who paid annually? I would like to see you state in this thread exactly what your policy will be for all of us. I would imagine that most will be looking to move sites and will want a refund for unused months. It looks like you’re trying to deal with each user individually which strikes me as disingenuous to say the least.

Thu, February 14th, 2013

stwwk1950 says:

What are your plans for financial compensation for all of your customers effected by this?

Thu, February 14th, 2013

Mike says:

I just looked at the “Any-Time Money Back guarantee” as per the posting at 2:07 pm EST.

I CAN’T BELIEVE YOU WOULD THINK THAT REFUNDING THE REMAING MONTHS OF SERVICE THAT WE PAID FOR ALONG WITH A COUPLE WEEKS OF FEBRUARY COMES CLOSE TO COMPENSATING US FOR THE LOSS OF BUSINESS WE INCURRED THROUGH YOUR FAULT OF HAVING NO REDUNDANCY!

Thu, February 14th, 2013

Ali says:

Too much disappointed IX, Everyday new ETA …

Why you are not opening our FTP connections? at least we can have our websites files… As per your updates, You are on Permission restoration phase, it’s mean that you have created our accounts and restored our websites files; So Can you please stop your professional supporting for Permission restoration and give us access to our website files, even with errors?

What kind of compensation can bring back our lost businesses? one month free hosting? 100$ refund? … This is terrible and unbelievable experience in web hosting companies. Who’s responsible for our loss of business?

Please open FTP connections urgently before other things …

Thu, February 14th, 2013

IX: John R. says:

Hello Ali,

Again I’m sorry for the issues you are experiencing with iis517. Right now we are not able to provide FTP access to the server until it has been completely restored. Once the issue has been fully resolved for all customers, we will be communicating compensation for the downtime regrading the server.

Thank you,

John Richards

Thu, February 14th, 2013

Derek says:

I would an answer to the post I made on 2/12/13 at 6:57 pm EST.

Thu, February 14th, 2013

Brian T. says:

20 more hours for Permissions takes us to Friday at noon then how much longer to carry out the last two steps? Another 48 hrs each step?

That seems to be the current trend making the service down into next week!

Thu, February 14th, 2013

IX: James H. says:

@Derek

Sorry if your previous question was not answered. Yes, customer content is included in the data mentioned and yes customer content has been restored. Sorry for the confusion. If single customer points of accessibility become an option our administrators will inform us at that time. Otherwise we unfortunately must wait until the permissions pass is complete.

Thu, February 14th, 2013

IX: James H. says:

@Brian T.

I’m sorry, I know it certainly seems that way at times. We simply try to provide as close to accurate time estimates based off of restoration/completion percentage, progress, and any new information that may have changed our previous estimate. If that means that we have to update that the 24 hours estimate before will no longer be accurate, we would rather be upfront and honest about it as opposed to waiting the full 24 hours and give you bad news we’d already known for hours. Hopefully that makes sense and you will agree that this is preferable.

Fri, February 15th, 2013

duha-bridget says:

sufilovepath appears to be back!! hooray. i have yet to check it all as it is taking an age to download. but thanks. DB

Fri, February 15th, 2013

Brian T. says:

The site is back just no luck connecting via ftp as yet.

Fri, February 15th, 2013

IX: Kevin M. says:

@Brian T.,

I have just updated the status above. FTP service is not yet available. Our administrators are first focusing on data integrity, web services, and server stability. FTP will be brought online as soon as possible.

Thank you all for your patience. Our server administrators are working diligently to return this server full online status.

Fri, February 15th, 2013

yoo says:

I wanna know the actual problem which caused for the downfall of the servers more than a week.

It really sensitive matter which we have to rethink about the security of our files and especially the business we lost over the week.

Fri, February 15th, 2013

Eiad says:

My websites worked for a few hours and now stopped again, I want to explain why this?Please do not ignore answer to the question.

Fri, February 15th, 2013

IX: Kevin M. says:

@Eiad,

While our administrators were working on the server, they began to have difficulty communicating to the server. In attempting to isolate the cause of this, a number of services, including the web service, were disabled. Web service has been restored for now. Please be aware that until this blog post is resolved, we are actively troubleshooting and working on this server. We are cognizant of your sites’ availability, but sometimes it is necessary to take services offline to conduct repairs.

Fri, February 15th, 2013

duha-bridget says:

Thanks for the explanation above – i was worried – but site so far looks good – better than before – or is that just cos i haven’t seen it for so long??
Love Duha

Fri, February 15th, 2013

Scott says:

None of my sites that use a microsoft database works. Is that something you are working on still?

Fri, February 15th, 2013

IX: James H. says:

@ Scott,

I have reviewed all of the domains in the product linked to the email address you provided, and while I do note one domain that is not working currently it is giving a file not found error, not a database related error. Once maintenance is fully completed, please submit a ticket if you are still experiencing issues and we’ll be happy to troubleshoot further.

Sat, February 16th, 2013

Brian T. says:

The FTP servers have not been properly setup.
The Control panel is slower than molasess on a cold morning in January.

Please correct your ftp setup as only my master account is now working.

Sat, February 16th, 2013

IX: Robyn M. says:

Hi Brian,

We are looking into this issue and it should be resolved shortly.

Thanks.

Sun, February 17th, 2013

duha-bridget says:

I was on site this am(UK time) and all was ok but just tried to re-enter and you keep rejecting my password even though i am as sure as I can be that it is the right one.

Sun, February 17th, 2013

duha-bridget says:

It is OK now. It just doesn’t go in as slickly as it used to. sorry for my impatience.
DB

Sun, February 17th, 2013

IX: Jared says:

Hello Duha-Bridget,

While the initial shock of the server being back online has subsided, it should return to some very quick speeds, and have no future issues. If you notice any problems, however, do not hesitate to contact our chat, ticket, or phone support.

 
© 2011 IX Web Hosting.