Sam says:

Wow, time to move!!!

Link | Posted at 6:12 am EST

Bon says:

lets move out!

Link | Posted at 6:13 am EST

manish says:

yeah agree, time to move, they are having problem since continuous 3 days. pathetic situation

Link | Posted at 6:21 am EST

IX: Robyn M. says:

Hi Robert,

We are working to get everything up and get an ETA. All of our administrators are here and all working on this issue.

Thank you,
Robyn McKee

Link | Posted at 6:22 am EST

Kevin Shelley says:

I noticed something was up last night. Not just IXWebhosting but huge ‘chunks’ of the Internet.

You can’t really blame IX for this, except they should have their backup generators inspected.

No I don’t work for IX. Watching this space.

Link | Posted at 6:24 am EST

chirasthi says:

my clients eating meeee… can you.. how long it take to up the web site?

Link | Posted at 6:26 am EST

Allan says:

Yeah! An ETA atleast!

Link | Posted at 6:27 am EST

Bon says:

yep, good idea, status blog on other server or other location..

Link | Posted at 6:28 am EST

Mike says:

@Muneeb,your sites back online now?

Link | Posted at 6:29 am EST

Bill says:

Weekly generator tests. I do it for my company – start doing it for yours.

Link | Posted at 6:32 am EST

Peter says:

All my sites are still down & have been down for a long time now. Email servers also down. Totally dissappointing and unacceptable.

Link | Posted at 6:33 am EST

Sebastian says:

All my customers are affected by this. But we shouldn’t complain as much, I mean: in four years I’m a customer, this is the first time they have an outage as this. Please be patient people… These folks are very reliable.

Link | Posted at 6:34 am EST

Sam says:

Robyn,
Thank you for your updates.
But, seriously, this situation highlights how poorly IXWebhosting is doing:
1. Slow at times.
2. Fixes take a while to get solved.
3. Online chat support guys are don’t have much power to do things and then tickets are meant to be created, and those might take a while to get resolved.
4. Don’t seem to get newer features on the control panel… Limited options like file upload and unzip on file manager.
5. And now this! Power cuts happen, surely, but come on, having your generators fail as well? How poor is that?

I honestly think IXWebhosting has bitten more than they can chew!

Slim down, charge a bit more, but give top service!

This is catastrophic for many of us!

Link | Posted at 6:36 am EST

Kevin Shelley says:

Yep. I had a similar problem when I was at ebay. I can imagine what it’s like there, but sounds like you’re over the worst of it?

These things happen people. Life goes on . . .

Link | Posted at 6:38 am EST

Leonard says:

It seems I can email out, but I can not receive any email. My web is still down.

Link | Posted at 6:40 am EST

Phu Tran says:

Waiting …

Link | Posted at 6:42 am EST

Kevin Shelley says:

Mavdes.

Basically what’s happening is the staff are in the shop but the doors aren’t open yet.

Link | Posted at 6:44 am EST

Maya Aryan says:

@Sam you are absolutely right in all terms that you have provided …. I am 110% agree to them ..

Link | Posted at 6:46 am EST

iftikhar says:

now we are getting this error…..

Could not connect: Lost connection to MySQL server at ‘reading initial communication packet’, system error: 111

Link | Posted at 6:47 am EST

charles says:

this is the third time in the week, IX going down. let’s move to some other hosting…

Link | Posted at 6:50 am EST

Maya Aryan says:

You are right @Charles
It is really the time to move on …

Link | Posted at 6:53 am EST

ossi says:

thx Robyn

Link | Posted at 6:54 am EST

Hometakes says:

This is the error I am getting: Warning: mysql_pconnect(): Can’t connect to MySQL server on ‘mysql301.ixwebhosting.com’ (4) in /hsphere/local/home/c100802/Connections/hometake.php on line 9 Fatal error: Can’t connect to MySQL server on ‘mysql301.ixwebhosting.com’ (4) in /hsphere/local/home/c100802/Connections/hometake.php on line 9

Link | Posted at 6:57 am EST

Mavdes says:

anyone wana see the differences.. have a hosting with godaddy .. in similar situation they will accuse u that u made peoblems in ur hosting lol .. even whos says they r the best in hosting and and.. they r so much liars.. i know am frustrated now from this issue .. but am not angry yet i was so much angry at many companies before.. i faced too many problems with IX and the support managed to solve it sometime maybe took time.. but they solved it.. am still have hosting with 3 companies and slowly am moving to IX .. and yes i dun work in IX even almost am new here

Link | Posted at 6:59 am EST

IX: Robyn M. says:

Hi Hometakes,

You are getting this message because your mysql server is down. We are working to get it up as soon as possible.

Thank you,
Robyn McKee

Link | Posted at 7:01 am EST

Mavdes says:

plz Robyn .. can u solve web1418.opentransfer.com .. not working at all

Link | Posted at 7:05 am EST

Kevin Shelley says:

IX.

Just be prepared for when the floodgates open. There’s going to be a HUGE amount of traffic.

Link | Posted at 7:06 am EST

IX: Robyn M. says:

Hi Mavdes,

We are definitely working to get all servers including web1418 up.

Thank you.

Link | Posted at 7:08 am EST

Jameel Alayyan says:

I thought the world was ending. Thank goodness for your update.

Link | Posted at 7:09 am EST

Ramin says:

i am going to move my hosting to another platform. wtf service is that…

Link | Posted at 7:12 am EST

Muneeb Zafar says:

“right now our systems are placing emails into a queue until the mail servers are back up” as posted on twitter

Hope to see a resolution soon

Link | Posted at 7:15 am EST

Can Nguyen Van says:

so disappointed, no notice has been sent so far! all my effort for SEOs loosing now. Pls explain!!!!!!!!!!!

Link | Posted at 7:17 am EST

Kaveshan Chetty says:

Hi guys..

Ok so i seriously saw my ass this morning with clients and this down time… But then again in this world that we live in we have to be prepared for some failures at times right… These things happen and when they do its a learning curve.. So lets not all go and move hosters… i love IX and have been with them since 2009.. No issues or complaints except for today ofcourse but then again who wouldnt be pissed off.. 1 error doesnt make for me to move… Service has always been awesome…

I just hope this doesnt happen again…

Link | Posted at 7:19 am EST

Gareth Atkinson says:

This is a really frustrating issue and I’m in total agreement about how damaging it is to business.
But . . . I’ve been with IX for over 6 years now and have had almost no issues up to now. No denying this is a biggie but if you guys get it sorted quickly a lot of our ire will fade.
I agree with some of the other contributors in that I appreciate your honest updates. I was wondering what on earth was going on when my e-mails stopped sending and arriving and then noticed all my sites down. Your status update has been useful.
Learn from this and I for one will still be a happy customer. If something preventable like this does happen again though, you’ll have to expect a lot of clients to walk.
Good luck.

Link | Posted at 7:24 am EST

Kevin Shelley says:

This is pretty cool though. Without realising it, we’re creating our own Social Blogging Site. Let’s call it IXU.

Link | Posted at 7:28 am EST

abdul says:

hey,
morning still my server down. i cant do anything with this. i understand something happened. but atleast try to alternate my email server. this is very important. do u know today how much i lose..?

please act fast.

Link | Posted at 7:36 am EST

Dan Larry says:

Thanks IX support team. I’m sure you are doing your best. Your support is alwats great.
Good Luck !

Link | Posted at 7:37 am EST

Rajesh says:

NO, Still our IP is not getting ping. it means server is not started pls have a look : 93.130.0.160

Link | Posted at 7:38 am EST

Dan Larry says:

Thanks IX support team. I’m sure you are doing your best. Your support is alwats great.
Good Luck!

Link | Posted at 7:38 am EST

Edd says:

Ixhosting you guys really are a joke, I get problems with my sites every week!! I have 300 sites with you and have been a customer for 4 years. I have started migrating them i hope i never have to use you guys again

Link | Posted at 7:41 am EST

Kevin Shelley says:

Hi Rashid,

I completely understand your situation but I’m sure IX is just as frustrated. Afterall, it’s their business to and will be loosing just as much business as a result of this, if not more.

Rest assured, they won’t have their feet up, drinking coffee and throwing paper aeroplanes

Link | Posted at 7:42 am EST

Edd says:

Why are we waiting, because you suck. If your raid has failed and i’ve lost my sql databases I’m going to come visit you guys.

Link | Posted at 7:43 am EST

Abdulkareem Jabr says:

Thanks IX for your effort.
My site is up but not running because of database that is still not running.
This is the second time I feel bad about IX, First was arranged when you moved from city to another with planned outages.
Also, from my point of view (as a senior systems engineer) you are a big company and you have to move your data-center to a tier-4 data-center and to move (or backup) your status blog to another hosting even outside USA.
Again, Thanks for your effort and I hope this will be finished soon.

Link | Posted at 7:47 am EST

IX: Kevin M. says:

@abdul, we are working to bring all the servers online as fast as we can. Changing your account to a different server would actually extend your downtime while propagation occurs.

@Rajesh, that is not one of our Server IPs. That IP is leased to an ISP in Germany.

Thanks for your patience and support everyone!

Kevin M.
IX Tech

Link | Posted at 7:48 am EST

Peter says:

How many email servers are still down? Mine are still unreachable!

Link | Posted at 7:48 am EST

Jay says:

It’s really disappointed ah

Link | Posted at 7:49 am EST

Daniel Blanco says:

i hate Murphi!
I moved a small company to ix mail just yesterday…
now i look bad

db

Link | Posted at 7:51 am EST

Heidi Sutherland says:

Hi can you tell me when POP3 email accounts will be back working please?

Link | Posted at 7:51 am EST

ROXXITY says:

We have sent home our sales staff because they can’t work. We have invested about $ 5000 for ads in today’s magazines – gone for nothing!
And you lie to your customers and don’t answer calls…

Link | Posted at 7:52 am EST

Alfred says:

Hey guys, we need to give some kudos to IX, maybe for me, they have done superbly well so far. IX customer service is always available. How many website hosting companies really do this? Just for this once that I have really experienced this trouble wont make me move.

Though some of my clients are disturbed, please put in more efforts to get the issue sorted out ASAP.

Or do you need some Data Recovery and Backup Solution? I can be of great help if you need.

Link | Posted at 7:53 am EST

Andrea says:

Im sure IX will learn the lesson but i hope people (customers) will do as well.
If u just rely everything on internet u r missing something… We r expecting IX to give the best service but noone can prevent internet to stop working, servers to blow up, dns to die and so on. And u wanna base all ur business on this? Internet should be a way to make ur business better, not just ur business. Or if u do, at least expect to face these problems. Moving stuff somewhere else wont keep u safe from facing the risk of another blow up

Link | Posted at 7:54 am EST

miki says:

4 hours when there is no return to normal.
lets move out!

Link | Posted at 7:55 am EST

Edd says:

According to https://www.aepohio.com/outages there have been no outages.

Link | Posted at 7:55 am EST

Mavdes says:

web1418 still off (((

Link | Posted at 7:59 am EST

Bon says:

Oh my God, how long will it take?! its been hours now!

Link | Posted at 8:00 am EST

Jonathan Mann says:

[img]http://www.personal.psu.edu/nmc15/blogs/anthspace/images/01-frustrated.jpg[/img]

Link | Posted at 8:01 am EST

Hometakes says:

ANy idea when hometakesvideos.com/admin will be up as I cant work until its up and running? This is the error I get:

Warning: mysql_pconnect(): Can’t connect to MySQL server on ‘mysql301.ixwebhosting.com’ (4) in /hsphere/local/home/c100802/Connections/hometake.php on line 9 Fatal error: Can’t connect to MySQL server on ‘mysql301.ixwebhosting.com’ (4) in /hsphere/local/home/c100802/Connections/hometake.php on line 9

Link | Posted at 8:05 am EST

ROXXITY says:

Your phone was not available for about six hours and now you guys are lying to us.
Ohio media don’t know anything about electricity blackouts.
We have lost thousands of $ today for useless ads and the facts that we had to send home our staff because they can’t work because sites and E-mails are not working.
And you still lie to us! Poor!

Link | Posted at 8:07 am EST

Kevin Shelley says:

All my sites and emails now back up. Checked emails and guess what? All Spam

I suggest that people take this time to do something else. Read a book, take the kids to the park, go for a walk, it’s lovely out there. Yes, out there. The place you always wished you were whilst your stuck in that stuffy office. Phone that friend you haven’t spoken to in ages because you’ve never ‘had the time’.

Top job IX.

Link | Posted at 8:08 am EST

philippines says:

would it be too much to ask if occurences like abrupt internet service interuptions such as what happened be communciated to us via our cell phones?

Link | Posted at 8:12 am EST

Phu Tran says:

Please fix:

mail1306.opentransfer.com

Thanks a lot!

Link | Posted at 8:13 am EST

David Goldman says:

Still offßline completely ((

Link | Posted at 8:14 am EST

Kevin Shelley says:

@ mavdes.

There you are. You wouldn’t have done that if it weren’t for the outage. Who knows where it will take you. Life’s an adventure of unexplored paths. Have fun. Peace.

Link | Posted at 8:16 am EST

miki says:

please open our website
our friend wants to buy products from us
my domain: goshopbrand.com

Link | Posted at 8:18 am EST

Jim says:

??????????????

Link | Posted at 8:21 am EST

Tom Flowers says:

I appreciate this status page. My website is still down, but I know y’all are doing your best because I’ve been with you for years; your service has been exemplary — even in extraordinary events such as this! Thank you, IX!

Link | Posted at 8:23 am EST

richard says:

when will the server be open??

Link | Posted at 8:25 am EST

harshal says:

hi i am also suffering from down time all my websites are down…. this is not what we expect from ix.. its been a 3 years now i am with you but now i feel you are degrading in terms of quality…

Link | Posted at 8:27 am EST

f leinbach says:

Well my sites are up but still waiting on Email. I have been with ixwebhosting for over 6 years and this is the first outage I have ever experianced. I’m sure you’ll have everything back in short order from the looks of it. I will stick with you as I feel I have gotten excellet service.

Link | Posted at 8:28 am EST

qiuyc says:

my web still not available

Link | Posted at 8:29 am EST

Robert says:

Hurry up! My site is http://www.dailyyoga.com

Link | Posted at 8:31 am EST

philippines says:

how about CP13?

Link | Posted at 8:32 am EST

mahmoud says:

that’s very very very very disappointing .
it’s not the first time, surely it won’t be the last one.
seems we’ve selected an unstable host.
so where’s 99.99% uptime?

Link | Posted at 8:35 am EST

Oliver says:

How come their websites are not down???????

Link | Posted at 8:36 am EST

Bon says:

@robert because of this delay am afraid you will change it to weeklyyoga.com haha.

Link | Posted at 8:36 am EST

Kevin Shelley says:

A little tip for all those who’s website is still down. Say something nice. Try it, it works.

Link | Posted at 8:37 am EST

Eran says:

To be honest: IX do have the best customer support I know ,
If they only could make maintenance better ..

Link | Posted at 8:37 am EST

Eran says:

mysql509 is still down

Link | Posted at 8:38 am EST

Edd says:

Any news on mysql405

Link | Posted at 8:39 am EST

ROXXITY says:

99.9% availability over 700 million years?
That’s the true and not the promise you make.
I am looking forward to the comment of the BBB and I am still waiting for John Richards’ return call with new funny stories about power outages that nobody in Columbus, Ohio noticed. Of course out of ixwebhosting.com!

Link | Posted at 8:42 am EST

At Velev says:

I’ve got two servers. The more recently purchased one is now up, while the older one, from 6 years ago is still down. Do I assume older customers are less important, or is it just that the newer servers are better? Either way, I think after 6 hours with no email and clients starting to sharpen their knives, I can’t afford to risk this happening again. Sorry IX, as one of your older customers I agree that you’ve been great, but you’ve let yourselves go.

Link | Posted at 8:44 am EST

IX: Lisa Grice says:

Okay everybody, we’re trying to answer everyone and get updates as fast as we can. We’re being as transparent as we can, however, we will remove anything that is false or inappropriate.

Lisa

Link | Posted at 8:45 am EST

Edd says:

Its all to do with George Bush dude not Obama. Remember building number 7!!

http://911research.wtc7.net/wtc/attack/wtc7.html

Seriously though I do believe it was a power cut. But the service has been shocking

Link | Posted at 8:48 am EST

Mavdes says:

pleaseeeeeeeeeeeeeeeeeeeeeee web1418.opentransfer.com is still off (((((( pleaseeeeeeeeeeeeeeeeeee my clients eating me ((

Link | Posted at 8:49 am EST

Aamer Rehman says:

It’s really disappointing that no one is giving the clear statement.

“Server is up but with the exception of file system check”. How long it will take actually??????

Link | Posted at 8:49 am EST

Kevin Shelley says:

@alessandro.

Yeah, it’s the Aliens maaaan.

Link | Posted at 8:51 am EST

David Goldman says:

@Kevin Shelley

No, it does not.

Link | Posted at 8:52 am EST

Edd says:

I love you ixhosting you are the greatest host ever as you give me so much free time. Ok fingers crossed come on mysql405.

Link | Posted at 8:54 am EST

f leinbach says:

My email Just came back. I’m very happy with the service. As for the backup generators not working believe me I’m very aware of that happening. It happend to me 2 years ago. My generator self tests every week but when the power failed the generator came on as expexted but produced no power. S it happens.

Link | Posted at 8:54 am EST

Alessandro Ituarte says:

@Edd 5 hours! that’s quite a lot.
I can only keep on fscking for like, 5 minutes.

Link | Posted at 8:58 am EST

Edd says:

Yeah before you can fsck it you have to unzip, and check thats it clean. Then begins the hours a fscking. It a really pain in the ass (for some)

Link | Posted at 9:01 am EST

udeme says:

At Velev,
I agree. We older customers seem to be the last to get reenabled. Very poor form IX

Link | Posted at 9:02 am EST

Brian says:

Nice updates every 30 minutes. Been almost an hour now. We have customers affected by this that are asking questions. You may not care but we do!! Very Unprofessional

Link | Posted at 9:06 am EST

IX: Lisa Grice says:

@Brian, two large updates posted just now

Link | Posted at 9:07 am EST

Bon says:

oh no!! am at web1419!! and am included in the “except” whew!

Link | Posted at 9:10 am EST

IX: Kevin Nelson says:

David,

You are on web602 and mysql506. While your MySQL server is up your webserver is still currently offline.

–
Kevin N

Link | Posted at 9:15 am EST

miki says:

oh no!! am at web1419!! and am included in the “except” whew!

god!!!Also how long it takes？

Link | Posted at 9:16 am EST

Daniel Blanco says:

mail1303 is down !

Link | Posted at 9:19 am EST

Armin says:

Poor CP11
It’s the poorest server on earth :’(
My site will be the last one that restores!!!

Do you know where exactly all cp11 will back again?

Thanks IX guys

Link | Posted at 9:22 am EST

IX: Lisa Grice says:

@Dennis, what is your domain name and we can find your server names

Link | Posted at 9:25 am EST

Daniel Blanco says:

it is friday the CP13 for me…

Link | Posted at 9:26 am EST

Bon says:

i just took a back up of my SQL haha (better than sorry)

Link | Posted at 9:28 am EST

ade says:

Thanks our site is back up…..Good work and response time

Link | Posted at 9:29 am EST

Alves says:

how do I find out what my server name is?

Link | Posted at 9:31 am EST

IX: Lisa Grice says:

@Alves, what is your domain name?

Link | Posted at 9:32 am EST

hurry says:

when will be back?!

Link | Posted at 9:33 am EST

Satheesh says:

So your hosting company has no backup power ?

Link | Posted at 9:33 am EST

Alves says:

my domain name:
http://www.alvesfilho.com

Link | Posted at 9:36 am EST

Aamer Rehman says:

CP 1306 is not responding till yet

Link | Posted at 9:37 am EST

Pierre says:

It’s obvious IX does NOT

Link | Posted at 9:38 am EST

Kathy B says:

Hi – can you tell me the server name for my website(s)?
http://www.hillsidewebsolutions.com or (more importantly)- http://www.beardsleewebwork.com

Link | Posted at 9:38 am EST

IX: Lisa Grice says:

@John, so far no data loss- but we’re not saying anything out loud and we’re knocking wood repeatedly.

@Alves – your servers are web307 mail308, mail312
and mysql304

@Satheesh – power has been back for a while, many servers are in fsck which takes a lot of time

Link | Posted at 9:40 am EST

Brian says:

Anybody know of a good hosting company that is professional and actually has a back up system?? (what a concept)

Link | Posted at 9:41 am EST

IX: Lisa Grice says:

@Sherry, you are on: web1306, mail1301, mysql1302
and pgsql1301

Link | Posted at 9:42 am EST

Sharafat says:

We are being informed that services will be up in next half an Hr, but it’s now many Hrs. When mail1306 will be up?

Link | Posted at 9:43 am EST

Sreekanth says:

Lost, 50 $ in adsense in past 5 hours…. Wow….

Link | Posted at 9:45 am EST

Sreekanth says:

I am in for a big time if there is db loss…. i will be down and out….Please dont say u wont have backup’s…

Link | Posted at 9:47 am EST

hurry says:

……………………………….

Link | Posted at 9:48 am EST

Pam says:

update on Cp13 please? Mysql1304 and mysql1306

Link | Posted at 9:48 am EST

Bon says:

my f5 key (refresh) button is almost broken lols! keep on pressing since 6am.

Link | Posted at 9:49 am EST

Edd says:

@IX: Lisa Grice I will make backups as soon as I have access, is there anyway that this can be automated?

Link | Posted at 9:50 am EST

Gary says:

any update on mail1308

Link | Posted at 9:51 am EST

speed says:

please HURRY UP!!!!

Link | Posted at 9:52 am EST

Satheesh says:

ss

Link | Posted at 9:52 am EST

IX: Kevin Nelson says:

Kathy,

You are on web1403 and mysql1407.

–
Kevin N

Link | Posted at 9:53 am EST

IX: Lisa Grice says:

Everyone – almost all servers are powered and booted, many are going through file system checks, which can take an hour or more and are difficult to estimate. We are updating server lists above around every 15-20 minuts

Link | Posted at 9:54 am EST

Edd says:

search.za.com is one of our domains

Link | Posted at 9:54 am EST

speed says:

faint!

Link | Posted at 9:55 am EST

IX: Lisa Grice says:

@Dennis, you’re on web1109, mysql1118

Link | Posted at 9:57 am EST

IX: Lisa Grice says:

@Pierre – NS servers are nameservers and those are up, your web and database servers are separate. What is your domain and I can check for you?

Link | Posted at 9:58 am EST

IX: Lisa Grice says:

@ Sreekanth, so far we do not have any RAID or harddrive failures, but we can never guarantee no loss of data on individual shared accounts – we only snapshot the server, which takes days, and do not back up individual accounts in real time. Shared hosting does not include that, so all Shared Hosting customers are encourage to make their own.

Link | Posted at 10:00 am EST

Jimmy says:

Yes, me too, still DOWN…. DOWN…i am having a big order through website. Now you’re putting me into risk.

When down, incoming email still can receive? Or all down? OHHHHHHHH… sooo frustrate

Link | Posted at 10:02 am EST

Dennis says:

@Lisa …. Of course the one that is still not up.. jesus.. this is ridiculous. I noticed that anyone that got newer servers in the last couple of years were up pretty much in a couple hours. You would think you would be interested more in the customers that have been with you for years.

I have some servers on your newer ones that were up by 6:45am.. So why does mine takw more than 3 more hours than those servers?

Link | Posted at 10:05 am EST

jhonny alexander caballero mestra says:

Hi, can you tell me what server I am on please?

http://www.asoproagros.org

Link | Posted at 10:06 am EST

Sharafat says:

Pls make mail1306 up? our Email domain @descon.com Eamils are not working?

Link | Posted at 10:10 am EST

IX: Kevin Nelson says:

Jimmy,

Our spam filtering hardware, which is up and running, will queue email that is incoming while your mail server is down and deliver it once the server is reachable again so you should not lose any email.

For reference you are on web1317 and mysql303.

–
Kevin N

Link | Posted at 10:11 am EST

caps says:

Hi, Kevin

web1126 web1428 web1420 web1411,
please back these severs.thank u..all down..

Link | Posted at 10:13 am EST

Jimmy says:

I have to say your support is still good. But really hope all incoming Email (new) is there.

Link | Posted at 10:14 am EST

IX: Lisa Grice says:

@Jimmy, thank you, we really do feel terrible and are hustling. As far as we know email bounced back to sending server while our gateway was down, and those servers should resend after a wait period. Once the gateway restored, all mall was queued and will deliver to individual mail servers as they restore.

Link | Posted at 10:18 am EST

Eran says:

Can any of you guys tell me what’s going on with mysql509 ?

I suffer a loss of customers and money every minute…

Please help !

Link | Posted at 10:19 am EST

Mavdes says:

please web1418.opentransfer.com is still off from more than 8 hours pleaseeeeee am losing my clients

Link | Posted at 10:21 am EST

Dorothy says:

@ Kevin N Thank you!

Link | Posted at 10:23 am EST

f leinbach says:

I once ran my own web servers with 3 differant ISP’s since the mid 90′s. Two of them Comcast and Verizon two of the biggest companies aroud lost service for several hours not once but several times during that period. So to put it in perspective I think Ix is doing great job. You can lose up time with companies far larger then them.

Link | Posted at 10:25 am EST

Mark says:

1307 doesn’t appear to be back up. My website and FTP at web1307.ixwebhosting.com is still down

Link | Posted at 10:25 am EST

IX: Kevin Nelson says:

Jhonny,

You are on web311 and mysql303. Both of these servers are up, however our administrators have noticed an issue with the mysql service itself which is being looked into now. You should be up and running soon.

–
Kevin N

Link | Posted at 10:29 am EST

f leinbach says:

Mike Z
It’s not possible to redirect email through the working servers if your records ar on a differant one.

Link | Posted at 10:30 am EST

Nitin says:

Hosting Control Panel Access Issue – Need to access urgently

Please resolve asap … really frustrating

HTTP Status 404 – Servlet psoft.hsphere.CP is not available

type Status report

message Servlet psoft.hsphere.CP is not available

description The requested resource (Servlet psoft.hsphere.CP is not available) is not available.
Apache Tomcat/5.5.16

Link | Posted at 10:30 am EST

miki says:

All CP14 linux servers are up except: web1401-web1407, web1409-web1430

why?pls。。。。。。。。。。。。。。。。。。。。。。。

Link | Posted at 10:31 am EST

Edd says:

Kevin Nelson: when you say an issue?? what does this mean? any data lose?

Link | Posted at 10:32 am EST

IX: Lisa Grice says:

@ Sreekanth – I can make a guess, but it would not be based on any facts. FSCK checks are almost impossible to estimate.

Link | Posted at 10:34 am EST

Mavdes says:

All CP14 linux servers are up except: web1401-web1407, web1409-web1430 >>

web1418.opentransfer.com is offline ((((((((((((((((((((( from more than 8 hours ((((

Link | Posted at 10:35 am EST

Bruce says:

@My Dear Lisa Grice
web1416
mysql1417
please back these severs.thank u..all down..
thankkkkkkssssssss,i beg you

Link | Posted at 10:36 am EST

Jim the Engineer says:

Starting to receive some emails,
from 9:51 AM EST to now.

Link | Posted at 10:38 am EST

Edd says:

Its says. All CP7 linux servers are up except: web406, web410, web422 What about mysql405!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! i’m dying here dudes!!!!! 7:30 hours ………………..

Link | Posted at 10:40 am EST

Ash says:

Hi,
This is a very unfortunate situation, i know your doing your best but its quite unacceptable to have no service for almost 10 core hours of the day. Just wondering if you have a disaster recovery plan in place?
Ash

Link | Posted at 10:41 am EST

Edd says:

Also are you using raid with redundancy?

Link | Posted at 10:42 am EST

Jimmy says:

If this come back, you have to tell All of us:

1. What had happened exactly on this.
2. How to compensate it, what’s the gaurantee you guarantee 99.9%
3. How you will prevent to happen it again. List in step you will do.

I am your customer for 5 years already. I am not interested to change but I must regain the trust back. And i suggested alot of new customers for you in the pass.

Link | Posted at 10:42 am EST

Fawad says:

All CP13 linux servers are up except: mail1301, mail1302, mail1305, mail1308-mail1320, web1301-web1317, web1319-web1337

I think mail1306 is still not working. Please update.

Link | Posted at 10:44 am EST

Armin says:

CP11 fsc was 65% at 8:12 AM!! Would you please tell me an estimated time? Why these servers are taking too long?
My host is on web1114

Link | Posted at 10:45 am EST

Sreekanth says:

@ Edd… till now no one posted here, they had any data loss… Lisa can u give us some support here…

Link | Posted at 10:46 am EST

Lex says:

Still down – hurry up!

Link | Posted at 10:47 am EST

Lex says:

hey don’t rush – we want ZERO data loss … please

Link | Posted at 10:48 am EST

Andrew says:

It is unfathomable that back up power for a 24×7 service oriented company failed within 24 hours. Are you guys not testing the equipment on a regular basis?

I really like IX Webhosting but c’mon! Electric Backup is like #1 issue you need to be on top of. Having the power go out on my internet hosting service is just really, really scary.

CP11 is still out to lunch as I write this.

Link | Posted at 10:49 am EST

Chuck says:

I am currently unable to access the MANAGE section of my account. my website cornerofgrey-street.com is still down. I understand you’re having server issues due to the power outage so I am cool for now in waiting for my site to be back up; but why I can’t I access the MANAGE section once I log into my account?

Link | Posted at 10:51 am EST

Edd says:

I’m on CP7 as well and mysql405 is not up yet. So its not only you Sreekanth

Link | Posted at 10:52 am EST

David Goldman says:

As I may see my CP8 web602 is still down. This is already looks like a real problem.

I think IXhosting should seriously consider to discuss a reward for disappointed customers. Maybe a bonus month or two at least.

Link | Posted at 10:53 am EST

IX: Lisa Grice says:

@Jimmy, this is our first power loss in 3.5 years – I promise that we are taking all steps to investigate and resolve the failure.

@Lauren, yes, this will be fully resolved today. Our hope is within the next couple of hours, but some individual servers could be longer if we have any hardware failures (so far we do not).

Link | Posted at 10:59 am EST

Gary says:

mail1308 seems to be back up and running, along with websites on my account.

Link | Posted at 10:59 am EST

caps says:

web1126 web1428 web1420 web1411

please

Link | Posted at 11:01 am EST

akuamedia says:

Phil, I’m with you and in the same situation.
This really sicks…

I’m talking with some other ISP right now. Obviously, IXWH is not in the top of service providing… (mode ironic: off)…

Link | Posted at 11:02 am EST

Ash says:

What about madinainstitute.org please?

Link | Posted at 11:03 am EST

IX: Lisa Grice says:

@Lauren – what is your domain?

Link | Posted at 11:05 am EST

IX: Kevin Nelson says:

Edd,

Once the server went up we did noticed some problems with mysql405. This will be getting its own status blog post as this server will need to be recreated This post will be up soon.

–
Kevin N

Link | Posted at 11:07 am EST

Lauren says:

http://www.chambertheatre.com

Link | Posted at 11:08 am EST

Phil says:

I’m with Jimmy.

I expect a response. Not a canned, “we’re sorry, it won’t happen again”.

I’ve been with IX for longer than 5 years. Off the top of my head, I believe it has been about 8 years.

I grew to rely on your service, and trusted it enough to use it for my businesses. I took a big $$ hit today because IX wasn’t prepared…which is what I pay for. There are other hosting companies out there who cost less.

I am going to need answers. Don’t email them to my IX hosted e-mail accounts….I’m not sure that I’ll get them that way.

Link | Posted at 11:09 am EST

Bon says:

5 hours and counting!? we’re still down…

what’s up IX people!! are we still doing good business here?

Link | Posted at 11:10 am EST

caps says:

web1126 web1428 web1420 web1411

Link | Posted at 11:11 am EST

Pam says:

status of web1327? Thanks

Link | Posted at 11:15 am EST

Chuck says:

my site and MANAGE sections are still down. ETA?

Link | Posted at 11:21 am EST

IX: Tiberiu Ungureanu says:

@PAM: web1327 is going to be up in within the next 10 minutes.

Link | Posted at 11:21 am EST

Edd says:

Any news on mysql405??????

Link | Posted at 11:23 am EST

Mavdes says:

what about web1418 ???

Link | Posted at 11:24 am EST

caps says:

@Tiberiu Ungureanu

Great Thanks!!

web1411,web1420,web1428

Link | Posted at 11:26 am EST

Edd says:

Any news on mysql405?

Link | Posted at 11:29 am EST

IX: Lisa Grice says:

@Edd – bad news, please see: http://status.ixwebhosting.com/2012/06/29/mysql405-serverrecreation/ This will take a long time I’m sorry, there are 24TB of data on this drive. I will try to get a date on the backup they’re using for you.

Link | Posted at 11:31 am EST

Kip says:

Any updates if “mail501.opentransfer.com” is up and running?

Link | Posted at 11:33 am EST

IX: Lisa Grice says:

@Ash and Kip – we’re still under fsck for those

Link | Posted at 11:35 am EST

Bon says:

IX: Lisa Grice

how about web1419….

Link | Posted at 11:36 am EST

Mavdes says:

@Lisa Grice please what about web1418.opentransfer.com

Link | Posted at 11:36 am EST

Ash says:

@Lisa,
Thanks for the update!

Link | Posted at 11:36 am EST

Mohammed says:

Hi all my website is down because A network-related or instance-specific error occurred while establishing a connection to SQL Server

Is this same problem and when will be solved?

Link | Posted at 11:38 am EST

craig dobbins says:

Back online for now. Email is still slow.

Link | Posted at 11:40 am EST

IX: Lisa Grice says:

@Mavdes – web1410-web1425 are in the ‘except’ list, that server is down

Link | Posted at 11:43 am EST

Mavdes says:

@Lisa .. sites not opening and cp14 also dun open for web1418

Link | Posted at 11:48 am EST

Abdulkareem Jabr says:

I think CP6 is still having problem with the Control panel:
HTTP Status 404 – Servlet psoft.hsphere.CP is not available

Link | Posted at 11:50 am EST

Mohammed says:

What about mssql401

Link | Posted at 11:55 am EST

Tom Flowers says:

My site’s back up! Thank you! =)

Link | Posted at 11:55 am EST

Mavdes says:

@Lisa

All CP14 linux servers are up except: web1401, web1403-web1407, web1410-web1425, web1427-web30

web1418 is not int he except list

Link | Posted at 12:00 pm EST

jack says:

@Lisa Grice About web1405?

Link | Posted at 12:01 pm EST

Mavdes says:

oops 1410-1425 means from 1410 to 1425 lol omg i even stopped reading well

Link | Posted at 12:01 pm EST

Sreekanth says:

@Lisa… any news about mssql401?

Link | Posted at 12:02 pm EST

IX: Lisa Grice says:

@Sreekanth – that server is up

Link | Posted at 12:05 pm EST

Gibby says:

Been with you guys 2 days and this happens, but your service seems quick and honest, so I am happy to see this through, as I am sure you guys will be helping me out lots in the future

Link | Posted at 12:08 pm EST

IX: Lisa Grice says:

@Mota – you should be up

Link | Posted at 12:13 pm EST

caps says:

still web1411,web1420,web1428

Link | Posted at 12:17 pm EST

jack says:

@Lisa I’d like to know the approximate time? 15 minutes or 5 hours? Because we really etc so long.

Link | Posted at 12:17 pm EST

IX: Lisa Grice says:

@All – I am handing over comment responses to the Customer Relations team so I can attend to some other things. We will keep updating until this is all complete.

Link | Posted at 12:22 pm EST

Robert says:

Any word as to what is going on, its been six and a half hours since they went down. There hasn’t been an update in almost an hour. This is costing me time and money.

Link | Posted at 12:25 pm EST

Aurelio Martinez says:

Been with you guys for many years, and I must say this past year has been terrible. Just don’t know what to say…

Link | Posted at 12:27 pm EST

Angel Parrales says:

Been with you since 2005 I can tell you guys provide a superior service. Not going anywhere else.
Your response is quick and honest as usual, situations like show the commitment to your customers.

Link | Posted at 12:28 pm EST

Samuel Ling says:

I hope everyone can be patient with the IX team. I am sure that they are doing their best to resolved the situation. Instead of complain, please give them the encouragements. It is not a situation that they expect and they want to be in.

let the team concentrate to get everything up first and you can complain later.

I am sure many of the team have already missed breakfast and lunch trying to stabilize the situation.

Link | Posted at 12:32 pm EST

jack says:

I agree with @ Samuel said.

Link | Posted at 12:34 pm EST

Dave says:

Maybe if they spent more time keeping things upto date, like a good UPS they would not miss their breakfast.

IX is a failure.

Link | Posted at 12:37 pm EST

IX: Chris B says:

@Harriet – I can assure you that as of right now the credit cards are not reported as being compromised in anyway. This was a power failure that caused all servers to go down.

Link | Posted at 12:43 pm EST

Samuel Ling says:

Dave, any UPS will at most provide backup for 60 mins. With the commercial power melt down as mentioned earlier by Lisa, things may not be as simple as what we think!

Link | Posted at 12:44 pm EST

caps says:

still web1411,web1420,web1428

please

Link | Posted at 12:51 pm EST

Sreekanth says:

IX tea, cannot see any update “CP7 Windows server mssql401″.. Any progress?

Link | Posted at 12:57 pm EST

Sreekanth says:

sorry… Typo “IX team, cannot see any update “CP7 Windows server mssql401″.. Any progress?

Link | Posted at 1:00 pm EST

IX: Chris B says:

@Dustin – I am sorry you are still seeing that timeout. I just checked again and even checked your a proxy server to get outside of our network here. The sites still came up for me. Can you please recheck once more using a proxy server and let me know if anything comes up?

@Sreekanth – As of right now the mssql401 should be up and running. I loaded up a couple of your sites and was unable to see an issue at this time. If any of your sites are down please let me know which one and I will look into it.

Link | Posted at 1:01 pm EST

Chuck says:

i am still down…..IX please provide ETA

Link | Posted at 1:03 pm EST

CJGzr says:

I’ve been with IX since Oct 2004 and I have to say they are 1000% better than they have been. I like this new transparency -the status blog. It’s much better than the past when they either denied having a problem or mentioned it later. I’ve also seen customer service improved A LOT since the early days.

Yes, 3 years ago they said they would fix the problem and it wouldn’t happen again and maybe that is the case. This might be a new problem or a new wrinkle on the old problem.

I worked in electronics in the Marines and we had what we called FM’s ‘frackin mysteries’ (well, not frackin but you know…) There are times when you take apart something and put it back together and it works… an FM. I was trained in 29 palms ca, in the middle of the frickin Mojave dessert, our classes were from midnight to 8 am -because of the heat. Air conditioners dying and power outages were common. But, we were there for that reason. You guys should hire a Marine ground radio repair instructor or technician from one of these kids coming home now. Marines repair until it flatlines. We were expected to understand that when it sucks, it pours and people die, so think on your feet and anticipate every cascading series of WTF problems.

That said, if you’ve been having problems more often lately due to the heat and your UPS and generators have been handling it. Maybe they have but htey also might be over stressing their capabilities. You might be depleting the UPS charges and not fully recharging or components are getting worn and holding minimal charges preventing full recharging etc. Annual checks work for 90% of problems -it sounds like the recent heat wave has placed extraordinary burden on back-up equipment. Maybe, rotating out 30% for preventative maintenance and full massage recharging might be useful.

Then again it could be an FM. Good luck. I’m not going anywhere.

Link | Posted at 1:05 pm EST

Sreekanth says:

@Chris B … Thank you…. site up… No DB loss aswell…. Great… Thanks a bunch

Link | Posted at 1:08 pm EST

caps says:

still web1411,web1420,web1428

still down..gosh

Link | Posted at 1:09 pm EST

IX: Chris B says:

@Jose, caps, Bon – I am very sorry for the issue still persisting as it is. We are working as fast as we can to get all servers up and running and to ensure no data loss, etc. Please bear with us on this as we get everything back up.

Link | Posted at 1:12 pm EST

Bon says:

i almost forgot how my website looks like! missing my site. =(

Link | Posted at 1:13 pm EST

Phil says:

So far I am happy with the answers I’m getting through the trouble ticket process.

I’m going through my sites now, so far everything is working properly and no data loss.

E-mails appear to be working ok, but all e-mails from 2305 PST to 0753 PST appear to be forever lost.

I can’t quantify how much business or $$ was lost.

I can think of a few ways to prevent a 8 hour interruption in the future….let’s hope you guys do to. It’s hard to predict every possible thing that could go wrong…but it isn’t hard to come up with solutions to minimize down time in the future. It almost appears as if IX didn’t know their servers had failed (UPS batteries being used….and dying) until they came into work this morning. How is it possible for all your servers to go down without any alarms being sent??? As smart as you guys are….I’m sure this could have been prevented.

Within the next few days I’ll be expecting to hear exactly WHAT will be done to prevent this from being so catastrophic in the future.

Stuff fails, but you’ve got to get it back up in a timely fashion.

Link | Posted at 1:15 pm EST

Eran says:

Thank you guys , although it took a lot of time , I understand that it was neccessery to run all these data checks before running up again ,

finally , here in Israel the catastrophy went on from 12:30 to 20:15 … I was hooked to the computer and refreshing every 5 minutes..

I’m staying with IX but hoping you will find a fair way to compensate the customers according to the time we’ve been down.

Anyway friends – BACKUPS are a CRITICAL thing !

Hope we’ll all meet on a better circumstances , weddings etc`…

Link | Posted at 1:19 pm EST

Jeff says:

I was told my site would be looked at 4 hours ago and I haven’t heard anything and the site is still down. Any updates please??????

Site Name: http://www.diversitysalesjobs.com

Link | Posted at 1:22 pm EST

Sreekanth says:

@Eran…. Totally agreed…

Link | Posted at 1:25 pm EST

Bon says:

though i’ve waited 7 hours.. and am up now.. thank you for all the efforts and updates!

Link | Posted at 1:42 pm EST

Jim says:

CP1413 is back online.

Link | Posted at 1:45 pm EST

Mavdes says:

web1418 back online thank u alot .. hope it keeps working .. and NO ill not shift to other hosting

Link | Posted at 1:48 pm EST

Mayoflor says:

One of my sites, we’re being moved to another server when the power failure put all down…

IIS1015 is up from some hours, but my site Mayoflor.com is showing an error page.
The tickets I have open only tells me you are working on… And I know.

BUT, that it means my website is going to be up the last one just because it’s marked as “moving”????

Why don’t just put the domain in its original situation as the server is up and running?
I have some other domain in the same machine and it goes like a charm… Why not Mayoflor.com ?

I can not understand…

Link | Posted at 1:51 pm EST

Candy says:

web1410 still down

Link | Posted at 1:55 pm EST

IX: Chris B says:

@Art – I am sorry about that. We have created a ticket to notify our VPS department about this. Please look to the ticket for more updates and information.

Link | Posted at 2:02 pm EST

caps says:

web1411,web1420
still down.please……

Link | Posted at 2:09 pm EST

caps says:

web1411,web1420
still down.please……

!

Link | Posted at 2:22 pm EST

Jose says:

Linux web1321 still down for me

Link | Posted at 2:30 pm EST

IX: Angela says:

@caps–I’m so sorry that these servers are taking longer than others to get back up and running. I assure you the admins are working very hard on getting them back online for you.
@Sheena–I have checked and your account resides on iis500cf, which our Windows admins are still working on getting back online. This server now has it’s own blog post at http://status.ixwebhosting.com/2012/06/29/iis500-server-recreation/ so please refer to that for more info! Please also let us know if you have other questions or comments for us.

Link | Posted at 2:33 pm EST

IX: Kevin Nelson says:

Jose,

The website you provided which is on web1321 is up for us. If you are still experiencing issues with this or another domain please let us know.

–
Kevin N

Link | Posted at 2:42 pm EST

Jose says:

@IX: Kevin Nelson – Everything working now, I was just a little worried that one hour after your “All CP13 linux servers are up!” message the site still was down.

Thanks guys for your work and updates.

Link | Posted at 2:53 pm EST

David Goldman says:

@IX: Angela

OK, I’m busy with the status blog for the whole day, it’s a kind od desperate experience until now, I would say

Link | Posted at 2:54 pm EST

IX: Angela says:

@Anthony — We are so very sorry that it’s taking longer to get the servers that you are located on back up and running. Our admins are working very diligently on getting everything back online. We weren’t able to move everyone to an internal working server because, unfortunately, our entire data center was down. However, you should be receiving another update via this status blog very shortly. Please let us know if you have other questions or concerns Anthony.

Link | Posted at 3:21 pm EST

Nicole says:

web602?

Link | Posted at 5:04 pm EST

Nicole says:

So it’s not that it has a specific problem, but that it happens to just be dead last?

Link | Posted at 5:43 pm EST

Brian says:

Although there is a separate post for iis500cf, it appears that no one is monitoring or responding to comments, so I’ll ask the question here as well:

This outage has now gone on for 12+ hours, and is severely impacting a number of our clients. Please provide an update on the status. Thanks.

Link | Posted at 6:33 pm EST

Alvin Frizzle says:

It’s good that we are back online again, but I do hope this wont happen again, specially this took so long to be resolved.

It’s really bad for business, not only our website was down, our whole staff cannot send and receive any emails. The RSS feeds to our LED boards were down also! It’s really bad, I got calls from our clients, my boss was furious and I cant reach anybody from IX! I don’t know what to say to my boss and our clients.

It’s really bad, I hope this won’t happen again.

Thank you.

Link | Posted at 11:38 pm EST

Dina Narain says:

I’ve been an IX customer for 8 years now and this is the one and only time something of this magnitude has occurred, which goes to show they have a pretty decent track record in terms of reliability. I found the constant updates reassuring and certainly appreciate the honesty and transparency about the situation. A big thank you goes to the entire team at IX for doing their best to resolve this as fast as possible under such tremendous pressure, especially while having to deal with this verbal tirade from disgruntled customers. I am confident that IX will take on board some of the constructive criticism posted in this thread to avoid such situations in the future.

Link | Posted at 11:58 pm EST