The Stars Aligned and My Blog Went Down
All right, so as you guys may've noticed, my blog was down for a good 24 hour period through most of yesterday. A real pity considering the amount of traffic that was coming over trying to pull up the site. (And REALLY frustrating. I think fellow webmasters will know what I'm talking about, and for the rest of you, imagine having sent out a party invitation to your house to tens of thousands of people then having to leave the entire day with the house dark and locked up.) So what happened?
Mass Linkage
The contest winners post ended up getting picked up by quite a few blogs and Mac news sites. Special thanks to Digg users, Gizmodo, Cult of Mac, MacMinute, Macworld, The Unofficial Apple Blog, MacBytes, MacAddict, The Mac Observer, MacNN, the O'Reilly Mac DevCenter Blog, MacNews.de, and many... many more.
The one of note probably being Slashdot.
Now, remarkably, the site held up pretty well during the day before yesterday's onslaught from all these websites, pulling in an average of 3-4,000 hits an hour. Then, at around 10pm or so, I noticed a few random Slashdot referrals show up in my Mint installation, eagerly checked out the site, found nothing, resumed watching 24, then returned to my computer to find my blog mostly inaccessible. (I guess those random hits were from a queue or something.)
So beginning at 11pm on the 27th, my blog was in a constant state of, well, dead..ness. It didn't help that...
Dreamhost Blows
When I signed up for Dreamhost hosting last year, it was with happy thoughts, as if you Google around you'll actually notice a lot of good feedback for the guys. But the last couple of months have been rough. Really rough. Here's a snippet from a Dreamhost forum thread about these problems:
Quote, liberalleft: "As we all know reliability spanning the past year has been extremely sporadic, the past couple months have been garbage and the past month has been completely unacceptable. What's more, every response from a support request goes like this..."
"THAT'S JUST GREAT, I CAN'T FIND A DECENT WEBHOST! THIS IS MY 4TH WEBHOST IN A YEAR, I AM EXPERIENCING THE SAME THINGS DESCRIBED IN THIS THREAD, ALL MY WEBMASTER BUDDIES ARE ALL TELLING ME 'DREAMHOST ROCKS, DUDE," AND THEN I COME ONBOARD AND I CAN'T EVEN SEND AN EMAIL? WHAT IS THE WORLD COMING TOO?
WELL, ONCE GREENLAND MELTS, I GUESS IT WON'T MATTER, I'LL BE EATING DRIED SPAM IN A CAVE IN MONTANA AND SHOOTING AT MY COUSINS."
Anyway, so end summary, Dreamhost sucks, has been sucking particularly hard the last few months, and even harder, like, now. So I began to very painfully slowly backup everything to get ready to move to a new dedicated server from 1and1, all set up and ready to go. (Seriously, how much does that suck, I was about a day away from the Slashdotting and Dreamhost mess to a brand new dedicated host. Gah.)
But then, uh, things got even shittier.
The Data Center Shits Itself
So then, my blog goes ENTIRELY down. As in, not even attempting to load for 30 seconds then failing miserably, but not even attempting to load and throwing up an error. Apparently, according to my blogger friend Paul Stamatiou, who also knows a lot more about this kind of stuff than me, hell was breaking loose at the data center where DreamHost's servers (and others, including MySpace.com's, MediaTemple's, and others) are stored. You know, the earthquake proof, fire proof, biometric hand security system armed Garland Building. What? Just, say that again? So I learned after stepping out of my moving-to-a-new-house-so-I-don't-listen-to-news bunker that California (where the Garland Building is located) is undergoing a major heat wave, the kind that kills people. Yeah. I guess I'll bold this just for emphasis.
People Killing Heat Wave in California
So anyway, as one of the contest sponsors, Andy Kim of Potion Factory, informed me via chat, it's hot. REALLY hot in Los Angeles. Apparently, hot enough to cause power failures, and hot enough to cause an emergency power generator at the Garland Building to catch on fire, hosing a bunch of servers and taking down most of the blogosphere with it for several hours.
So this was the point where lightning starts flashing outside of our new house in Newton, MA, and the rain starts pouring. I'm frantically trying to figure out what the hell is going wrong with Dreamhost, the Garland Building, California, and all, when suddenly I see a bright flash of light, hear really loud thunder, something outside starts glowing a really weird blue, the lights flash in and out for a few minutes, and the power goes dead.
Yeah. So there goes my wi-fi, as apparently...
Lightning Hits a Power Line About 50 feet Away...
at a neighbor's house. (They're ok.) This managed to knock out the power on our street block, and just about nowhere else.
So at this point, I kind of gave up, and went with my family for some dinner and the new Woody Allen movie, Scoop. By the end of the movie, I was comfortably passed out in the movie theater seat, and resumed this state of being passed out when we got back to a dark, power-less home.
Oh yeah, and let's not forget...
Wil Shipley
With extensive words of praise and criticism on the winners and runners up, the man behind Delicious Monster was quite a hit with the readers, but also unexpectedly resulted in increased page load times, CPU load, and exacerbated the Slashdotting. (And he also doubled the article's word length, which made me seem like a much more competent blogger.) So yeah, Shipley deserves a bolded section. (Just kidding. 
)
Well, at least the blog's now back up. Enjoy.
Comments
Up until this point, there have been 18 responses to “The Stars Aligned and My Blog Went Down”:
Crap. I just started a new site with Dreamhost and I knew it sounded to good to be true. I'll keep my fingers crossed that I don't have the same issues.
As far as hosts go, I love ASmallOrange. I think it's a great company with very friendly help.
Thomas
twistermc.com
I'm gonna have to second KZeni on 1and1. They are craptastical. Myself and several of my friends have had terrible experiences with them. For two examples: One freind was having cronic uptime (about 1 day down every week), when he complained they game him a load of marketing speel which boiled down to "you want more uptime pay us more". Another let the renewal of one of his domains lapse by a couple days and 1and1 hijacked his domain name and was selling it through a domain auction service for £500.
I've not used Dreamhost or heard much about them but I know that they share Data Centers with (my host of choice) MediaTemple (as you mentioned in your post). MT in my opinion are excellent. I've never had sites big enough but I have heard they work very closely with high traffice sites to ensure they work as expected without braking the bank.
Phill Ryu
Jon, I've heard some bad stuff about 1and1 shared hosting, which I have some suspicion is related to their massive '2 years free hosting' giveaway they were doing for a while. But a $160/month dedicated host? I'll be expecting good service and uptime from them for that I hope.
I'd also be wary of 1and1. I know of a few people, hosted both on the shared servers and dedicated servers that have had terrible experiences with them (one involving the November Reboot when an immense amount of traffic was pouring in, support requests went unanswered for 24+ hours).
I would highly recommend MediaTemple. I realize that they're also in the Garland building, but they'll be moving to a new datacenter within the next week that is supposedly much higher quality than the Garland building.
I really recommend Surftown, i know they're not based in USA but they are really cheap and has unlimited bandwidth, ive got 2 accounts there and there has been absolutely nothing bad ever with any of them.
Great to see you're back anyhow 
For what it's worth, TextDrive has been a great host. It was more than flakey, as in, the sparse flakes were uptime (okay, that's an exaggeration, but you get me), but they have significantly pulled their performance up to what I presume is the best I can get for the price, which is, as it stands, $0, as we participated in life-time hosting for a one-time pay. Good deal that was. Best of luck with 1and1. I hope their dedicated plans will be pretty darn decent.
I've been with Dreamhost for years and I've never seen as many problems as in the last few months. They've had a rough time with some technical issues (a few out-of-their-control) and I imagine with the surge in accounts. That said, they have the best support I have come across with shared web hosts.
The real problem comes into play when you are using shared hosting and your site is pulling in major traffic. I've never been with a host who could handle that well after a certain point. Adding in a heat wave and power failures never helps matters either.
Would suggest that Dreamhost also has dedicated servers (Here I am sounding like a Dreamhost lackey...), but you've obviously been jaded by the recent experience. With that, hope things from this point on go a lot smoother for you.
Truly enjoy what you've been doing with the site.
What do you guys think of Godaddy? Ive been with GD for almost a year now and have had good servcie and not issues with stability. My blog doesn't receive nearly as many hits as you Phil but still a good steady thousand hits every 5 days or so.. (I know almost any server could handle that but I have also seen on my blog some spurts of hits and nothing went down)
I'd just like some feedback on if I should renew for a second year or not..
Hey Phill,
Trust me, 1and1 is no good. I moved from a dedicated server with them very quickly a few months ago. Don't trust anyone with a big name, because they tend to have lousy support.
I've got a host now, so if you need help, feel free to drop me a line.
Anyways, great blog, and I eagerly watched your recent contest unfolding. Good choices for 1st place! Also, thanks for introducing me to AppZapper following the recent ad campaign, it's a great product, that I grabbed during a macZot.
And Sebastien, GoDaddy are good and pretty reliable, but not cheap. They are great for domains, but not so good for hosting, not very cost effective.
I've been using RailsPlayground.com for a few months. They've been able to handle my 5 daily visits quite well 
All kidding aside, they've been great, and their customer support is always quick and helpful.
Phill Ryu
Thanks everyone for the suggestions, hopefully this post and its comments will be helpful for others making hosting decisions as well.
I don't know if it helps, but I'm currently with BlueHost. They have had some issues recently (I believe with power outages, possibly related to heat) but under normal circumstances they are great. Support, uptime, features, and all is what you would expect from a quality hosting company.
I have a few sites with CrystalTech for well over 2 years now and I've never had any problems with them. No idea where there data centre is though – I'm in Aus(tralia) so it doesn't make a hell of a lot of difference!
Lovin the blog Phill.
I have found no host to be more reliable, accessible, dependable and rock-solid than Incognito Networks. I believe their data center is in Kansas.
Every shared hosting setup I ever tried turned out to be a piece of crap (for one reason or another).
I've had a dedicated server with servermatrix for 2 years now, and no complaints. I would recommend them to people looking for a dedicated server. I believe their data center is in Texas.
I'm using Media Temple Shared Server for my blog and a number of email domains. They're quite quick with support and I've never had any significant downtime - all for less than US$10/month if you prepay. I believe they don't "oversell" either, which is always good.
The Little App Factory website was just moved to a Segment Publishing Shared Server. They provide a 99.5% uptime guarantee (and give you the month free if it falls below this). Support turn-around is quick and we haven't had any problems. (Fingers crossed.) You do pay for the service, though!
Hope this helps!
[...] As I wrote a couple of days ago, the blog was in the throws of some rather major hosting disasters, partially due to DreamHost’s service. Imagine my surprise when these problems continued AFTER the data center was back up, and the Slashdotting was over. So I sent them an email, and here’s what I got: [...]
KZeni
July 29th, 2006 at 10:52amI have had terrible experiences from 1and1. And so has anyone that has used them for any ammount of time. It sucks to hear that you're constantly having webhost issues, but 1and1 is a step down from dreamhost.