In Spring 2006, my site was plagued with recurrent hardware problems causing serious downtime. At the time, the site was hosted on a dedicated server and I had no failover strategy whatsoever so when a hard disk failed on the server, you could expect a few days of downtime.
At the beginning of the summer, I got fed up and started investigating possible solutions to this problem and, after some experimentation, finally settled on DNS failover. Here are the results of the experimentation, originally posted on WebHosting Talk:
I run a site with about 1,000,000 unique visitors per month and recent server failures made me decide to get a failover server to minimize downtime. My goal wasn't to get 99.999% uptime but to be able to be back on track after a failure in a "reasonable" amount of time. After evaluating several solutions, I decided to go with DNS failover. Here's how the setup work:
1) mydomain.com points to main server with a very low TTL (time to live)
2) failover server replicates data from main server
3) when main server goes down, mydomain.com is changed to point to failover server
The drawback is the DNS propagation time since some DNS servers don't honor the TTL and there is some caching happening on the user's machine and browser. I looked for empirical data to gauge the extent of the problem but couldn't find any so I decided to setup my own experiment:
I start with mydomain.com pointing to the main server with a TTL of 1800 seconds (1/2 hour). I then change it to point to the failover server which simply port forwards to the main server. On the main server, I periodically compute the percentage of requests coming from the failover server which gives me the percentage of people for which the DNS change has propagated.
I made the DNS change at exactly 16:04 on 06/21/06 and here are the percentage of propagated users:
06/21/06 16:00 0 %
06/21/06 16:05 3 %
06/21/06 16:10 20 %
06/21/06 16:15 37 %
06/21/06 16:20 59 %
06/21/06 16:25 69 %
06/21/06 16:30 76 %
06/21/06 16:35 80 %
06/21/06 16:40 86 %
06/21/06 16:45 90 %
06/21/06 16:50 91 %
06/21/06 16:55 92 %
06/21/06 17:00 93 %
06/21/06 17:05 94 %
06/21/06 17:10 94 %
06/21/06 17:15 95 %
06/21/06 17:35 95 %
06/21/06 17:40 96 %
06/21/06 17:45 97 %
...
06/22/06 10:40 99 %
So even after 18 hours, there is still a certain percentage of users going to the old server so DNS failover is obviously not a 99.999% uptime solution. However, since more than 90% of the users are propagated in the first hour, the solution works well enough for me.
Monday, September 3, 2007
Subscribe to:
Post Comments (Atom)
22 comments:
This is not the first time that I have landed on this page.
But, this time there is something to share.
A pretty popular media site has setup *double* failover using the .
For branding reasons they determined that they should have a separate failover site rather than the free failover page that is included with the service.
As a bonus, if their production servers *and* failover servers go out of action, they will then failover to the default failover site maintained by the dns provider.
DNS Utils has some free utils to help with DNS problems.
http://www.dns-utils.com
Ok, having been here 5 times before :D
I read the article one more time, and it seems that it is a glass half-full and glass half-empty situation.
The change was made at 16:04, and the TTL was 30 minutes. Well, at 16:35, 31 minutes later, 80 percent had migrated over. In other words, within the allowed TTL, 80 percent had swung over. If the TTL had been 5 minutes, then 80 percent would have moved over by 16:10.
If you can keep 80 percent alive, or leave 100 percent dead, which is better?
Whoops!
Sorry, I was concentrating on the numbers so much that I missed your very last sentence.
Your post is being quoted at stackoverflow.com as a justification for *not* depending on dns failover as a means of improving availability. Obviously, this is not what you said in the last sentence of your original post.
I used http://www.postpart.com for failover and geo dns. It works for me.
Great information, thank you so much for posting this!
debra
Can anyone recommend the robust Network Management software for a small IT service company like mine? Does anyone use Kaseya.com or GFI.com? How do they compare to these guys I found recently: [url=http://www.n-able.com] N-able N-central security management
[/url] ? What is your best take in cost vs performance among those three? I need a good advice please... Thanks in advance!
Hello. And Bye. great site great yes yess.
[url=http://maturepornlist.com] hot milf[/url]
hot milf
http://maturepornlist.com
I hope you have no problems with your sites and servers any more
Good day
We do not agree with this year BRIT awards decision.
Please attend our little web poll
http://micropoll.com/t/KDqOnZBCWt
Lady Gaga can not be better than heavy metal
Poll supported by BRIT awards 2010 sponsor femmestyle
[url=http://www.femmestyle.li/brustvergroesserung/klinik.html]brustvergrößerung klinik[/url]
PRINCE HARRY WISHES HAPPY BIRTHDAY TO THE BRIT AWARDS
With a special birthday message from Prince Harry for the 30th Anniversary of the BRIT Awards
Genial post and this post helped me alot in my college assignement. Thank you on your information.
Hi,
I mostly visits this website[url=http://www.weightrapidloss.com/lose-10-pounds-in-2-weeks-quick-weight-loss-tips].[/url]You have really contiributed very good info here blog.pyromod.com. I am sure due to busy scedules we really do not get time to care about our health. Are you really serious about your weight?. Recent Scientific Research presents that about 50% of all USA adults are either chubby or weighty[url=http://www.weightrapidloss.com/lose-10-pounds-in-2-weeks-quick-weight-loss-tips].[/url] Therefore if you're one of these people, you're not alone. Its true that we all can't be like Brad Pitt, Angelina Jolie, Megan Fox, and have sexy and perfect six pack abs. Now the question is how you are planning to have quick weight loss? You can easily lose with with little effort. Some improvement in of daily activity can help us in losing weight quickly.
About me: I am writer of [url=http://www.weightrapidloss.com/lose-10-pounds-in-2-weeks-quick-weight-loss-tips]Quick weight loss tips[/url]. I am also health trainer who can help you lose weight quickly. If you do not want to go under hard training program than you may also try [url=http://www.weightrapidloss.com/acai-berry-for-quick-weight-loss]Acai Berry[/url] or [url=http://www.weightrapidloss.com/colon-cleanse-for-weight-loss]Colon Cleansing[/url] for quick weight loss.
i very much enjoy your own writing choice, very interesting,
don't give up as well as keep writing as a result it just well worth to follow it,
looking forward to looked over alot more of your own stories, thanks ;)
Howdy,
Do you guys watch movies in theater or on internet? I use to rent DVD movies from [b]Bigflix.com[/b]. Recently I discovered that we can watch all new movies on internet on day, they are released. So why should I spend money on renting movies??? So, can you guys please tell me where I can [url=http://www.watchhotmoviesfree.com]watch latest movie Oceans 2010[/url] for free?? I have searched [url=http://www.watchhotmoviesfree.com]Youtube.com[/url], [url=http://www.watchhotmoviesfree.com]Dailymotion.com[/url], [url=http://www.watchhotmoviesfree.com]Megavideo.com[/url] but, Could not find a good working link. If you know any working link please share it with me.
Thanks
I really liked this information on DNS, it is good to learn about it because I relate a lot with these environments
Reads my reviews about male enhancement pills befor your buy male enhancement products that really works and effective to increase your health and better your life. thank you. you can reads this reviews at http://www.male-sexual.com - vimax pills - penis enlargement
HI friends, this information is very interesting, I would like read more information about this topic, thanks for sharing.
homes for sale in costa rica
I am sorry, this variant does not approach me. Who else, what can prompt?
I can suggest to come on a site where there is a lot of information on a theme interesting you.
hope you got stuff worked out hostgator is pretty awesome
I suggest to use post part. It can surely solve your problems. trust me!
This may be irrelevant anymore but I just found this post because I was looking for a solution to a similar problem. My site is hosted by NameCheap, and when last week they were maintaining their servers and changing the HDD my site was on, the site was down for hours. And those problems lasted several days.
I am not really technical but love your idea of saving over 90% of all visitors when a server is down. Not sure I can use it though... Hope you continue writing posts for your blog! Darren
Post a Comment