Monday, September 3, 2007

Viability of DNS failover

In Spring 2006, my site was plagued with recurrent hardware problems causing serious downtime. At the time, the site was hosted on a dedicated server and I had no failover strategy whatsoever so when a hard disk failed on the server, you could expect a few days of downtime.

At the beginning of the summer, I got fed up and started investigating possible solutions to this problem and, after some experimentation, finally settled on DNS failover. Here are the results of the experimentation, originally posted on WebHosting Talk:

I run a site with about 1,000,000 unique visitors per month and recent server failures made me decide to get a failover server to minimize downtime. My goal wasn't to get 99.999% uptime but to be able to be back on track after a failure in a "reasonable" amount of time. After evaluating several solutions, I decided to go with DNS failover. Here's how the setup work:

1) mydomain.com points to main server with a very low TTL (time to live)
2) failover server replicates data from main server
3) when main server goes down, mydomain.com is changed to point to failover server

The drawback is the DNS propagation time since some DNS servers don't honor the TTL and there is some caching happening on the user's machine and browser. I looked for empirical data to gauge the extent of the problem but couldn't find any so I decided to setup my own experiment:

I start with mydomain.com pointing to the main server with a TTL of 1800 seconds (1/2 hour). I then change it to point to the failover server which simply port forwards to the main server. On the main server, I periodically compute the percentage of requests coming from the failover server which gives me the percentage of people for which the DNS change has propagated.

I made the DNS change at exactly 16:04 on 06/21/06 and here are the percentage of propagated users:

06/21/06 16:00 0 %
06/21/06 16:05 3 %
06/21/06 16:10 20 %
06/21/06 16:15 37 %
06/21/06 16:20 59 %
06/21/06 16:25 69 %
06/21/06 16:30 76 %
06/21/06 16:35 80 %
06/21/06 16:40 86 %
06/21/06 16:45 90 %
06/21/06 16:50 91 %
06/21/06 16:55 92 %
06/21/06 17:00 93 %
06/21/06 17:05 94 %
06/21/06 17:10 94 %
06/21/06 17:15 95 %
06/21/06 17:35 95 %
06/21/06 17:40 96 %
06/21/06 17:45 97 %
...
06/22/06 10:40 99 %

So even after 18 hours, there is still a certain percentage of users going to the old server so DNS failover is obviously not a 99.999% uptime solution. However, since more than 90% of the users are propagated in the first hour, the solution works well enough for me.

23 comments:

NuvoDev Technologies said...

Great blog article about this topic, I have been lately in your blog once or twice now. I just wanted to say my thanks for the information provided here.

http://www.nuvodev.com

Servesh Singh said...

Hello VIP’s this is Koena a most attractive Delhi Escort.

Hi! Guys this is Tashu sizzling Delhi Escorts.

Hello welcome to Good Night Delhi Escorts.

We guarantee your satisfaction to the fullest and promise to make you feel craving for our

Delhi Escorts.


High profile Delhi Escorts.


Most attractive Delhi Escorts for only VIP

Gentlemen.

Hire the most beautiful Escort girl from upscale Delhi Escorts

Agency
.


Hi! This is Dipti Sinha,I am a positive, well knowledgeable Independent Escorts in Pune.


Hi! myself Nirja Bhave an Independent Escorts in Pune.



Hi! This is Kiran Joglekar mind blowing Pune Escort.


Nisha Gohalekar-I have an ideal fashioned body, scrambling shapes and sleek skin Escort in Pune.



Beautiful Sexy Goa Escorts For memorable time in

Goa.


Are You looking beautiful Indian Escorts

Diana Goyal said...

Hyderabad Escorts, Hello my name is Ashka, I am an Independent Escorts in Hyderabad

Jaya Mittal said...

This Independent Escorts in Hyderabad Ashia Khan is a very well-educated lady and want to serve top class unbeatable VIP Hyderabad Escorts Services with full satisfaction.

Tanita Mittal said...


7 Star Hi Female Escorts Services In Delhi.
This is the best and unique 7 star Delhi Escorts Service and get all stylish unforgettable Independent Escorts in Delhi.
.
.
Catreen Independent Lady With Generous Personality From Goa Escorts Service.
Now Catreen a Goa Escorts Girl also availbale for hot service for your full night entertainment like an Independent Escorts in Goa.

Mona Rathore said...

Welcome to Ahmedabad escorts for your dream satisfaction premium high profile female Independent escorts in Ahmedabad and Independent Ahmedabad Escort Girl providing high class Ahmedabad escort service.

Jaya Mittal said...

Awesome Jaipur Escorts Services offer from Sweeti Sinha who upcoming charming classic new exciting Independent Escorts in Jaipur.

Mona Rathore said...

Eiliyah national Independent Jaipur Escort girls is coming in your city to feels you real meaning of Jaipur Escorts Service. Independent escorts in Jaipur provides full details of Jaipur Escorts with their contact details and Jaipur Call Girls portfolios.

Mona Rathore said...

Eiliyah national Independent Jaipur Escort girls is coming in your city to feels you real meaning of Jaipur Escorts Service. Independent escorts in Jaipur provides full details of Jaipur Escorts with their contact details and Jaipur Call Girls portfolios.

raushni khan said...

Luxury Escorts Service Is Available In Chennai

Absolute Deluxe with Royal Independent Escort in Chennai and this Chennai Escorts Agency offered only affordable girls for you.

You can get quickly best Chennai Escorts Service with our leading Models with memorable experiences from Independent Escorts in Chennai.

Avantika Singh said...

We are presenting amazing companionship Pune Escorts Girls for your upcoming day and night. This is the exact place to choose one of most attractive Pune Escorts Agency for service enjoyment. Only our Independent Pune Escorts gives incredibly clean young service as u wants.

Mona Rathore said...

Anywhere anytime available fabulous 3Star Hyderabad escorts Girls. Our Escorts Agency provide high class Hyderabad Escorts services.
More Services:-
Independent Hyderabad Escorts
Independent Escorts in Hyderabad
Hyderabad Escorts Agency

Jaya Mittal said...

New Kolkata Escorts arrange some new comer 24/7 safe and secure Independent Escorts in Kolkata with enjoyable extraordinary moments.

Kolkata Escorts Agency
Kolkata Escorts Service
Independent Kolkata Escorts

Miss Sarheen said...

From Hyderabad Escorts Agency. Friends appoint this Independent Escorts in Hyderabad that is name Sarheen. Sarheen is a most dependable full stylish dark queen in Hyderabad Escorts.

raushni khan said...

Our Independent Hyderabad Escorts are a luxurious girls group for attraction and best safety Hyderabad Escorts services. Our Hyderabad Escorts Girls are very greatly expert in providing confidentiality booking service.

Komal Oberoi said...

I am Simran – One of the very professional and attractive Hyderabad Escorts. You are welcome to my website if you are looking for a right companion for the most private times spent together to deeply satisfy you.

Avantika Singh said...

Hyderabad Escorts Yashna Reddy is starting her stunning and affordable service. Who are arranged a satisfy Independent Hyderabad Escorts service for best deal with care and passion.

Miss Sarheen said...

Myself Swarali Seth an Independent Jaipur Escorts Girl and brings for you my especially VIP and awesome worldwide service Escorts in Jaipur.

Jaipur Escorts Services
Independent Escorts in Jaipur
Jaipur Escorts Agency

raushni khan said...

http://blog.pyromod.com/2007/09/viability-of-dns-failover.html
http://blog.pyromod.com/2007/09/viability-of-dns-failover.html
http://blog.pyromod.com/2007/09/viability-of-dns-failover.html
http://blog.pyromod.com/2007/09/viability-of-dns-failover.html
http://blog.pyromod.com/2007/09/viability-of-dns-failover.html

Mahi Mittle said...

Aurangabad Escorts is really a demanding agency who’s providing by Anjeela Seth. These Aurangabad Independent Escorts girls can know how to serve romantic requirements and feelings.

Aurangabad Escorts Services
Aurangabad Escorts Agency
Independent Escorts in Aurangabad

Avantika Singh said...

Fahima Naz high class female escorts in Aurangabad. If you are a looking for a stunning Independent Aurangabad escorts. I have strong love making desires and know different types of love making technique.
Aurangabad Escorts Agency
Auranagabad Escorts Services
Auranagabad Escorts
Independent Escorts in Aurangabad

Checkout My Independent Yukta Saikh Escort Girl in Aurangabad

You have booked a date with beautiful and sexy Escorts in Aurangabad. Hello I am Yukta Saikh one of the high class independent Aurangabad escorts. Meet me and get the best escorts services in Aurangabad.
Auranagabad Escorts
Independent Escorts in Aurangabad
Aurangabad Escorts Agency
Auranagabad Escorts Services
Auranagabad Model Escorts
Auranagabad Female Escorts

Mahi Mittle said...

Dream Night Jaipur Escorts here to enjoy relaxing dates and they have some strange romantic Independent Jaipur Escorts for Friendly appointment.

Jaipur Escorts Agency
Jaipur Escorts Services
Independent Escorts in Jaipur

Mona Rathore said...

Jaipur Night Queen have lot of most relaxing energetic deals Escorts in Jaipur with get them for loving partner by Jaipur Escorts Agency.

Independent Escorts in Jaipur
Independent Jaipur Escorts
Jaipur Escorts Services
Jaipur Escorts Girls