Introduction

If you’re looking to improve your website’s uptime and availability, then this solution might be the one you’re looking for.

We’ll be using

• Nibbleblog as our blogging software to keep this guide simple.
• Apache, PHP, PHP modules (php5gd, and php5mcrypt), Unison for syncing files,
• Monit for monitoring
• Cloudflare for our domain’s DNS.

This setup will be done on one monitoring box, and three content serving boxes, but you can expand this to as many boxes as you want with some modification. The best explanation of how our system will be setup is described in the Unison manual.

We’ll organize the machines into a “star topology,” with one machine designated as the “hub” and the rest as “spokes,” and with each spoke machine synchronizing only with the hub. Basically, the monitoring box will monitor both the hub and the spokes for downtime. If any of the machines are found offline the monitoring box will tell Cloudflare to knock the specific offline hub or spoke off of the roundrobin list till it comes back online.
Here are the boxes I picked used for this guide:

• 256M LA KVM box for monitoring.
• LowEndSpirit Dallas box acting as a hub.
• LowEndSpirit UK box acting as a spoke.
• LowEndSpirit Italy box acting as a spoke.

This guide will be divided into four different sections.

1. How to setup Cloudflare for this tutorial.
2. How to setup the hub server for serving content.
3. How to setup spoke servers with Unison for syncing and for serving content.
4. How to setup the monitoring server to use Cloudflare API to remove and add servers to the roundrobin list.

A note about using IPv6: If you do not have some form of IPv4 (e.g at least NAT’d) on your servers then this tutorial isn’t for you at the moment. Your monitoring server must have a dedicated IPv4 address because Monit doesn’t support IPv6 at the time of this writing.

 

1. Setting up Cloudflare

Requirements:

• Cloudflare account with domain already setup.

 

First we’re going to login to our account and navigate to the DNS settings of our domain.

 

 

Once there you will want to setup your records similar to this setup. You will want your monitoring server with Cloudflare disabled, you want your hub and spoke IPs in roundrobin, and then you will want to dedicate separate names for each of those servers.

• A / monitor / points to xx.xxx.xxx.171 <CF Disabled
• AAAA / blog/ points to xxxx:xxxx:xxx:xxx:xxxx:xxxx:xxxx:5764 <CF Enabled
• AAAA / blog/ points to xxxx:xxxx:xxxx::xxxx:7be3 <CF Enabled
• AAAA / blog/ points to xxxx:xxxx:xxxx:x:xxxx:xxxx:xxxx:6fc9 <CF Enabled
• AAAA / hub / points to xxxx:xxxx:xxx:xxx:xxxx:xxxx:xxxx:5764 <CF Disabled
• AAAA / spoke1 / points to xxxx:xxxx:xxxx::xxxx:7be <CF Disabled
• AAAA / spoke2 / points to xxxx:xxxx:xxxx:x:xxxx:xxxx:xxxx:6fc9 <CF Disabled

Scroll down and hit “I’m done entering my DNS records”.
Before we head over to the servers, we will need one last thing from Cloudflare. Go to your “Account” page and jot down both your email and your API key. We’ll use these later

2. Setting up your hub server

Requirements:

• A freshly installed Debian 7 VPS.

Part A:

First we’re going to change our root password to something more secure while we’re setting up the box:

Next, make sure the system is up to date:

Then we’ll install sudo and nano:

We’re going to add a user, and then add him to the sudo group so we don’t have to login as root later:

After that, we’ll install Apache, PHP5, some modules, Unison, and unzip:

NOTE [4/25/14]: LowEndSpirit users! Edit /etc/apache2/ports.conf and add a NAT’d port to the configuration like so. Monit doesn’t appear to support IPv6 at the time of this writing so this is a temporary fix until then. You’ll also need your external IP of your node later so make sure you have it!

We’re going to restart Apache to make sure everything is active:

Remove any files in the /var/www folder:

We’ll need SSH keys so our spokes can login to our hub and vice versa without hassle. Make sure you don’t set a passphrase. You’ll want to generate these keys for the root user:

Enter file in which to save the key (/root/.ssh/id_rsa): Hit enter.
Enter passphrase (empty for no passphrase): Hit enter.
Enter same passphrase again: Hit enter.

Part B:

Let’s get Nibble now: (assuming that you are still in the /var/www directory)

You can navigate to your blog via IP address and setup your blog now! Come back when finished. 🙂
We’re done for now, leave your SSH connection up and going as we will need it again later.

 

Setting up the spoke servers (Repeat for each spoke)

Requirements:

• A Debian 7 VPS and use the exact steps performed above, but skipping part B.

Now we’ll copy the SSH key that you generated on your hub server over to your spoke machine with this command:

We’ll also need copy the SSH key that you generated on your spoke server over to your hub with the command:

On your spoke server we can sync Unison for the first time with this command. Hit the F key till you reach the end of the sync list,  and then hit the Y key to sync:

Now that is out of the way we’re going to edit Unison’s configuration file on the spoke server so we can sync automatically:

Here’s the configuration used in this guide.

Change hub.example.tld to your hub’s hostname (root = ssh://hub.example.tld//var/www), save the configuration, and exit.
You can also uncomment this line in the Unison file if you want to be able to edit each of the site’s Nibble configurations separate of each other:

On your your spoke server make a new Cron Job under root for unison:

Add this to the very end of the file (This will sync the server to the hub every 5 minutes.):

You’ll probably need to reset your permissions on your spokes after the initial sync to be able to post.. You can do this with these commands:

Let’s finish cleaning up, leaving the root password isn’t a smart idea. So perform these commands on both the hub and any spoke servers.
Disable root login unless they have a key in the SSH configuration. (Required for Unison):

change PermitRootLogin yes to PermitRootLogin without-password
We’ll remove any of root’s password access:

Reload your SSH settings, or reboot your container to apply the settings:
service ssh reload
Everything’s coming together pretty nicely huh!? Woo! Ready for more configuration madness?

 

Setting up server monitoring and Cloudflare fallback

Requirements:

• A Debian 7 VPS with the steps from “Setting up your hub server” stop before you install Apache and come back. You’ll want to disable root login as well.

First we’ll need cURL, Monit, and unzip:

Next we’ll grab all the scripts required and unzip them:

We’ll allow the scripts to be executable:

Now this is probably the most boring and extremely tedious task ever, but you’ll need to edit each and every one of the Cloudflare scripts. Make sure you follow these instructions and input the correct information exactly or you run the risk of fouling up the DNS records in your account.

You need to gather some record IDs so Cloudflare knows exactly which ones we will be using when we the rename records. Let’s nano into cfid.sh, fill out your Cloudflare information and save the file and run the script below:

You’ll get a huge blob of JSON magic. Copy all of it to your clipboard and navigate to this JSON viewer and paste it inside the “Text” tab. Switch to the “Viewer” tab. Type the name of your record that has the multiple IPs into the search bar and press “Go” to search for one, then hit next till all of them are expanded.

You’ll want to grab the IDs for all of the records that are in the round robin. Take note to which server they belong to.
Nano into cfhub.sh, fill out your Cloudflare details, and then fill out your  server details. Repeat for all of these:

• cfkillhub.sh
• cfkillspoke1.sh
• cfkillspoke2.sh
• cfspoke1.sh
• cfspoke2.sh

Alright, let’s configure Monit:

change set daemon 120 to set daemon 5
uncomment “set httpd port 2812 and”
uncomment “use address localhost” and set your IP in place of localhost.
uncomment “allow admin:monit”

CTRL + O & CTRL + X
We need to set our checks up for our hub and our spokes:

Paste in this configuration and edit it to your needs:
Save and exit the configuration file.

LowEndSpirit users! This is where you use your external IP and your NAT’d port. Change address to your external IP and change port 80 to your port!
WARNING: You might want to backup all of your DNS records in case something goes wrong!

Start monit:

You’re done! Congrats!

To see if it works shut off one of your VMs and wait a bit for CF to pull the record. Monit is easily configurable and customizable to your needs incase you want to change the check frequency and sensitivity. I hope this tutorial helped you and gave you an idea on how to make your website more redundant.