Getting probed. The ACE Cisco way!

Now with a title such as this you’d almost think I was going to start talking about Cisco’s awesome prices and comparative rates but alas, It is part three of my dive into load balancers.

Previously we have discussed what a load balancer can offer and a simple one-arm configuration. That’s great! Now lets look at how to keep your servers up and not up and down like a merry-go-round.

Health Probes are one way to ensure a server is still servicing correctly. It can let a load balancer know that requests to that server will still be there or conversely let it know that the party’s over and it’s out of action. There are some important things to understand when talking about health probes. Timers. How often, how long and what is being checked. Sounds like a horrible day at t
he doctor’s office. Or for some – the daily grind!

Now – back on topic Anthony! Health probes are easily set up but can be a headache. I mentioned that timers are important and I will discuss why. Health checks will place a server in or out of service and it allows the ACE to make informed load balancing decisions. The two categories are Passed – denoting a valid response, and Failed – indicating a server which has failed to provide a response after a specified number of retires.

There are over 1000 unique configurations for probes. Ping, HTTP, ICMP and more! Oh the possibilities. This allows compatibility with a raft of services and software.

Lets get probin’

First we configure the health probe with its name, type of probe, and the attributes associated to it.

probe http ACE-PROBE-HTTP-GET
 interval 10
 faildetect 2
 passdetect interval 30
 passdetect count 2
 request method get url /index.html
 expect status 200 200

Note above the following settings. First we have defined the probe type including a name for it. Next the interval command specifies how frequently the ACE sends probes to the server marked as passed. This is important. The Goldilocks rule applies here. Timers that are too aggressive can load the server and cause a failure. Timers that are too slow will not be efficient in detecting a failure. Measured in seconds the interval default is 15. Here we have set it to 10 seconds.

Faildetect works by setting a counter and allows you to choose how many probes can fail before a server is marked as failed and placed out of service. Default is 3 and here we have manually defined 2.

Passdetect interval is the next interesting feature and it needs to be set correctly. When a server has failed to respond to probes the passdetect interval must pass before a new probe is sent. That is by default 60 seconds. Too long for me. 30 seconds will suit. To put the server back inservice it must pass 2 health checks defined with passdetect count at the interval value of 10 seconds.

The way I am checking this probe is through a HTTP get request. This allows me to confirm that IIS/Apache is working. If I was just to ping the real server IP, IIS/Apache could fall over and the server would still receive HTTP request and yield unsatisfactory results.

The next important field is expect status 200 200. By default the device will not have an expected statuses set. This means that it will never expect anything and the probe will fail. By defining what to expect this will allow the probe to mark a successful pass! You can define a high and low range with this setting and that will allow you to expect a all sorts of http response codes.

My take

The importance of health probes is paramount to achieving HA. Ineffective timers can cause headaches. What I have shown here is but a fraction with many 00’s of the possibilities that are out there. Just remember if you are being aggressive that the probe you are placing on a server farm,  that there could be hundreds or thousands of simultaneous probes occurring. This will place more load onto the ACE’s finite resources.

More resources

Interval, time, and count are very important to settings to tweak to get the health check “just right”. A particular post from Tony Bourke sums up the importance of timings and intervals.

Health Checking On Load Balancers: More Art Than Science

For more information on health checks and load balancing tips and tricks – Please go check out one of the best CCIE DC candidates blog! Lot’s of ACE goodies aswell as Nexus and FC resources. Oh and it’s full of meme’s!

http://datacenteroverlords.com

Previous ACE Articles by @pandom_

Cisco ACE : I’ve not gone green and flipped a table yet.
Cisco Application Control Engine 4710