Uptime
Frequent testing to ensure uptime
Cron is reliable and easy to configure. It can do something for you on the server as often as you need it done. I use it to run backups, renew Let’s Encrypt certificates, and to send me XMPP messages.
Most Linux system clocks are syncronized accurately. We want to avoid everyone in the world sending a message in the first second of each minute. To spread out messages throughout each minute, please add the following line of code in the scripts called from cron:
# send instant message at a random second within the minute
sleep $((1 + $RANDOM %55))
For scripts that are called frequently, you can lower the number of random seconds the script sleeps. For example:
sleep $((1 + $RANDOM %15))
# edit crontab:
crontab -e
# list crontab:
crontab -l
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12)
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7)
# | | | | |
# * * * * * user-name command to be executed
Here I set up a quick crontab every minute of every hour of every day, just to try it out:
* * * * * /usr/local/bin/monitor.chat.sh '<CONFOUNDED> A message every minute from cron is an inadequate use of an excellent monitoring service!' 1>/dev/null 2>&1
A cron every minute could represent a heartbeat. It is your server saying it is awake. This example provides you with almost no useful information about your server.
We ask that you limit the frequency of messages no more than 1 message per host every 120 seconds. This is enforced as a rule at 90 seconds. Even if you break this rule frequenly, the only likely result would be that you see warnings at the end of your messages. If someone abuses this rule by blasting our server, the IP address of the offender will be blocked by our firewall temporarily.
The key to success is in the quality of the information you receive from your servers, not the frequency. The messages you receive will depend on the kinds of applications you are running and the limits of your imagination to create scripts that query your applications for valuable information.
The following script sends a valuable piece of information, but it only makes sense to run it once per hour. This script will count the number of unique visitors to an apache2 website in the last completed hour. This script will keep you informed on the trends in the traffic to your web server is over its lifespan. This script requires that you use a custom log for the domain in question. For example, in my apache2 sites_enabled configuration file, you see CustomLog ${APACHE_LOG_DIR}/e2e.ee.log combined
which creates the file /var/log/apache2/e2e.ee.log
.
#!/bin/bash
. /etc/profile
. /root/.profile
sleep $((1 + $RANDOM %55)) # send at a random second within the minute
# How popular is my website?
# A quick bash script by Edward Stoever
#
# A Sample text from Apache2 access.log:
# 187.95.14.103 - - [15/Nov/2020:14:41:21 +0000] "GET /favicon.ico
DOMAIN='e2e.ee'
THISHOUR=`date "+%k" | xargs`
if [ "$THISHOUR" = "0" ]; then
LOG="/var/log/apache2/$DOMAIN.log.1"
else
LOG="/var/log/apache2/$DOMAIN.log"
fi
HOUR=`date -d -1hour "+%k" | xargs`
if [ "$HOUR" = "1" ]; then
ORDINAL='st';
elif [ "$HOUR" = "2" ]; then
ORDINAL='nd';
elif [ "$HOUR" = "3" ]; then
ORDINAL='rd';
elif [ "$HOUR" = "21" ]; then
ORDINAL='st';
elif [ "$HOUR" = "22" ]; then
ORDINAL='nd';
elif [ "$HOUR" = "23" ]; then
ORDINAL='rd';
elif [ "$HOUR" = "0" ]; then
ORDINAL='';
else
ORDINAL='th';
fi
LAST_COMPLETED_HOUR=`date -d -1hour "+%d/%b/%Y:%H"`
UNIQ=`grep "$LAST_COMPLETED_HOUR" $LOG|grep -E '(GET|POST)'|grep -iv bot|awk '{ print $1 }'|sort | uniq| wc -l`
if [ "$UNIQ" = "1" ]; then
VERB='was'; VISI='visitor';
else
VERB='were'; VISI='visitors';
fi
/usr/local/bin/monitor.chat.sh "<POINTRIGHT> There $VERB $UNIQ unique $VISI to the $DOMAIN website in the $HOUR$ORDINAL hour."
echo "There $VERB $UNIQ unique visitors to the $DOMAIN website in the $HOUR$ORDINAL hour."
This is an excellent method of knowing just how busy a webserver really is.
I run this script once per hour, 2 minutes after the hour begins. Here is my crontab entry:
2 * * * * /root/bin/unique_web_visitors.sh >> /root/bin/log/unique_web_visitors.log 2>&1
Frequent testing to ensure uptime