Adding statistics to my static website

bgstack15

2021-10-17 08:48

Adding statistics

My previous blog solution had a statistics page, and I wanted to see something similar. I chose awstats as my solution.

Installing awstats

yum install awstats htmldoc

Set up file /etc/awstats.doc7-01a.conf with these interesting settings. See included file awstats.doc7-01a.conf.example for the full file.

LogFile="/usr/share/awstats/tools/logresolvemerge.pl /var/log/nginx/access.log /var/log/nginx/access.log*.gz |"
LogType=W
LogFormat="%host - %logname %time1 %methodurl %code %bytesd %refererquot %uaquot %otherquot "
LogSeparator=" "
SiteDomain="bgstack15.ddns.net"
HostAliases="bgstack15.ddns.net"
DNSLookup=1
DynamicDNSLookup=0
DirData="/var/lib/awstats/regular"
SkipFiles="REGEX[^\/stats] REGEX[^\/isso] /blog/tagcloud.html"
Logo="favicon_128.png"
LogoLink="https://bgstack15.ddns.net/blog/"
KeepBackupOfHistoricFiles=1

This LogFormat setting can parse the default nginx log_format directive.

I have set up my 128x128 logo as file /usr/share/awstats/wwwroot/icon/other/favicon_128.png.

Configuring nginx for awstats

Write /etc/nginx/default.d/awstats.conf with these contents.

location /stats/ {
   alias /var/lib/awstats/static/;
   index awstats.doc7-01a.html;
   location /stats/css/ {
      alias /usr/share/awstats/wwwroot/css/;
   }
   location /stats/icon/ {
      alias /usr/share/awstats/wwwroot/icon/;
   }
   auth_basic "Admin";
   auth_basic_user_file /etc/nginx/awstats.htpasswd;
}
location /awstatscss/ {
   alias /usr/share/awstats/wwwroot/css/;
}
location /awstatsicons/ {
   alias /usr/share/awstats/wwwroot/icon/;
}

Reload nginx.

Add a htpassword file, manually Reference 1.

printf "`read -p Username:\ ; echo $REPLY`:`openssl passwd -apr1`\n" | sudo tee /etc/nginx/awstats.htpasswd

Choose a suitable username and password.

Generating stats

Write a new script, /usr/local/bin/run-awstats.sh with contents:

#!/bin/sh
# Startdate: 2021-09-26 12:25
/usr/share/awstats/wwwroot/cgi-bin/awstats.pl -update -config=doc7-01a -configdir="/etc/awstats" -showdropped
/usr/share/awstats/tools/awstats_buildstaticpages.pl -config=doc7-01a -configdir="/etc/awstats" -dir=/var/lib/awstats/static -buildpdf=/usr/bin/htmldoc

The first command generates the raw awstats entry files which are probably used by the cgi scripts (not used in my solution), and the second command generates the static files that the previous nginx config points to.

Add a cron job for this script, /etc/cron.d/90_awstats_cron.

# file: /etc/cron.d/90_awstats_cron
3 0 * * *   root  /usr/local/bin/run-awstats.sh 1>/dev/null 2>&1
5 5 * * 2   root  /usr/local/bin/run-awstats-lastmonth.sh 1>/dev/null 2>&1

The lastmonth script is designed to generate the end-of-month pages for historical daily data from that month. File run-awstats-lastmonth.sh contains these contents:

#!/bin/sh
# Startdate: 2021-10-03 14:46
# Documentation: blog-README.md
# Reference:
#    https://joshuakugler.com/generating-static-pages-for-all-awstats-files.html
test -z "${M}" && {
   YM="$( date --date "now-1 month" "+%Y-%m" )"
   Y="$( echo "${YM}" | awk -F'-' '{print $1}' )"
   M="$( echo "${YM}" | awk -F'-' '{print $2}' )"
}
echo "Y=${Y}"
echo "M=${M}"
/usr/share/awstats/tools/awstats_buildstaticpages.pl -config=doc7-01a -configdir=/etc/awstats -dir=/var/lib/awstats/static -builddate="${YM}" -year="${Y}" -month="${M}"

I generated menu.html manually, with links to the month pages. To simplify the matter, the links exist, but the pages do not yet, for future months. Some trimmed contents of this file are below.

<html>
<head>
<base target="_top" href="/stats/">
<style type="text/css">
a.gone {
   text-decoration: line-through;
   color: #808080;
}
</style>
</head>
<body>
<table>
<tbody>
<tr><th><a href="">current</a></th></tr>
<tr>
<th>2021</th>
<td><a href="awstats.doc7-01a.2021-01.html" class="gone">01</a></td>
<td><a href="awstats.doc7-01a.2021-02.html" class="gone">02</a></td>
<td><a href="awstats.doc7-01a.2021-09.html">09</a></td>
<td><a href="awstats.doc7-01a.2021-10.html">10</a></td>
</tr>
<th>2022</th>
<td><a href="awstats.doc7-01a.2022-01.html">01</a></td>
<td><a href="awstats.doc7-01a.2022-12.html">12</a></td>
</tr>
</tbody>
</table>
</body>
</html>

Configuring SELinux

I want to use SELinux, so I established this policy using filename awstats1.te.

module awstats1 1.0;

require {
    type httpd_t;
    type awstats_var_lib_t;
    class file { getattr open read };
}

#============= httpd_t ==============

allow httpd_t awstats_var_lib_t:file { open read getattr };

To apply, run these commands.

sudo checkmodule -M -m -o awstats1.mod awstats1.te && sudo semodule_package -o awstats1.pp -m awstats1.mod && sudo semodule -i awstats1.pp ;

Additional assets deployed at initialization time

Files on doc7-01a

/usr/local/bin/run-awstats.sh
/usr/local/bin/run-awstats-lastmonth.sh
/var/lib/awstats/static/menu.html

Operations

Changing observed statistics

Over time, I might learn what statistics need to be modified or excluded. The most interesting is SkipFiles in file /etc/awstats/awstats.doc7-01a.conf.

Knowledge Base