Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

Block ads within existing Bind9 infastructure

bgstack15

Comments

Someone recently shared with me how he uses a managed hosts file (no, not the famous APK Hosts File Engine) to help block ads. The project, from Github, links to a derivative project that translates the managed hosts file into bind9 zonefiles! I have set this up on my existing dns infrastructure, and here is the redacted documentation.

Overview

The goal of this project is to provide a mechanism for the internal network to block ads, by blocking domains that are known serve ads.

Blocking domains that serve ads

Bind9 configuration is modified on both dns1 and dns2. An additional include file is added, and many zones are added that have a basically empty config. A separate whitelist file is used to prevent bind9 from trying to control domain names that it struggles with for reasons beyond the scope of this document.

Summary of files involved

  • /etc/named.conf
  • /etc/installed/generate-zonefile.sh
  • /etc/named/named.conf.blocked
  • /etc/named/db.blocked

Named config

File /etc/named.conf includes a snippet. This is on both dns1 and dns2.

include "/etc/named/named.conf.blocked";

This line is appended at the very bottom of the file, underneath other inclusions for files in /etc/named/.

Zonefile Generator Script

This file is adapted from repo Mueller-ma, and exists only on dns1.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
 
#!/bin/bash -e
# -E makes the shell exit immediately if a command exits with a non-zero status

# v 1.1 2020-08-27 bgstack15

TMPFILE_CMD=mktemp

# VARS by distro
distro="$( awk -F'=' '$1 == "NAME" {print $2}' /etc/os-release )"
if echo "${distro}" | grep -qiE 'centos|rhel|fedora' ;
then
    dest_zonefile=/etc/named/named.conf.blocked
    blockedfile=/etc/named/db.blocked
    service_name=named
else
    # assume debian-like, debian|devuan|ubuntu|mint
    dest_zonefile=/etc/bind/named.conf.blocked
    blockedfile=/etc/bind/db.blocked
    service_name=bind9
fi

# display date
date

# Set tempfiles
# All_domains will contain all domains from all lists, but also duplicates and ones which are whitelisted
all_domains="$( "${TMPFILE_CMD}" )"
# Like above, but no duplicates or whitelisted URLs
all_domains_uniq="$( "${TMPFILE_CMD}" )"
# We don't write directly to the zonefile. Instead to this temp file and copy it to the right directory afterwards
zonefile="$( "${TMPFILE_CMD}" )"

# Define local black and white lists
# Comment these out if you have no local files
manual_blacklist="/etc/installed/dns-blacklist"
manual_whitelist="/etc/installed/dns-whitelist"

# change to this working directory
cd /tmp

# StevenBlack GitHub Hosts
# Uncomment ONE line containing the filter you want to apply
# See https://github.com/StevenBlack/hosts for more combinations
wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/porn/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/social/hosts
#wget -q -O StevenBlack-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn-social/hosts

# Filter out localhost and broadcast
< StevenBlack-hosts awk '/^0\.0\.0\.0/ && ! /127.0.0.1|255.255.255.255|::1/ {print $2}' >> "${all_domains}"

# Add local blacklist
if [ -f "${manual_blacklist}" ]
then
    cat "${manual_blacklist}" >> "${all_domains}"
fi

# Filter out comments and empty lines
< "${all_domains}" grep -vE '^\s*($|#)' | sort | uniq > "${all_domains_uniq}"

# Apply local whitelist
if [ -f "${manual_whitelist}" ]
then
    for i in $(cat "${manual_whitelist}")
    do
        #echo i $i
        sed --in-place= "/$i/d" "${all_domains_uniq}"
    done
fi

# Add zone information
< "${all_domains_uniq}" sed -r "s:(.*):zone \"\1\" { type master; file \"${blockedfile}\"; };:" > "${zonefile}"

# Copy temp file to right directory
cp -p "${zonefile}" "${dest_zonefile}"
chmod 0644 "${dest_zonefile}"

# Remove all tempfiles
rm -f "${all_domains}" "${all_domains_uniq}" "${zonefile}" StevenBlack-hosts

# Restart bind
#systemctl restart "${service_name}"

# For logfile
printf '%s\n\n' 'done'

This script could be adapted for use in cron, but I have not done so as of the time of this writing. Run this script, and it will automatically update the /etc/named/named.conf.blocked file with all the zones from the upstream, managed hosts file. I have disabled the systemctl restart named section for the internal network.

Blocked zones

The above script generates file /etc/named/named.conf.blocked which contains the zone definitions themselves. Only a brief snippet is shown here.

zone "0.0.0.0" { type master; file "/etc/named/db.blocked"; };
zone "0.0.0.0.beeglivesex.com" { type master; file "/etc/named/db.blocked"; };
zone "0.0.0.0.creative.hpyrdr.com" { type master; file "/etc/named/db.blocked"; };
zone "0.0.0.0.hpyrdr.com" { type master; file "/etc/named/db.blocked"; };
zone "000free.us" { type master; file "/etc/named/db.blocked"; };

Empty zone definition

So every zone uses the same database file, which is pretty much all empty. It must be adapted for whichever domain I am controlling.

$TTL 24h

@       IN SOA dns1.ipa.example.com. dnsmaster.ipa.example.com. (
               2003052800  86400  300  604800  3600 )

@       IN      NS   dns1.ipa.example.com.
@       IN      A    0.0.0.0
*       IN      A    0.0.0.0

Whitelist

Unfortunately some of the entries from the upstream hosts file cause issues in bind9. By adding them to the whitelist, we do not null-route requests to these domains. I am hoping these domains are used infrequently by my traffic. File /etc/installed/dns-whitelist contains exactly these contents.

___id___.c.mystat-in.net
philadelphia_cbslocal.us.intellitxt.com
www.zgarniij_vouchher.skroc.pl
zgarniij_vouchher.skroc.pl

Blacklist

An additional file could be used for my own list of domains to block, but this has not been implemented yet.

Implementation in internal network

One time setup

Generate all files on dns1. Copy to dns2 the two files /etc/named/db.blocked and /etc/named/named.conf.blocked. I had to learn how which domains to exclude. After generating the initial zones list, I had to validate it and look for errors:

time /usr/sbin/named-checkconf -z /etc/named.conf 2&gt;&amp;1 | grep -viE '^zone.*loaded serial 200305'

Recurring use and maintenance

To generate a current zone definition from upstream, run generate-zonefiles.sh on dns1. Manually copy the /etc/named/named.conf.blocked to dns2. Restart dns on both systems.

References

Weblinks

  1. Managed hosts file that blocks domains that serve ads
  2. Convert hosts file into bind9 zone files

Comments