Bulwark: block attacking IPs at the router level -

Inspired on Crowdsec firewall bouncer, and also with the intention of learning some Rust, I’ve created Bulwark, a small program to run in a OpenWrt router (or any Linux machine) to ban IPs of attacks detected by my servers.

Bulwark simply exposes an API over HTTP to ban/unban IPs for a predefined amount of time. I pair it with reaction in my servers, which detect brute force attacks, failed logins, etc. and report the offending IPs to bulwark to ban them for the whole network.

If you are going to run Bulwark in an off the shelf router hardware, it’s highly recommended to use some external storage, as it has to save the persistent state to disk and this will wear out the router flash memory, which typically don’t support many write cycles.

To compile for your router architecture, I recommend to use cross, you can see the instructions on the README file.

Then, copy the binary to the OpenWrt router and configure it:

/etc/init.d/bulwark:

#!/bin/sh /etc/rc.common

START=95
STOP=01
USE_PROCD=1

bulwark_conf_dir="/mnt/usb_disk/etc/bulwark"
bulwark_bin_dir="/mnt/usb_disk/bin"
bulwark_state_dir="/mnt/usb_disk/var/bulwark"

start_service() {
 procd_open_instance
 procd_set_param command "$bulwark_bin_dir/bulwark" -c "$bulwark_conf_dir/bulwark.toml"
 procd_set_param stdout 1
 procd_set_param stderr 1
 procd_set_param env PATH=/usr/sbin:/usr/bin:/sbin:/bin
 procd_add_jail bulwark ronly cgroupsns requirejail
 procd_set_param capabilities /etc/capabilities/bulwark.json
 procd_set_param no_new_privs 1
 procd_add_jail_mount_rw "$bulwark_state_dir"
 procd_add_jail_mount "$bulwark_bin_dir" "$bulwark_conf_dir"
 procd_add_jail_mount /usr /bin /sbin /lib
 procd_close_instance
}

service_stopped() {
 /usr/sbin/nft delete table inet bulwark >/dev/null 2>&1
}

/etc/capabilities/bulwark.json:

{
 "bounding": [
  "CAP_NET_ADMIN"
 ],
 "effective": [
  "CAP_NET_ADMIN"
 ],
 "ambient": [
  "CAP_NET_ADMIN"
 ],
 "permitted": [
  "CAP_NET_ADMIN"
 ],
 "inheritable": [
  "CAP_NET_ADMIN"
 ]
}

/mnt/usb_disk/etc/bulwark/bulwark.toml:

listen = "[::]:3000"
tls_cert = "/mnt/usb_disk/etc/bulwark/cert.pem"
tls_key = "/mnt/usb_disk/etc/bulwark/key.pem"
state_file = "/mnt/usb_disk/var/bulwark/bulwark-state.toml"
bearer_token = "YOUR_SECRET"
ttl = 7776000  # 90d

# Never block IPs in these networks:
ignored_networks = [
  "10.0.0.0/8",
  "192.168.0.0/16",
  "172.16.0.0/12",
  "127.0.0.0/8",
  "fc00::/7",
  "fe80::/10",
  "::1/128",
]

You will need to create the TLS certs defined in the configuration file. Then enable and start the service:

/etc/init.d/bulwark enable
/etc/init.d/bulwark start

To integrate it with reaction, you can use the following function in the reaction configuration file. Adjust the bulwark URL to your environment and place the bearer token secret in /run/secrets/bulwark_token:

local bulwark(time) = {
  ban_bulwark: {
    cmd: ['curl',
          '--insecure',
          '--fail',
          '--silent',
          '--show-error',
          '--variable',
          'BULWARK_TOKEN@/run/secrets/bulwark_token',
          '--request',
          'POST',
          '--data',
          '<ip>',
          '--expand-header',
          'Authorization: Bearer {{BULWARK_TOKEN:trim}}',
          'https://openwrt.lan:3000/ban'
          ],
  },
  unban_bulwark: {
    cmd: ['curl',
          '--insecure',
          '--fail',
          '--silent',
          '--show-error',
          '--variable',
          'BULWARK_TOKEN@/run/secrets/bulwark_token',
          '--request',
          'POST',
          '--data',
          '<ip>',
          '--expand-header',
          'Authorization: Bearer {{BULWARK_TOKEN:trim}}',
          'https://openwrt.lan:3000/unban'
          ],
    after: time,
  },
};

Then, you can call this function from your streams, for example:

    // Ban hosts failing to connect to Vaultwarden
    vaultwarden: {
      cmd: ['journalctl', '-fn0', '-u', 'vaultwarden.service'],
      filters: {
        failedlogin: {
          regex: [
            @'^.*?Username or password is incorrect\. Try again\. IP: <ip>\. Username:.*$',
            @'^.*Invalid admin token\. IP: <ip>.*$',
            @'^.*\[ERROR\] Invalid TOTP code! Server time: (.*) UTC IP: <ip>$',
          ],
          retry: 3,
          retryperiod: '2h',
          actions: banFor('24h') + killConnection() + bulwark('24h'),
        },
      },
    },

To see the full reaction configuration file, refer to the post from fail2ban to reaction.

See also