eoli3n
Managing FreeBSD Jails with Ansible - part 3
In the second part, we adapted our Ansible project to manage Jails with vnet.
The two first posts was useful to understand basic Jail creation, let’s now wrap it with Bastille.
Bastille is the BSD Docker-like toolset for managing containers.
That solution has many advantages, it wraps useful commands to manage Jails without having to rewrite it with Ansible.
It also has a template feature to automate Jail provisionning.
The template automate Jails creation with a Bastillefile. Note the docker reference, even the syntax looks similar.
So why wrapping Bastille with Ansible ?
Using Bastille let you provision a Jail easily, but it does not wrap Jails creation nor automate the configuration of your services. My goal is that Ansible manages automatically everything related to your service, if you need to update a config file or anything, just do it in your project and run ansible-playbook, to deploy and restart everything properly.
Following Bastille documentation, we will configure the server as if it was in DMZ, using pf as firewall to expose containers ports.
Clean all roles in the Ansible project, and configurations on the server, from previous parts.
By the way, this post shows how Ansible is kind of auto documented, as each tasks has a name, the Ansible code is pretty clean to read, even if you don’t know the tool.
Network role
We need to match network requirements.
Create a network role.
meta: flush_handlers task triggers handlers without waiting the end of the play.
roles/network/tasks/main.yml
---
- name: Add lo1 interface
community.general.sysrc:
name: cloned_interfaces
state: value_present
value: "lo1"
- name: Name lo1 interface bastille0
community.general.sysrc:
name: ifconfig_lo1_name
value: "bastille0"
notify: netif cloneup
- meta: flush_handlers
roles/network/handlers/main.yml
---
- name: netif cloneup
shell: service netif cloneup
Firewall role
Create a firewall role.
roles/firewall/tasks/main.yml
---
- name: enable pf
community.general.sysrc:
name: pf_enable
value: "YES"
notify: start pf
- name: enable pflog
community.general.sysrc:
name: pflog_enable
value: "YES"
notify: start pflog
- name: template pf.conf
template:
src: pf.conf.j2
dest: /etc/pf.conf
notify: reload pf
- meta: flush_handlers
roles/firewall/templates/pf.conf.j2
ext_if="{{ ansible_default_ipv4.interface }}"
### Default block policy is to return a reset packet
set block-policy return
### Reassemble fragmented packets
scrub in on $ext_if all fragment reassemble
### Ignore loopback interface
set skip on lo
### Allow empty table to exist
table <jails> persist
### Nat in jails table
nat on $ext_if from <jails> to any -> ($ext_if:0)
### Static rdr
# rdr pass inet proto tcp from any to any port {80, 443} -> 10.17.89.45
### Enable dynamic rdr (see below)
rdr-anchor "rdr/*"
### Block on incoming traffic
block in all
### Allow outgoing, skip others rules if match, and track connections
pass out quick keep state
### Block all incoming traffic from the $ext_if subnet which is not from $ext_if interface
### And block incoming traffic from $ext_if IP on $ext_if interface
antispoof for $ext_if inet
### Allow SSH
pass in inet proto tcp from any to any port ssh flags S/SA keep state
We use async on pf start handler to keep ansible connection up.
For the reload pf handler, we first test that the config file is valid with -n and apply the configuration only if it succeed.
roles/firewall/handlers/main.yml
---
- name: start pf
service:
name: pf
state: started
async: 45
poll: 5
- name: start pflog
service:
name: pflog
state: started
- name: reload pf
shell: pfctl -nf /etc/pf.conf && pfctl -f /etc/pf.conf
Jails role
Install and configure Bastille
Create a role jails.
Bastille will be configured to use ZFS.
roles/jails/tasks/main.yml
---
- name: install bastille
pkgng:
name: bastille
- name: enable bastille
community.general.sysrc:
name: bastille_enable
value: "YES"
- name: add bastille devfs rule
blockinfile:
path: /etc/devfs.rules
marker: "<!-- {mark} ANSIBLE MANAGED vnet -->"
create: yes
block: |
[bastille_vnet=13]
add path 'bpf*' unhide
- name: enable zfs for bastille
community.general.sysrc:
name: "{{ item.name }}"
value: "{{ item.value }}"
path: /usr/local/etc/bastille/bastille.conf
loop:
- { name: "bastille_zfs_enable", value: "YES" }
- { name: "bastille_zfs_zpool", value: "zroot" }
Bootstrap a release
Bootstrap the latest realease and configure it to use latest pkgs.
Releases in Bastille is the template which will be use to layer up your jails.
So each configuration made to a release will be applied to all new jails created from this release.
Add a var to group_vars/all.yml
release: 13.0-RELEASE
Then, add tasks to bootstrap the release from that var.
- name: bootstrap {{ release }} release
shell: "bastille bootstrap {{ release }}"
args: creates="/usr/local/bastille/releases/{{ release }}"
- name: configure bootstrap to use latest pkgs
replace:
path: "/usr/local/bastille/releases/{{ release }}/etc/pkg/FreeBSD.conf"
regexp: '^(.*)quarterly(.*)$'
replace: '\1latest\2'
- name: update bootstrap
shell: "bastille update {{ release }}"
Web role
Prepare the nginx template
Create a role nginx.
Here’s the interesting part. With a Bastillefile, you automate your service provisionning.
Here we tell the template to install nginx and enable it. Then we create our /data/www dir in the jail, to bind the one from the host in it. We also overlay the nginx config file with CP usr .. Finally we check if the config file is valid and then restart the service.
The RDR line dynamically generate a rule for pf to redirect the http port from the host to the jail.
roles/nginx/files/Bastillefile
PKG nginx
SYSRC nginx_enable=YES
CMD mkdir -p /data/www
CP usr .
CMD nginx -t
SERVICE nginx restart
FSTAB /data/www data/www nullfs ro 0 0
RDR tcp 80 80
roles/nginx/tasks/main.yml
---
- name: create services template dir
file:
path: "/usr/local/bastille/templates/services/{{ role_name }}"
state: directory
recurse: yes
- name: copy template config files
copy:
src: Bastillefile
dest: "/usr/local/bastille/templates/services/{{ role_name }}/"
- name: create config path
file:
path: "/usr/local/bastille/templates/services/{{ role_name }}/usr/local/etc/nginx/"
state: directory
recurse: yes
- name: copy config file
copy:
src: nginx.conf
dest: "/usr/local/bastille/templates/services/{{ role_name }}/usr/local/etc/nginx/"
- name: create data/www dataset
community.general.zfs:
name: zroot/www
state: present
extra_zfs_properties:
mountpoint: /data/www
roles/nginx/files/nginx.conf
http {
server {
listen 80;
server_name localhost;
location / {
root /data/www;
index index.html index.htm;
}
}
}
Add the task to copy the website to the host dir, mounted in the jail.
- name: copy index.html
copy:
src: index.html
dest: /data/www/
roles/nginx/files/index.html
<html>
<p>A website without any JS !</p>
</html>
Create a nginx jail
We set, at jail creation, its static IP in any private subnet which differs from your gateway one, following the advice of the Bastille README.md.
Pick any private address and be done with it. These are all isolated networks. In the end, what matters is you can map host:port to container:port reliably, and we can.
Add your Jail IP to group_vars/all.yml
jails:
nginx: 10.0.0.1
roles/nginx/tasks/main.yml
- name: create jail
shell: "bastille create {{ role_name }} {{ release }} {{ jails[role_name] }}"
args:
creates: /usr/local/bastille/jails/{{ role_name }}
- name: start jail
# https://github.com/BastilleBSD/bastille/issues/342
shell: bastille start {{ role_name }} || true
Template the nginx jail
- name: template jail
shell: "bastille template {{ role_name }} services/{{ role_name }}"
Run the playbook
$ ansible-playbook playbook.yml -t network,firewall,jails,nginx
PLAY [host-test] ***************************************************************************************************
TASK [Gathering Facts] *********************************************************************************************
ok: [host-test]
TASK [network : Add lo1 interface] *********************************************************************************
changed: [host-test]
TASK [network : Name lo1 interface bastille0] **********************************************************************
changed: [host-test]
TASK [network : meta] **********************************************************************************************
RUNNING HANDLER [network : netif cloneup] **************************************************************************
changed: [host-test]
TASK [firewall : enable pf] ****************************************************************************************
changed: [host-test]
TASK [firewall : enable pflog] *************************************************************************************
changed: [host-test]
TASK [firewall : template pf.conf] *********************************************************************************
changed: [host-test]
TASK [firewall : meta] *********************************************************************************************
RUNNING HANDLER [firewall : start pf] ******************************************************************************
changed: [host-test]
RUNNING HANDLER [firewall : start pflog] ***************************************************************************
changed: [host-test]
RUNNING HANDLER [firewall : reload pf] *****************************************************************************
changed: [host-test]
TASK [jails : install bastille] ************************************************************************************
changed: [host-test]
TASK [jails : enable bastille] *************************************************************************************
changed: [host-test]
TASK [jails : add bastille devfs rule] *****************************************************************************
changed: [host-test]
TASK [jails : enable zfs for bastille] *****************************************************************************
changed: [host-test] => (item={'name': 'bastille_zfs_enable', 'value': 'YES'})
changed: [host-test] => (item={'name': 'bastille_zfs_zpool', 'value': 'zroot'})
TASK [jails : bootstrap 13.0-RELEASE release] **********************************************************************
changed: [host-test]
TASK [jails : configure bootstrap to use latest pkgs] **************************************************************
changed: [host-test]
TASK [jails : update bootstrap] ************************************************************************************
changed: [host-test]
TASK [nginx : create services template dir] ************************************************************************
changed: [host-test]
TASK [nginx : copy template config files] **************************************************************************
changed: [host-test]
TASK [nginx : create config path] **********************************************************************************
changed: [host-test]
TASK [nginx : copy config file] ************************************************************************************
changed: [host-test]
TASK [nginx : create data/www dir] *********************************************************************************
changed: [host-test]
TASK [nginx : copy index.html] *************************************************************************************
changed: [host-test]
TASK [nginx : create jail] *****************************************************************************************
changed: [host-test]
TASK [nginx : start jail] ******************************************************************************************
changed: [host-test]
TASK [nginx : template jail] ***************************************************************************************
changed: [host-test]
PLAY RECAP *********************************************************************************************************
host-test : ok=26 changed=25 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Test the service
From the server.
$ curl http://10.0.0.1
<html>
<p>A website without any JS !</p>
</html>
From a client in the gateway subnet, if the dynamic RDR worked, it should be reachable.
$ curl http://192.168.0.100
<html>
<p>A website without any JS !</p>
</html>
If your server is in your DMZ, then your service in reachable from internet too.
You can now easily add new services by creating one role per service and use the nginx one as exemple.