14 Jan 2022

Time to check Galera Cluster security?

There are plenty of articles on the Internet that tell you to setup a Galera Cluster by disabling an OS based firewall and also disabling SELinux. While we agree that this might be the fastest way to get your Galera Cluster setup, it is not necessarily good security hygiene, and we would prefer if you started 2022 with a bit more secure Galera Cluster!

What is SELinux? Is is Security-Enhanced Linux that allows administrators to allow who has more control over the system. It has permissive and enforcing mode, and is turned on by default in Red Hat Enterprise Linux and derivatives. It is important to remember that if you install Galera Cluster via a package that we provide, we have provided all the necessary contexts for it. You effectively do not have to disable SELinux to get started.

However, if you are using the rsync method for a physical state snapshot (SST), then you can test to see if having SELinux on makes sense.

To test it, on a donor node, type: setenforce 1 (this should be the default, for what it is worth). Then on a joiner node, trigger an SST (presuming you set it up to use rsync), by doing:

service mysql stop
rm -rf /var/lib/mysql/*
service mysql start

Look at the mysqld.log (you should always have error logging enabled; from a MySQL standpoint, this is where your generated password is stored for first login; and from a Galera Cluster standpoint this is where all kinds of important messages are written) and you’ll see it is failing to complete the SST. You can fix this quickly by doing setenforce 0 on the donor node. Now, the better thing to do is to write SELinux contexts, or just switch from using the rsync method to XtraBackup or the new CLONE SST method.

What about a firewall? We require TCP for 3306 (it is MySQL’s port), TCP & UDP for 4567 (cluster replication traffic), TCP for 4568 for Incremental State Transfers and TCP for 4444 for State Snapshot Transfers. We have extensive documentation of using firewalld, and iptables. For what it is worth, Galera Cluster also runs well on FreeBSD (and other BSD derivatives), so there is also documentation on pf.

How do you test if the firewall is the problem? On the joiner node, do:

iptables -A INPUT -p tcp --destination-port 4444 -j DROP

Now, attempt an SST just like above where you remove the data directory. Look at the log files. The fix, is of course simple:

iptables -D INPUT -p tcp --destination-port 4444 -j DROP

So here we hope you try running a more secure Galera Cluster environment this year. The Galera Manager configures this automatically for you, so you might also consider using it. We have a very active support group on GitHub issues going on as well. Do try it and check it out.