e6a51b
socat tunnel for encrypted rsync SST
e6a51b
====================================
e6a51b
e6a51b
`wsrep_sst_rsync_tunnel` is an extension of the rsync-based [SST](http://galeracluster.com/documentation-webpages/glossary.html#term-state-snapshot-transfer)
e6a51b
implementation that ships with mariadb. Its purpose is to encrypt
e6a51b
communication between the donor and the joiner during an SST.
e6a51b
e6a51b
Encryption is implemented by means of a socat tunnel, using OPENSSL
e6a51b
addresses. It can be configured via the regular openssl flags exposed
e6a51b
by socat.
e6a51b
e6a51b
e6a51b
## How to configure the script
e6a51b
e6a51b
This SST script can configured by setting a few keys in your favorite
e6a51b
mariadb option file in addition to the usual galera settings.
e6a51b
e6a51b
    [mysqld]
e6a51b
    ...
e6a51b
    bind_address=<node-name>
e6a51b
    wsrep_sst_method=rsync_tunnel
e6a51b
    ...
e6a51b
    
e6a51b
    [sst]
e6a51b
    tca=/path/to/your/ca-file.crt
e6a51b
    tcert=/path/to/node/certificate.crt
e6a51b
    tkey=/path/to/node/key.key
e6a51b
    sockopt=<openssl-address-options-as-per-socat-manual>
e6a51b
e6a51b
When a joiner node requests an SST, `wsrep_sst_rsync_tunnel` uses
e6a51b
socat to listen to incoming SSL connections on port 4444 in lieu of
e6a51b
the original rsync daemon. Received data will be forwarded to the
e6a51b
rscynd daemon started locally to replicate the database.
e6a51b
e6a51b
When a donor node serves the SST, `wsrep_sst_rsync_tunnel` makes
e6a51b
a series of rsync calls that target a locally started socat daemon.
e6a51b
The daemon tunnels all rsync traffic into an encrypted SSL connection
e6a51b
that targets the joiner's end of the socat tunnel.
e6a51b
e6a51b
Encryption parameters are specified under the `[sst]` group in the
e6a51b
mariadb option file, where `tkey` and `tcert` are respectively the key
e6a51b
and the certificate that are used by both sides of the socat tunnel.
e6a51b
Each node typically has a different key and cert. Both key and
e6a51b
certificate can be combined into a single PEM file and referenced by
e6a51b
`tcert`. Option `tca` holds a list of the trusted signing
e6a51b
certificates.
e6a51b
e6a51b
In case you need to tweak the creation of the SSL connection, you can
e6a51b
pass valid socat options (as per socat manual) via the `sockopt` key.
e6a51b
For debugging purpose, the exact socat command that is being executed
e6a51b
shows up in the mariadb log file.
e6a51b
e6a51b
Note that socat verifies that the certificate's commonName matches
e6a51b
that of the host that is being targeted. The target name comes from
e6a51b
the value configured in `bind_address`, so it's important that it
e6a51b
matches the certificate's commonName. An IP address can be used for
e6a51b
`bind_address`, but you may get into trouble in case different
e6a51b
hostnames resolve to the same IP (e.g. multiple networks per host).
e6a51b
e6a51b
e6a51b
## Examples of use
e6a51b
e6a51b
Suppose you're running a 3-node galera cluster
e6a51b
`node1.my.cluster`, `node2.my.cluster`, `node3.my.cluster`.
e6a51b
e6a51b
### Scenario: using self-signed certificates
e6a51b
e6a51b
On each node, create a key and a certificate, and bundle them into a
e6a51b
single PEM file. For instance on `node1.my.cluster`:
e6a51b
e6a51b
    openssl genrsa -out /tls/mysql-$(hostname -f).key 2048
e6a51b
    openssl req -new -key /tls/mysql-$(hostname -f).key -x509 -days 365000 -subj "/CN=$(hostname -f)" -out /tls/mysql-$(hostname -f).crt -batch
e6a51b
    cat /tls/mysql-$(hostname -f).key /tls/mysql-$(hostname -f).crt > /tls/mysql.pem
e6a51b
e6a51b
Then, on each node, create a cafile that will contain all the certs to
e6a51b
trust:
e6a51b
e6a51b
    for n in node1.my.cluster node2.my.cluster node3.my.cluster; do
e6a51b
       ssh $n 'cat /tls/mysql-$(hostname -f).crt' >> /tls/all-mysql.crt
e6a51b
    done
e6a51b
e6a51b
Once you have those two files on each host, you can configure the SST
e6a51b
appropriately. For instance from `/etc/my.cnf.d/galera.cnf`:
e6a51b
e6a51b
    [mysqld]
e6a51b
    ...
e6a51b
    
e6a51b
    [sst]
e6a51b
    tca=/tls/all-mysql.crt
e6a51b
    tcert=/tls/mysql.pem
e6a51b
e6a51b
### Scenario: using self-signed certificates, without verification
e6a51b
e6a51b
By default, when socat tries to establish a SSL connection to a peer,
e6a51b
it also verifies that it can trust the peer's certificate. If for some
e6a51b
reason you need to disable that feature, you can amend the previous
e6a51b
configuration with a sockopt option:
e6a51b
e6a51b
    [mysqld]
e6a51b
    ...
e6a51b
    
e6a51b
    [sst]
e6a51b
    tca=/tls/all-mysql.crt
e6a51b
    tcert=/tls/mysql.pem
e6a51b
    sockopt="verify=0"
e6a51b
e6a51b
The associated sockopt value is passed to socat when
e6a51b
the donor or the joiner configures his part of the tunnel.
e6a51b
e6a51b
Note: please do not do so in production, this is inherently insecure
e6a51b
as you will not verify the identity of the peer you're connecting to!
e6a51b
e6a51b
### Scenario: using certificates from a CA
e6a51b
e6a51b
Suppose you have a FreeIPA service which generated a key file and a
e6a51b
certificate file for the three galera nodes, respectively located at
e6a51b
/tls/mysql.key and /tls/mysql.crt.
e6a51b
e6a51b
Assuming that the certificate for the FreeIPA server is available at
e6a51b
/etc/ipa/ca.crt, you can configure you galera servers as follows:
e6a51b
e6a51b
    [sst]
e6a51b
    tca=/etc/ipa/ca.crt
e6a51b
    tcert=/tls/mysql.crt
e6a51b
    tkey=/tls/mysql.key
e6a51b
e6a51b
## License
e6a51b
e6a51b
Copyright © 2017 [Damien Ciabrini](https://github.com/dciabrin).
e6a51b
This work is derived from the original `wsrep_rsync_sst`, copyright
e6a51b
© 2010-2014 [Codership Oy](https://github.com/codership).
e6a51b
Released under the GNU GPLv2.