From bb7efcd1dd05524afe7ffabec8b603e1aad790ba Mon Sep 17 00:00:00 2001 From: Fabian Arrotin Date: Jun 14 2022 06:06:11 +0000 Subject: Adding simple notes about lsyncd replication between pagure1 and pagure2 Signed-off-by: Fabian Arrotin --- diff --git a/docs/buildsys/git.md b/docs/buildsys/git.md index 50600c6..91cebe5 100644 --- a/docs/buildsys/git.md +++ b/docs/buildsys/git.md @@ -59,3 +59,23 @@ Assuming that we are given a [list](https://pagure.io/centos-infra/issue/393), w cat kmod-list |while read project ; do pagure-batch-create-repo $project "the $project repo" rpms prod;done ``` + +## Infra notes + +We host one public instance (git.centos.org) but we have a secondary one (temporary removed from ansible for the pagure role) that is kept in sync for the git repositories and lookaside content (not for mysql database though) +Both instances are declared in `pagure-src-nodes` ansible group , and so inheriting from variables declared at that level + +Worth knowing that it's a "poor man" replication as it's not using any shared storage or block device replication but simply [lsyncd](https://github.com/lsyncd/lsyncd) (packaged as rpm in fedora/epel/centos), so that one there is a push/write/mod on git.centos.org, it automatically triggers a rsync replication in *one way* to the second instance. See `host_vars` in ansible inventory about how lsyncd is configured and obviously how rsyncd is configured on the second instance to restrict which machine can push to which rsyncd module/target + +These two instances are Virtual Machines running RHEL8, on top of different hypervisors (see the `kvm_host` variable in inventory) to ensure that one physical node issue wouldn't put both instances down. + +Apart from replication, there is the obvious backup plan that saves mysqldump and whole lookaside and git repositories on central backup pool node. + +In case of failover, one has to : + + * restore last mysqldump + * replay the pagure role (pagure-src-nodes group membership) + * eventually just fix the permissions on directories + * update A records for git.centos.org + +