dcavalca / centos / centos-infra-docs
Forked from centos/centos-infra-docs 2 months agoCentOS Mirror managements SOP
Depending on the CentOS version, (8-stream, 7 or 9-stream), yum/dnf repositories on client machines are configured to either use mirrorlist.centos.org or mirrors.centos.org.
mirrors.centos.org is an alias to mirrors.fedoraproject.org and so is a MirrorManager instance hosted by Fedora infra team. Let's see which process to follow when we get a ticket (see Howto on the wiki ) to add/remove/modify a mirror entry in corresponding DB.
CentOS Stream 9
When we get a new request for CentOS Stream 9 mirror, the process goes like this :
- ticket is reviewed by infra-and-releng team and tagged with
mirrormanagerlabel -
assignee should :
- verify that mirror isn't located at GeoIP level from an embargoed country
- verify that provided URLs (http/https and - optional - rsync) are working
- follow the link to that mirror in mirrormanager (needs
sysadmin-webFAS group membership) - add the
CentOScategory and then add the URLs -
In
mirmon.rdu2.centos.org, update the file/etc/mirmon/stream-mirror.listwith the IPs to be add in ACL (one IP per line), like the following pattern:![]()
-
update ticket with info and close
CentOS Linux 7 and 8-stream
When we get a new request for CentOS Linux 7 or 8-stream, the process goes like this :
- ticket is reviewed by infra-and-releng team and tagged with
mirror-linuxlabel - assignee should :
- verify that mirror isn't located at GeoIP level from an embargoed country
- verify that provided URLs (http/https and - optional - rsync) are working
- update ticket with info and close
Adding new mirror
To help with these steps, there is a simple wrapper/helper script on the machine behind https://mirror-status.centos.org. It will run some pre-flights checks and output the SQL statements to copy/paste into mysql to add the new mirror in mirmon DB.
The process to launch, when one has a shell on the mirmon node, is /var/lib/centos-mirrors/mirror-geo-check.py
Follow the steps and then copy/paste mysql statement into mysql (against mirmon database)
Note
attention that some fields in mysql db are primary key and wouldn't accept duplicate, so in case one "sponsor" would like to have multiple mirrors, just slightly modify the name (like empty space before) before adding to MySQL
Updating existing mirror
Let's explore at a way to modify existing mirror in the database, and it will have to be done manually directly into mysql.
we have two tables that would need eventually to be updated :
- mirrors
- ipaddresses
Here are the schemas :
MariaDB [mirmon]> describe mirrors;
+---------------------+-------------------------------------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+-------------------------------------------------------------------------+------+-----+---------+----------------+
| mirror_id | mediumint(8) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(100) | NO | UNI | | |
| location-major | varchar(20) | NO | | | |
| locmajidx | int(11) | NO | | 0 | |
| location-minor | varchar(20) | NO | MUL | | |
| http | varchar(127) | NO | MUL | | |
| https | varchar(127) | NO | MUL | | |
| ftp | varchar(127) | NO | | | |
| rsync | varchar(127) | NO | | | |
| speed | enum('T1','T2','T3') | NO | | T1 | |
| bandwidth | varchar(10) | NO | | | |
| status | enum('Active','Disabled','Dead','Master','Gone') | NO | MUL | Active | |
| state | enum('current','behind','out of date','not syncing','timeout','broken') | NO | | current | |
| Contact-name | varchar(40) | NO | | | |
| Contact-tel | varchar(40) | NO | | | |
| contact-email | varchar(127) | NO | | | |
| comments | varchar(16384) | NO | | | |
| access-ips-or-hosts | varchar(128) | NO | | | |
| access-hosts | varchar(128) | NO | | | |
| Type | enum('Direct','Indirect','Personal','Internal','Master','Slave') | NO | MUL | Direct | |
| restructured | enum('no','yes') | NO | | yes | |
| centostext | varchar(20) | NO | | | |
| url | varchar(128) | NO | | | |
| info_note | varchar(40) | NO | | | |
| notes_private | varchar(16384) | NO | | | |
| infoblock | varchar(128) | NO | | | |
| graphic_url | varchar(128) | NO | | | |
| centos_2 | enum('yes','no') | NO | | yes | |
| centos_3 | enum('yes','no') | NO | | yes | |
| centos_4 | enum('yes','no') | NO | | yes | |
| centos_5 | enum('yes','no') | NO | | yes | |
| centos_6 | enum('yes','no') | NO | | yes | |
| centos_all | enum('yes','no') | NO | | yes | |
| arch_all | enum('yes','no') | NO | | yes | |
| arches | varchar(64) | NO | | | |
| dvd-iso | enum('no','yes') | NO | | no | |
| dvd_3 | enum('no','yes') | NO | | no | |
| dvd_4 | enum('no','yes') | NO | | no | |
| dvd_5 | enum('no','yes') | NO | | no | |
| dvd_6 | enum('no','yes') | NO | | yes | |
| dvd-iso-host | enum('no','yes') | NO | | yes | |
| rsync-dvd-host | varchar(127) | NO | | | |
| cc | char(2) | NO | MUL | | |
| continent | enum('us','eu','af','oc','ap','sa') | NO | MUL | us | |
| centos_code | char(2) | NO | MUL | | |
| priority | int(11) | NO | | 0 | |
| use_in_mirrorlists | enum('yes','no') | NO | | yes | |
| use_https_for_isos | enum('yes','no') | NO | | yes | |
| altarch | enum('no','yes') | NO | MUL | no | |
| altarch_http | varchar(127) | NO | | | |
| altarch_https | varchar(127) | NO | | | |
| altarch_ftp | varchar(127) | NO | | | |
| altarch_rsync | varchar(127) | NO | | | |
+---------------------+-------------------------------------------------------------------------+------+-----+---------+----------------+
53 rows in set (0.00 sec)
MariaDB [mirmon]> describe ipaddresses;
+-----------+-----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-----------------------+------+-----+---------+----------------+
| ip_id | mediumint(8) unsigned | NO | PRI | NULL | auto_increment |
| mirror_id | mediumint(8) unsigned | YES | MUL | NULL | |
| ip | varchar(43) | NO | MUL | NULL | |
| expires | date | YES | | NULL | |
| asn | int(10) unsigned | YES | MUL | NULL | |
+-----------+-----------------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
If a mirror would like to have its name or url changed, the relevant fields would be either name, url, http, https, rsync , etc ... If they ask for a new ip address to be removed or added or modified in the ACL (rsync isn't widely opened, but rather restricted to ip addresses in the DB), it should be updated in the ipaddresses table.
The "link" between mirrors and ipaddresses is mirror_id :
MariaDB [mirmon]> select mirror_id from mirrors where http like '%my.mirror.com%'; +-----------+ | mirror_id | +-----------+ | 445 | +-----------+ 1 row in set (0.00 sec) MariaDB [mirmon]> select * from ipaddresses where mirror_id='445'; +-------+-----------+--------------------+---------+------+ | ip_id | mirror_id | ip | expires | asn | +-------+-----------+--------------------+---------+------+ | 311 | 445 | 192.168.1.0/24 | NULL | NULL | | 1780 | 445 | 192.168.2.0/24 | NULL | NULL | | 1781 | 445 | 2620:10d:: | NULL | NULL | +-------+-----------+--------------------+---------+------+ 3 rows in set (0.00 sec)
Note
if we have a ticket about a mirror not able to sync anymore, but that we have its ip address in DB, we should always first check that status isn't Gone or Disabled as that would explain why they can't rsync from us anymore : The ACL is using only active mirrors
select mirror_id,state,status from mirrors where http like '%my.mirror.com%'; +-----------+---------+--------+ | mirror_id | state | status | +-----------+---------+--------+ | 74 | current | Active | +-----------+---------+--------+ 1 row in set (0.00 sec)
From that point, it's a matter or updating directly with some insert or update sql statements where needed