# CentOS Infrastructure Team

## Overview

CentOS Infra is mainly managed by the [Community Platform Engineering](https://docs.fedoraproject.org/en-US/cpe/) Team, but also accepting infra contributions from Community Contributors, which can be delegated rights on parts of the infra for service[s] they'd like to contribute to, or even be responsible for.

All that is possible at various levels :

 * Application level : granted needed credentials (usually authenticated through [ACO](authentication.md))

 * Ansible role level : through Pull Requests against our [Ansible](../ansible/index.md) roles and/or inventories

 * Machine/Infra level : granted shell/elevated rights based on Ansible rights

## Onboarding infra members

This is an overview of needed steps to onboard a new sysadmin, having so access everywhere :

!!! info

    Worth knowing that all explained steps *don't* have to be all applied. 

    Example: someone can be granted `koji` build right for the infra tags, because also of needed delegation for just koji/cbs.centos.org but not needing shell/access anywhere else, so don't apply *blindly* this process !

### IPA Group membership

While being part of the `sig-infra` group doesn't grant any shell/sudo permission *anywhere* , it at least reflect that new person joigning the team will be a PoC for infra and also automatically granted : 

 * rights to build/promote pkgs on the [infra koji tags](https://cbs.centos.org/koji/search?match=glob&type=tag&terms=infra*)

 * a `@centosproject.org` email address (see also the [postfix](mailservers.md) section)

### Ansible inventory access (for "full" sysadmin)

Based on the [Environment](../index.md#available-environments) that new infra team member needs access to (delegation, as some are in charge of CentOS CI but not -yet- other parts, etc), one needs to be added in a specific ansible list/variables in inventory, the [admins_list](https://github.com/CentOS/ansible-role-baseline/blob/master/defaults/main.yml#L5) that contains list of shell accounts to create , with their ssh pub key and if they are granted sudo rights.

From that point, next time Ansible will be ran across servers fleet (either automatically through central mgmt station *or* manually for the machines in the `manual-run` specific group), it will add the new sysadmin (or modify/remove) on the nodes.

To retrieve a sysadmin ssh public key, gpg key or other needed informations, you can directly query IPA through fajson (it needs first to have a working kerberos ticket, so don't forget to `kinit` first) :

```

fas_user="arrfab"

curl --silent --fail --negotiate -u : https://fasjson.fedoraproject.org/v1/users/${fas_user}/|jq

```

!!! danger ""

    It's *required* that CentOS Infra members, when they'll be in charge of multiple services and granted elevated rights, will :

      * use GnuPG for encryption (for mails and/or to be able to access ansible inventory git repositories) and so public gpg key available through fasjson/IPA/noggin portal

      * have their ssh pub key also passphrase protected and with publicy key available on fasjson/IPA/noggin

      * have enabled OTP on their account for authentication services

Once you have verified the GPG public key, you can (for `git-crypt`ed git repositories for ansible, add the new collaborator like this (do this on *each* [git repo](../ansible/topology.md) that the new sysadmin is granted access to).

So after you've added the gpg in your own keyring, you can add it to git-crypt

```

git-crypt add-gpg-user --trusted 

[master e28a784] Add 1 git-crypt collaborator

 1 file changed, 1 insertion(+)

 create mode 100644 .git-crypt/keys/default/0/<usually_user_gpg_pub_key>.gpg

git push

```

### Zabbix access (for "full" sysadmin)

One existing administrator can create (through WebUI - Administration/Users - or through Zabbix API), a new user with the following settings:

 * User section

   * Alias (username) => matching existing `nick` from authentication

   * Groups => can be `Zabbix administrators` for full admins

   * password => randomly generated *and* initial one sent to new sysadmin by gpg encrypted email, to be changed by user after (note that when we'll switch zabbix to new auth system, that step will be skipped completely)

 * Media section 

   * At least add the `email` media type, sending to his registered email in IPA

   * Filter out `Not classified` and `Information` severity levels but users can change/fine tune after

 * Permissions section

   * User Type => `Zabbix Super Admin` : Read-Write *everywhere* in Zabbix

### Various notes

#### IPA groups giving access 

##### git/pagure

The `centos-git-admins` IPA group will give you all needed rights in pagure/git.centos.org, so add user in that group if he needs to be able to administer this git forge solution.

##### Openshift

#### Git notifications

For the current setup for inventories (not the ones hosted on gitlab that is) we can quickly enable commits notifications like this (adapt to the needs/users/projects) :

```

# Some variables

git_basedir="/repositories/git/centos"

git_repos="ansible-filestore-ci ansible-pkistore-ci ansible-inventory-ci"

git_mailto="rcpt_1@domain.com, rcpt2@otherdomain.com"

git_mailfrom="git@centosproject.org"

pushd ${git_basedir}

for repo in ${git_repos}; do

  pushd ${repo}.git 

  git config multimailhook.mailingList "${git_mailto}"

  git config multimailhook.from ${git_mailfrom}

  pushd hooks/post-receive.d; test -e git_multimail.py || ln -s /usr/bin/git_multimail.py ; popd

  popd

done

popd

```