# Ansible Environments

  Apart from the roles and playbooks which are all common for *all* environments, each env will use its own :

  To keep it simple, we try to rely only on wordpress 'core' and just deploy some needed plugins, like : 

   * classic-editor : request from Community Manager to let people edit easily drafts/posts

   * akismet : classic one that is used against spammers

  ## Operations

  It's hosted at [https://lists.centos.org](https://lists.centos.org) and on a sponsored node.

  Some links:

  ## Authentication

  When a user is added in a SIG group , and logs in again, its membership will be reflected at the pagure/git.centos.org side.

  To build artifacts that will be released/pushed to the mirror network, CentOS Project relies on distributed build systems, mostly :

    * [Koji](https://pagure.io/koji)

  While reimzul is still being used for CentOS Linux 7 rpm builds, koji is now the one being used for CentOS Linux 8 and all Stream releases (but different environments)

  We adviced you to read upstream koji documentation to have details of all needed components and how they can be combined all together to be able to build/distribute packages, but here is a quick overview :

  ## Authentication

  Same rule applies for `users` : they need to be authenticated with valid TLS certificated signed by same CA but we'll consider two kind of users:

  ## Koji tags structure

  The proposed and agreed levels are :

  ## Available Environments

  While the same automation code should be used to configure all infra/services nodes within CentOS Infra, it's worth knowing that we still "divide" it into sub-sections, and so having different environments.

    * `CentOS main` : if not defined, all the nodes considered as "production" nodes and managed as such

      * `CentOS staging` (STG) : pre-prod environment, with limited number of nodes, but mostly used to test changes/deployments before being rolled-out to the `CentOS main` one

  ## CentOS Infra team

  CentOS Infra team though is in charge of the [haproxy](https://github/centos/ansible-role-haproxy) node in front of openshift and routing requests to correct openshift compute nodes from Fedora infra.

  All the needed variables for haproxy are stored in the CentOS ansible inventory, either through group_vars and/or host_vars variables

  ### Identity Provider (IdP)

  We also have a specific `acme.centos.org` zone, that is only use for one specific purpose : creating on the fly TXT records that will be used by LetsEncrypt/ACME for DNS challenge.

  For this we use [acme.sh](https://github.com/Neilpang/acme.sh) tool that will do that automatically for us : it will use nsupdate with specific allowed key to create dynamically the needed record that ACME server will verify to validate and then sign the CSR.

  Some pointers:

  Here is an an overview of the deployed ocp ci cluster : 

  Apart from the FAS/auth/EFS volumes, it's worth knowing that : 

  All that is possible at various levels :

   * Machine/Infra level : granted shell/elevated rights based on Ansible rights

  ## Onboarding infra members

  While being part of the `sig-infra` group doesn't grant any shell/sudo permission *anywhere* , it at least reflect that new person joigning the team will be a PoC for infra and also automatically granted : 

   * rights to build/promote pkgs on the [infra koji tags](https://cbs.centos.org/koji/search?match=glob&type=tag&terms=infra*)

  ### Ansible inventory access (for "full" sysadmin)

  From that point, next time Ansible will be ran across servers fleet (either automatically through central mgmt station *or* manually for the machines in the `manual-run` specific group), it will add the new sysadmin (or modify/remove) on the nodes.

        * have enabled OTP on their account for authentication services

  So after you've added the gpg in your own keyring, you can add it to git-crypt

  ```

  ## Steps

  2. Create an OpenShift namespace/project (done with ansible)

  3. Create a PV(Persistent Volume) and deploy Jenkins instance (done with ansible)

  ### Create an OpenShift namespace

  ## This guide is for admins to troubleshoot seamicro chassis when needed

  !!! note

    * Delete/reinstall the (virtual) machine (cleaning up)

      * If hosted within Red Hat DC, update [internal ip inventory](https://docs.google.com/spreadsheets/d/1K-aewLJ17z3pRC6K5qyBRJYtNXy1WcxRSVwPkGf4NXQ) (Obviously need RH SSO and permission)

    * Remove it from Ansible inventory (and search for references for that node in case of)

    * Remove it from Zabbix monitoring

    * (optional and if sponsored : remove sponsor logo from [website git repo](https://git.centos.org/centos/centos.org) )

    * we receive initial credentials

    * we collect needed informations (like ipv4/ipv6 address[es], dns resolvers, etc)

    * we perform remotely (without remote console access) a reinstall on itself (faster then auditing the state in which we receive a machine) that is reinstalled following our standards

  ## Adding node

  Once the node is available, we need *once* to initialize the node to confirm access and ssh host key/fingerprint and then sign it with our SSH CA.

    * (optional) retrieve the public IP and allow incoming connections from that new ip for some infra services restricted by iptables

    * retrieve ssh host keys, sign these and push the signed 

    * play the `baseline` role (common for *all* nodes but with different settings, based on inventory

      * if that's a RHEL host, either point to internal mirror, or RH CDN (based on `rhel_host_use_cdn` variable/boolean in ansible)

    * (optional): play other roles that are tied to ansible inventory group membership (if you added the host already in some specific groups) 

  Except for some identified ssh Jump Hosts, or for public services over ssh (like pushing to git.centos.org), the tcp/22 port used by sshd is [firewalled](https://github.com/CentOS/ansible-role-iptables/blob/master/defaults/main.yml#L11) on almost all the fleet.

    * a valid user pub key is injected

    * tcp/22 isn't blocked through iptables from ansible

    * Doesn't permit password authentication

    * specify the `HostCertificate` (see the node about CA sshd host above)

  ## Bastion hosts