# MBBox deployment guide

## Introduction

This guide will describe the deployment of [MBBox](https://github.com/fedora-infra/mbbox) operator in OpenShift 4 cluster and it's prerequisites.

MBBox upstream documentation can be found at: https://mbbox-operator.readthedocs.io/en/latest/.

The MBBox operator has kubernetes custom resources to deploy the following components:

* koji-hub (includes koji-web)
* koji-builder
* kojira
* MBS Backend
* MBS Frontend

## Dependencies

MBBox depends on two components/services to be already deployed/available:

* PostgreSQL (>= 10.4)
* Fedora Messaging

The operator does not deploy the above components but needs two secrets, one for each component, in the same namespace custom resources (koji, koji-builder, etc) are being created.

### PostgreSQL

Secret format (all `data` values need to be base64 encoded):

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: postgres
  labels:
    app: postgres
data:
  POSTGRES_HOST: cG9zdGdyZXNxbA== # postgresql
  POSTGRES_DB: bWJveGRi # mboxdb
  POSTGRES_USER: a29qaQ== # koji
  POSTGRES_PASSWORD: bWJveA== # mbox
```

Creating the secret from the command line:

```
oc create secret generic postgres \
--from-literal=POSTGRES_HOST=postgresql \
--from-literal=POSTGRES_DB=mboxdb \
--from-literal=POSTGRES_USER=koji \
--from-literal=POSTGRES_PASSWORD=mbox
```

### Fedora Messaging

Secret format (all `data` values need to be base64 encoded):

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-msg
data:
  koji.ca: |-
    Y2hhbmdlbWU=
  koji.crt: |-
    Y2hhbmdlbWU=
  koji.key: |-
    Y2hhbmdlbWU=
```

The secret can also be created from the command line:

```
oc create secret generic koji-hub-msg \
--from-file=koji.ca=/path/to/ca.pem \
--from-file=koji.crt=/path/to/cert.pem \
--from-file=koji.key=/path/to/key.pem
```

All certificates must be in PEM format.

### Storage

**NOTE:** this section describes how to deal with Openshift Persistent Volumes if the cluster being used is not using any storage provisioning operator which may require extra PV setup from the cluster admin.

The MBBox needs several Persistent Volumes created in prior to deployment:

* mbox-registry (Recommended: 100 Gi) - Used as image registry
* httpd (Minimum: 1Gi) - Used by koji-hub httpd server
* koji (Minimum: 50Gi) - Used for shared koji space
* postgres (Minimum: 5Gi) - Used by postgreSQL database

#### Persistent Volumes

##### ClaimRef

ClaimRef should be set in created Persistent Volumes (PVs) if no storage class is set in the cluster scope:

```yaml
kind: PersistentVolume
apiVersion: v1
metadata:
  annotations:
    pv.kubernetes.io/bound-by-controller: 'yes'
  selfLink: /api/v1/persistentvolumes/stg-postgres
  resourceVersion: '39237794'
  name: stg-postgres
  uid: d0e3960e-af9e-45af-8f87-ac03def7b8a5
  creationTimestamp: '2020-11-23T01:01:35Z'
...
spec:
  capacity:
    storage: 5Gi
  claimRef:
    apiVersion: v1 
    kind: PersistentVolumeClaim
    namespace: mbox # pvc namespace
    name: postgres # pvc name to bound to
  persistentVolumeReclaimPolicy: Retain
  storageClassName: slow
  volumeMode: Filesystem
  ...
```

Both `uid` and `resourceVersion` will need to be removed from the `claimRef` property if the PV needs to be rebound to another PVC:

```yaml
claimRef:
    apiVersion: v1 
    kind: PersistentVolumeClaim
    namespace: mbox-stg # new namesapce
    name: postgres-stg # new pvc name
    uid: 775c0823-47c1-46a2-b07c-612d59592430 # remove this
    resourceVersion: '39235159' #remove this
```

Both PVC and PV should have a `bound` status after a few seconds.

A note on volumes: recently created volumes may need a permission change the volume itself, the oficial openshift docs has a page dedicated to this problem and how to fix it: https://access.redhat.com/solutions/3508731


## Deployment

There are three flavors of deployment:

* Development
* Production: Root CA Only
* Production: All Certificates

What differs each deployment flavor is the amount of required steps before the actual deployment process.

You can follow the upstream documentation once those preparation steps are done: https://mbbox-operator.readthedocs.io/en/latest/deployment-guide.html

### Development

Ideal for running MBBox locally and automated tests.

The only requirement is a running openshift cluster with admin access.

All certificates, incluing the Root CA will be self signed using openssl.

### Production: Root CA Only

This deployment path requires an openshift secret in the same namespace other components will be created, such as koji.

This secret should contain a root CA certificate and private key, this way the operator will create and sign all required certificates from this root CA instead of using a self signed one.

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-ca-cert
  labels:
    app: koji-hub
data:
  cert: -|
    cert_in_base64_format
  key: -|
    key_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic koji-hub-ca-cert \
--from-file=cert=/ca.pem \
--from-file=key=/key.pem
```

You may then proceed in deployment MBBox.

### Production: All Certificates

Similar to the previous section but it needs all certificates and keys to be manually created.

#### [Koji-Hub] Root CA Secret

The root certifcate which all other certificates were generated/signed from.

We only need the certificate in this case.

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-ca-cert
  labels:
    app: koji-hub
data:
  cert: -|
    cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic koji-hub-ca-cert \
--from-file=cert=/ca.pem
```

#### [Koji-Hub] HTTPD Secret

Apache HTTPD certificate and private key.

**NOTES:**

* The certificate CN field must use the same value as the public hostname for koji;
* The following subject altnames must be used:
    * DNS:$KOJI_PUBLIC_HOSTNAME
    * DNS:koji-hub
    * DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc
    * DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc.cluster

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-service-cert
  labels:
    app: koji-hub
data:
  tls.crt: -|
    cert_in_base64_format
  tls.key: -|
    key_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic koji-hub-service-cert \
--from-file=cert=/cert.pem \
--from-file=key=/key.pem
```

#### [Koji-Hub] Web Client Secret

Koji-Web client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;
* The certificate `CN` field must use the same value used in `web_client_username`

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-web-client-cert
  labels:
    app: koji-hub
data:
  client.pem: -|
    key_and_cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic koji-hub-web-client-cert \
--from-file=client.pem=/key_and_cert.pem
```

#### [Koji-Hub] Admin Client Secret

Koji-Hub admin client secret for koji-hub API authentication.

This certificate is used in several admin level operations by the operator.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;
* The certificate `CN` field must use the same value used in `admin_username`

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-admin-cert
  labels:
    app: koji-hub
data:
  client.pem: -|
    key_and_cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic koji-hub-admin-cert \
--from-file=client.pem=/key_and_cert.pem
```

#### [Koji-Builder] Client Secret

Koji-Builder client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;
* The certificate `CN` field must use the same value used in `host_name`

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: koji-builder-client-cert
  labels:
    app: koji-builder
data:
  client.pem: -|
    key_and_cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic koji-builder-client-cert \
--from-file=client.pem=/key_and_cert.pem
```

#### [Kojira] Client Secret

Kojira client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;
* The certificate `CN` field must use the same value used in `hub_username`

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: kojira-client-cert
  labels:
    app: kojira
data:
  client.pem: -|
    key_and_cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic kojira-client-cert \
--from-file=client.pem=/key_and_cert.pem
```

#### [MBS-Backend] Client Secret

MBS Backend client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;
* The certificate `CN` field must use the same value used in `hub_username`

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: mbs-client-cert
  labels:
    app: mbs-backend
data:
  client.pem: -|
    key_and_cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic mbs-client-cert \
--from-file=client.pem=/key_and_cert.pem
```

#### [MBS-Frontend] Client Secret

MBS Frontend client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;
* The certificate `CN` field must use the same value used in `hub_username`

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: mbs-frontend-client-cert
  labels:
    app: mbs-frontend
data:
  client.pem: -|
    key_and_cert_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic mbs-frontend-client-cert \
--from-file=client.pem=/key_and_cert.pem
```

#### [MBS-Frontend] HTTPD Secret

Apache HTTPD certificate and private key.

**NOTES:**

* The certificate CN field must use the same value as the public hostname for mbs-frontend;
* The following subject altnames must be used:
    * DNS:$MBS_FRONTEND_PUBLIC_HOSTNAME
    * DNS:mbs-frontend
    * DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc
    * DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc.cluster

Secret format:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: mbs-frontend-service-cert
  labels:
    app: koji-hub
data:
  tls.crt: -|
    cert_in_base64_format
  tls.key: -|
    key_in_base64_format
```

You may also create it from the command line:

```sh
oc create secret generic mbs-frontend-service-cert \
--from-file=cert=/cert.pem \
--from-file=key=/key.pem
```