# MBBox deployment guide

## Introduction

This guide will describe the deployment of [MBBox](https://github.com/fedora-infra/mbbox) operator in OpenShift 4 cluster and it's prerequisites.

MBBox upstream documentation can be found at: https://mbbox-operator.readthedocs.io/en/latest/.

The MBBox operator has kubernetes custom resources to deploy the following components:

* koji-hub (includes koji-web)

* koji-builder

* kojira

* MBS Backend

* MBS Frontend

## Dependencies

MBBox depends on two components/services to be already deployed/available:

* PostgreSQL (>= 10.4)

* Fedora Messaging

The operator does not deploy the above components but needs two secrets, one for each component, in the same namespace custom resources (koji, koji-builder, etc) are being created.

### PostgreSQL

Secret format (all `data` values need to be base64 encoded):

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: postgres

  labels:

    app: postgres

data:

  POSTGRES_HOST: cG9zdGdyZXNxbA== # postgresql

  POSTGRES_DB: bWJveGRi # mboxdb

  POSTGRES_USER: a29qaQ== # koji

  POSTGRES_PASSWORD: bWJveA== # mbox

```

Creating the secret from the command line:

```

oc create secret generic postgres \

--from-literal=POSTGRES_HOST=postgresql \

--from-literal=POSTGRES_DB=mboxdb \

--from-literal=POSTGRES_USER=koji \

--from-literal=POSTGRES_PASSWORD=mbox

```

### Fedora Messaging

Secret format (all `data` values need to be base64 encoded):

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-hub-msg

data:

  koji.ca: |-

    Y2hhbmdlbWU=

  koji.crt: |-

    Y2hhbmdlbWU=

  koji.key: |-

    Y2hhbmdlbWU=

```

The secret can also be created from the command line:

```

oc create secret generic koji-hub-msg \

--from-file=koji.ca=/path/to/ca.pem \

--from-file=koji.crt=/path/to/cert.pem \

--from-file=koji.key=/path/to/key.pem

```

All certificates must be in PEM format.

### Storage

**NOTE:** this section describes how to deal with Openshift Persistent Volumes if the cluster being used is not using any storage provisioning operator which may require extra PV setup from the cluster admin.

The MBBox needs several Persistent Volumes created in prior to deployment:

* mbox-registry (Recommended: 100 Gi) - Used as image registry

* httpd (Minimum: 1Gi) - Used by koji-hub httpd server

* koji (Minimum: 50Gi) - Used for shared koji space

* postgres (Minimum: 5Gi) - Used by postgreSQL database

#### Persistent Volumes

##### ClaimRef

ClaimRef should be set in created Persistent Volumes (PVs) if no storage class is set in the cluster scope:

```yaml

kind: PersistentVolume

apiVersion: v1

metadata:

  annotations:

    pv.kubernetes.io/bound-by-controller: 'yes'

  selfLink: /api/v1/persistentvolumes/stg-postgres

  resourceVersion: '39237794'

  name: stg-postgres

  uid: d0e3960e-af9e-45af-8f87-ac03def7b8a5

  creationTimestamp: '2020-11-23T01:01:35Z'

...

spec:

  capacity:

    storage: 5Gi

  claimRef:

    apiVersion: v1 

    kind: PersistentVolumeClaim

    namespace: mbox # pvc namespace

    name: postgres # pvc name to bound to

  persistentVolumeReclaimPolicy: Retain

  storageClassName: slow

  volumeMode: Filesystem

  ...

```

Both `uid` and `resourceVersion` will need to be removed from the `claimRef` property if the PV needs to be rebound to another PVC:

```yaml

claimRef:

    apiVersion: v1 

    kind: PersistentVolumeClaim

    namespace: mbox-stg # new namesapce

    name: postgres-stg # new pvc name

    uid: 775c0823-47c1-46a2-b07c-612d59592430 # remove this

    resourceVersion: '39235159' #remove this

```

Both PVC and PV should have a `bound` status after a few seconds.

A note on volumes: recently created volumes may need a permission change the volume itself, the oficial openshift docs has a page dedicated to this problem and how to fix it: https://access.redhat.com/solutions/3508731

## Deployment

There are three flavors of deployment:

* Development

* Production: Root CA Only

* Production: All Certificates

What differs each deployment flavor is the amount of required steps before the actual deployment process.

You can follow the upstream documentation once those preparation steps are done: https://mbbox-operator.readthedocs.io/en/latest/deployment-guide.html

### Development

Ideal for running MBBox locally and automated tests.

The only requirement is a running openshift cluster with admin access.

All certificates, incluing the Root CA will be self signed using openssl.

### Production: Root CA Only

This deployment path requires an openshift secret in the same namespace other components will be created, such as koji.

This secret should contain a root CA certificate and private key, this way the operator will create and sign all required certificates from this root CA instead of using a self signed one.

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-hub-ca-cert

  labels:

    app: koji-hub

data:

  cert: -|

    cert_in_base64_format

  key: -|

    key_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic koji-hub-ca-cert \

--from-file=cert=/ca.pem \

--from-file=key=/key.pem

```

You may then proceed in deployment MBBox.

### Production: All Certificates

Similar to the previous section but it needs all certificates and keys to be manually created.

#### [Koji-Hub] Root CA Secret

The root certifcate which all other certificates were generated/signed from.

We only need the certificate in this case.

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-hub-ca-cert

  labels:

    app: koji-hub

data:

  cert: -|

    cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic koji-hub-ca-cert \

--from-file=cert=/ca.pem

```

#### [Koji-Hub] HTTPD Secret

Apache HTTPD certificate and private key.

**NOTES:**

* The certificate CN field must use the same value as the public hostname for koji;

* The following subject altnames must be used:

    * DNS:$KOJI_PUBLIC_HOSTNAME

    * DNS:koji-hub

    * DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc

    * DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc.cluster

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-hub-service-cert

  labels:

    app: koji-hub

data:

  tls.crt: -|

    cert_in_base64_format

  tls.key: -|

    key_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic koji-hub-service-cert \

--from-file=cert=/cert.pem \

--from-file=key=/key.pem

```

#### [Koji-Hub] Web Client Secret

Koji-Web client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;

* The certificate `CN` field must use the same value used in `web_client_username`

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-hub-web-client-cert

  labels:

    app: koji-hub

data:

  client.pem: -|

    key_and_cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic koji-hub-web-client-cert \

--from-file=client.pem=/key_and_cert.pem

```

#### [Koji-Hub] Admin Client Secret

Koji-Hub admin client secret for koji-hub API authentication.

This certificate is used in several admin level operations by the operator.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;

* The certificate `CN` field must use the same value used in `admin_username`

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-hub-admin-cert

  labels:

    app: koji-hub

data:

  client.pem: -|

    key_and_cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic koji-hub-admin-cert \

--from-file=client.pem=/key_and_cert.pem

```

#### [Koji-Builder] Client Secret

Koji-Builder client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;

* The certificate `CN` field must use the same value used in `host_name`

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: koji-builder-client-cert

  labels:

    app: koji-builder

data:

  client.pem: -|

    key_and_cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic koji-builder-client-cert \

--from-file=client.pem=/key_and_cert.pem

```

#### [Kojira] Client Secret

Kojira client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;

* The certificate `CN` field must use the same value used in `hub_username`

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: kojira-client-cert

  labels:

    app: kojira

data:

  client.pem: -|

    key_and_cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic kojira-client-cert \

--from-file=client.pem=/key_and_cert.pem

```

#### [MBS-Backend] Client Secret

MBS Backend client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;

* The certificate `CN` field must use the same value used in `hub_username`

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: mbs-client-cert

  labels:

    app: mbs-backend

data:

  client.pem: -|

    key_and_cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic mbs-client-cert \

--from-file=client.pem=/key_and_cert.pem

```

#### [MBS-Frontend] Client Secret

MBS Frontend client secret for koji-hub API authentication.

**NOTES:**

* The `client.pem` file must contain BOTH private key and certificate;

* The certificate `CN` field must use the same value used in `hub_username`

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: mbs-frontend-client-cert

  labels:

    app: mbs-frontend

data:

  client.pem: -|

    key_and_cert_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic mbs-frontend-client-cert \

--from-file=client.pem=/key_and_cert.pem

```

#### [MBS-Frontend] HTTPD Secret

Apache HTTPD certificate and private key.

**NOTES:**

* The certificate CN field must use the same value as the public hostname for mbs-frontend;

* The following subject altnames must be used:

    * DNS:$MBS_FRONTEND_PUBLIC_HOSTNAME

    * DNS:mbs-frontend

    * DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc

    * DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc.cluster

Secret format:

```yaml

apiVersion: v1

kind: Secret

metadata:

  name: mbs-frontend-service-cert

  labels:

    app: koji-hub

data:

  tls.crt: -|

    cert_in_base64_format

  tls.key: -|

    key_in_base64_format

```

You may also create it from the command line:

```sh

oc create secret generic mbs-frontend-service-cert \

--from-file=cert=/cert.pem \

--from-file=key=/key.pem

```