## Example 4: Deploy Simple OpenShift Template using nginx and GlusterFS PersistentVolume and Claim
---


**Summary**: 

At this point, we have seen how to deploy simple nginx applications that use multiple techniques to persist data storage.  In OSEv3, there is also a simple technique to be able to import Templates, which are preconfigurd quick application that can be deployed by users of a project.  These templates can be simple pod/service/rc/route type of applications or can be actual source code that can be built and put into a template - known as source to image (S2I).  For more information on this see [S2I Introduction training example](https://github.com/openshift/training/blob/master/08-S2I-Introduction.md). For this simple example, we will deploy another NGINX application, utilizing our existing storage (PV and PVC).  At this point we already have our endpoints and pv and pvc created and bound (if you do not, see the previous examples to complete those steps).  For this example we are using the Gluster volume "myVol2".  It has not been used by any of the previous examples.


        [root@ose1 usr_configs]# oc get endpoints
        NAME                ENDPOINTS
        glusterfs-cluster   192.168.122.243:1,192.168.122.245:1
        kubernetes          192.168.122.251:8443


        [root@ose1 nginx_gluster_pvc_template]# oc get pv
        NAME                      LABELS    CAPACITY     ACCESSMODES   STATUS    CLAIM                    REASON
        gluster-default-volume    <none>    2147483648   RWX           Bound     default/claim-default    
        gluster-default-volume2   <none>    2147483648   RWX           Bound     default/claim-default2   


        [root@ose1 nginx_gluster_pvc_template]# oc get pvc
        NAME             LABELS    STATUS    VOLUME
        claim-default    map[]     Bound     gluster-default-volume
        claim-default2   map[]     Bound     gluster-default-volume2







[root@ose1 nginx_gluster_pvc_template]# oc create -f gluster-nginx-template.json 
templates/glusterfs-persistent-nginx-template
















- `(OPTIONAL – only if they do not already exist)` For this example, you will have to define the GlusterFS Cluster as “EndPoints” within Kubernets/OpenShift platform.  See [endpoints configuration file](gluster-endpoints.json) for an example.  For this file, it's important to note that you must define the IP/Hostname of your gluster servers and the port that you want to use.  The port value can be any numeric value within the accepted range of ports.



        oc create -f gluster-endpoints.json

        [root@ose1 nginx_gluster]# oc create -f gluster-endpoints.json 
        endpoints/glusterfs-cluster 


        oc get endpoints

        [root@ose1 nginx_gluster]# oc get endpoints 
        NAME                ENDPOINTS 
        glusterfs-cluster   192.168.122.221:1,192.168.122.222:1 
        kubernetes          192.168.122.251:8443 



- Create the [PersistentVolume configuration file](gluster-nginx-pv.json) that will register your gluster volume with OpenShift and make it available to be used for claims and storage across all the pods/containers in your current project.

        { 
           "apiVersion": "v1", 
           "kind": "PersistentVolume", 
           "metadata": { 
               "name": "gluster-default-volume" 
           }, 
           "spec": { 
                "capacity": { 
                     "storage": "2Gi" 
                     }, 
                "accessModes": [ "ReadWriteMany" ], 
                "glusterfs": { 
                    "endpoints": "glusterfs-cluster", 
                    "path": "myVol1", 
                    "readOnly": false, 
                }, 
                "persistentVolumeReclaimPolicy": "Recycle" 
            } 
        }



        oc create -f gluster-nginx-pv.json

        [root@ose1 nginx_gluster_pvc]# oc create -f gluster-nginx-pv.json 
        persistentvolumes/gluster-default-volume 



        oc get pv

        [root@ose1 nginx_gluster_pvc]# oc get pv 
        NAME                     LABELS    CAPACITY     ACCESSMODES   STATUS      CLAIM     REASON 
        gluster-default-volume   <none>    2147483648   RWX           Available  


*Notice, we now have a Persistent Volume – it shows `Available` but is not yet bound.*



        oc describe pv gluster-default-volume    (name from get pv)

        [root@ose1 nginx_gluster_pvc]# oc describe pv gluster-default-volume 
        Name:		gluster-default-volume 
        Labels:		<none> 
        Status:		Available 
        Claim:		 
        Reclaim Policy:	%!d(api.PersistentVolumeReclaimPolicy=Recycle) 
        Message:	%!d(string=) 



- Create the [PersistentVolumeClaim configuration file](gluster-nginx-pvc.json).

        { 
            "apiVersion": "v1", 
            "kind": "PersistentVolumeClaim", 
            "metadata": { 
                "name": "claim-default" 
            }, 
            "spec": { 
                 "accessModes": [ "ReadWriteMany" ], 
                 "resources": { 
                      "requests": { 
                          "storage": "2Gi" 
                      } 
                 } 
            } 
         }



        oc create -f gluster-nginx-pvc.json

        [root@ose1 nginx_gluster_pvc]# oc create -f gluster-nginx-pvc.json 
        persistentvolumeclaims/claim-default 



        oc get pv 

        [root@ose1 nginx_gluster_pvc2]# oc get pv 
        NAME                      LABELS    CAPACITY     ACCESSMODES   STATUS      CLAIM                   REASON 
        gluster-default-volume    <none>    2147483648   RWX           Bound       default/claim-default   


*Notice now, that the PV Status becomes bound.*



         oc get pvc

         [root@ose1 nginx_gluster_pvc]# oc get pvc 
         NAME            LABELS    STATUS    VOLUME 
         claim-default   map[]     Bound     gluster-default-volume 



         oc describe pvc claim-default    (name from get pv)

         [root@ose1 nginx_gluster_pvc]# oc get pvc 
         NAME            LABELS    STATUS    VOLUME 
         claim-default   map[]     Bound     gluster-default-volume 

         [root@ose1 nginx_gluster_pvc]# oc describe pvc claim-default 
         Name:		claim-default 
         Namespace:	default 
         Status:		%!d(api.PersistentVolumeClaimPhase=Bound) 
         Volume:		%!d(string=gluster-default-volume) 




- Create the [pod configuration file](gluster-nginx-pvc-pod.json) with the reference to the PVC embedded in the configuration to utilize the “glusterfs” plugin


        { 
            "apiVersion": "v1", 
            "id": "glusterfs-nginx-pvc", 
            "kind": "Pod", 
            "metadata": { 
                "name": "glusterfs-nginx-pvc" 
            }, 
            "spec": { 
                "containers": [ 
                    { 
                        "name": "glusterfs-nginx-pvc", 
                        "image": "fedora/nginx", 
                        "volumeMounts": [ 
                            { 
                                 "mountPath": "/usr/share/nginx/html/test", 
                                 "name": "nginxglustervolpvc" 
                            } 
                        ], 
                        "securityContext": { 
                            "capabilities": {}, 
                            "privileged": true 
                        } 
                   } 
               ], 
               "volumes": [ 
                   { 
                       "name": "nginxglustervolpvc", 
                       "persistentVolumeClaim": { 
                           "claimName": claim-default 
                       } 
                   } 
               ] 
            } 
        } 




        oc create -f gluster-nginx-pvc-pod.json

        [root@ose1 nginx_gluster_pvc]# oc create -f gluster-nginx-pvc-pod.json 
        pods/glusterfs-nginx-pvc 



- After a few minutes (this may vary), check and make sure the pod is running

        oc get pods

        [root@ose1 nginx_gluster_pvc]# oc get pods 
        NAME                     READY     STATUS    RESTARTS   AGE 
        glusterfs-nginx          1/1       Running   0          1h 
        glusterfs-nginx-pvc      1/1       Running   0          14s 
        glusterfs-nginx1         1/1       Running   0          18m 
        local-nginx              1/1       Running   0          20h 
        local-nginx-to-gluster   1/1       Running   0          17h 



- You should now also see it in your OSE web GUI  (https://host:8443/console)  (if AllowAll was enabled, should just be able to login with any id and password)


![OSE nginx](../images/example3_ose.png)


- From the OSE Console, Note the “IP on node” and “Node” values, which will tell you what ip and node the nginx application is running on.   We will use this in a later step to curl the web server



- Create a sample html page to serve from the Gluster Storage Server.  SSH to one of the gluster hosts and create your page.  For this example I did the following:

        ssh root@gluster1.rhs   (gluster server)


   Navigate to the mounted directory, in this case for my example


        cd /mnt/glustervol1      (mapped to gluster volume --> myVol1)
        mkdir glusterfs-nginx-pvc
   
     
   create a helloworld.html file

                                
        This is using GlusterFS Distributed Storage to serve this page.  Created by OSEv3 Console using PersistentVolume and PersistentVolumeClaim.  OSE + GLUSTERFS!!! 



- SSH into the node where the container is running (remember the details from the OSE Console Web GUI) and using the container_id obtained from “docker ps” command, jump into the container shell and then we will explore what we have.

      From the Atomic Host Where the Container is running:


        docker ps

        [root@ose2 data2]# docker ps 
        CONTAINER ID        IMAGE                         COMMAND             CREATED             STATUS              PORTS               NAMES 
        eefb2a76f6c5        fedora/nginx                  "/usr/sbin/nginx"   6 minutes ago       Up 6 minutes                            k8s_glusterfs-nginx-pvc.ccb56be2_glusterfs-nginx-pvc_default_4e49d7b6-4679-11e5               



        docker exec -it 8dda35c5f9ad bash

        [root@ose2 data2]# docker exec -it eefb2a76f6c5 bash 
        bash-4.3# ls 
        bin  boot  dev	etc  home  lib	lib64  lost+found  media  mnt  opt  proc  root	run  sbin  srv	sys  tmp  usr  var 
        bash-4.3# cd /usr/share/nginx/html/test 
        bash-4.3# ls 
        glusterfs-nginx  glusterfs-nginx-atomichost-mount  glusterfs-nginx-pvc	helloworld.html  test1	test2  test3 


     From the Container:

        mount   (will show the mounted gluster volume)

        192.168.122.221:myVol1 on /usr/share/nginx/html/test type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) 

              
*Notice when we go to our mounted nginx root test directory, we can now see all the contents of our GlusterFS Storage mount, including the directory we created “glusterfs-nginx”*

*Notice we also see our helloworld.html file that we created within that directory*


- Enter simple curl command from the docker container to serve the page

        curl http://10.1.0.12/test/glusterfs-nginx-pvc/helloworld.html

        [root@ose2 data2]# docker exec -it eefb2a76f6c5 bash 
        bash-4.3# curl http://10.1.0.12/test/glusterfs-nginx-pvchelloworld.html 
        This is using GlusterFS Distributed Storage to serve this page.  Created by OSEv3 Console using PersistentVolume and PersistentVolumeClaim.  OSE + GLUSTERFS!!! 



## Review: 

At this point, we have successfully completed 4 examples, with multiple ways to utilize distributed storage from OpenShift and Docker/Kubernetes. From your Atomic Host where these pods/containers are running you can issue the `mount` command and you should see all your pods/containers and their current mounts.  Below we have 3 active mounts that show up on the host, one from each pod that we were running.  The last one is from this example.  Cool!!!!



        gluster1.rhs:/myVol1 on /mnt/gvol1 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) 
        gluster1.rhs:/myVol2 on /mnt/gvol2 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) 
        tmpfs on /var/lib/openshift/openshift.local.volumes/pods/d7747b33-45ce-11e5-ae70-52540008f001/volumes/kubernetes.io~secret/default-token-b7rvs type tmpfs (rw,relatime) 
        tmpfs on /var/lib/openshift/openshift.local.volumes/pods/ca995463-45e8-11e5-ae70-52540008f001/volumes/kubernetes.io~secret/default-token-b7rvs type tmpfs (rw,relatime) 
        192.168.122.221:myVol1 on /var/lib/openshift/openshift.local.volumes/pods/c978d0b4-466b-11e5-ae70-52540008f001/volumes/kubernetes.io~glusterfs/nginxglustervol type fuse.glusterfs             (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) 
        tmpfs on /var/lib/openshift/openshift.local.volumes/pods/c978d0b4-466b-11e5-ae70-52540008f001/volumes/kubernetes.io~secret/default-token-b7rvs type tmpfs (rw,relatime) 
        192.168.122.221:myVol1 on /var/lib/openshift/openshift.local.volumes/pods/b445f4cd-4676-11e5-ae70-52540008f001/volumes/kubernetes.io~glusterfs/nginxglustervol1 type fuse.glusterfs  (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) 
        tmpfs on /var/lib/openshift/openshift.local.volumes/pods/b445f4cd-4676-11e5-ae70-52540008f001/volumes/kubernetes.io~secret/default-token-b7rvs type tmpfs (rw,relatime) 
        192.168.122.221:myVol1 on /var/lib/openshift/openshift.local.volumes/pods/4e49d7b6-4679-11e5-ae70-52540008f001/volumes/kubernetes.io~glusterfs/gluster-default-volume type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) 



===

[Previous](../nginx_gluster_plugin)  |  [Main List](../)  |  [Next](TBD)

===



