Install with Kubernetes

StorageOS can be used as a storage provider for your Kubernetes cluster, making local storage accessible from any node within the Kubernetes cluster. Data can be replicated to protect against node failure.

At its core, StorageOS provides block storage. You may choose the filesystem type to install to make devices usable from within containers.

Try it in your browser for up to one hour »

Prerequisites

You will need a Kubernetes 1.8+ cluster with Beta APIs enabled. The following prerequisites are met by default for Kubernetes 1.10+.

  1. Install StorageOS CLI.
  2. Make sure your docker installation has mount propagation enabled.
    # A successful run is proof of mount propagation enabled
    docker run -it --rm -v /mnt:/mnt:shared busybox sh -c /bin/date
    
  3. Enable MountPropagation in Kubernetes (the procedure to enable mount propagation flags depends on the cluster’s boot procedure):
    • Append flag --feature-gates MountPropagation=true to the deployments kube-apiserver and kube-controller-manager, usually found under /etc/kubernetes/manifests in the master node.
    • Add flag in the kubelet service config KUBELET_EXTRA_ARGS=--feature-gates=MountPropagation=true. For systemd, this usually is located in /etc/systemd/system/.
  4. For deployments where the kubelet runs in a container (eg. OpenShift, CoreOS, Rancher), add --volume=/var/lib/storageos:/var/lib/storageos:rshared to each of the kubelets. This will be fixed in Kubernetes 1.10+.

For Kubernetes 1.7, you can install the container directly in Docker.

Installation

You can install StorageOS with Helm or creating Kubernetes spec files.

Install with Helm

Firstly, install Helm. If you are running a rbac enabled cluster, you will need to create a service account.

The StorageOS helm chart triggers an init container to enable required kernel configuration on your host. If that procedure doesn’t finish successfully, StorageOS container will not start.

If that is the case, check out the os support page as you can manually prepare the settings for the kernel.

$ git clone https://github.com/storageos/helm-chart.git storageos
$ cd storageos
$ helm install --name my-release .

# Follow the instructions printed by helm install to update the link between Kubernetes and StorageOS.
$ ClusterIP=$(kubectl get svc/storageos --namespace kube-system -o custom-columns=IP:spec.clusterIP --no-headers=true)
$ ApiAddress=$(echo -n "tcp://$ClusterIP:5705" | base64)
$ kubectl patch secret/storageos-api --namespace kube-system --patch "{\"data\":{\"apiAddress\": \"$ApiAddress\"}}"

To uninstall the release with all related Kubernetes components:

$ helm delete --purge my-release

See further configuration options.

Install without Helm

  1. Create service account

     kubectl create -f - <<END
     apiVersion: v1
     kind: ServiceAccount
     metadata:
       name: storageos
       labels:
         app: storageos
     END
    
  2. Create role

     kubectl create -f - <<END
     apiVersion: rbac.authorization.k8s.io/v1
     kind: Role
     metadata:
       name: storageos
       labels:
         app: storageos
     rules:
     - apiGroups:
         - ""
       resources:
         - secrets
       verbs:
         - create
         - get
         - list
         - delete
     END
    
  3. Create role binding

     kubectl create -f - <<END
     apiVersion: rbac.authorization.k8s.io/v1
     kind: RoleBinding
     metadata:
       name: storageos
       labels:
         app: storageos
     roleRef:
       apiGroup: rbac.authorization.k8s.io
       kind: Role
       name: storageos
     subjects:
       - kind: ServiceAccount
         name: storageos
     END
    
  4. Create StorageOS service

     kubectl create -f - <<END
     apiVersion: v1
     kind: Service
     metadata:
       name: storageos
       namespace: default
       labels:
         app: storageos
     spec:
       type: ClusterIP
       ports:
         - port: 5705
           targetPort: 5705
           protocol: TCP
           name: storageos
       selector:
         app: storageos
     END
    
  5. Create secret

     CLUSTER_IP=$(kubectl get svc/storageos -o custom-columns=IP:spec.clusterIP --no-headers=true)
     API_ADDRESS=$(echo -n "tcp://$CLUSTER_IP:5705" | base64)
     kubectl create -f - <<END
     apiVersion: v1
     kind: Secret
     metadata:
       name: storageos-api
       namespace: default
       labels:
         app: storageos
     type: "kubernetes.io/storageos"
     data:
       # Address build from the ClusterIP of the storageos service. In the format $(echo -n "tcp://$ClusterIP:5705" | base64)
       apiAddress: "$API_ADDRESS"
       apiUsername: "c3RvcmFnZW9z"
       apiPassword: "c3RvcmFnZW9z"
     END
    

    The secret hosts information regarding the API endpoint of StorageOS. It is required to add the StorageOS service IP in base64 for the apiAddress field.

    apiUsername and apiPassword also need to be specified in base64. This example uses echo "storageos" | base64 for both fields.

  6. Create storage class

     kubectl create -f - <<END
     apiVersion: storage.k8s.io/v1beta1
     kind: StorageClass
     metadata:
       name: fast
       labels:
         app: storageos
     provisioner: kubernetes.io/storageos
     parameters:
       pool: default
       description: Kubernetes volume
       fsType: ext4
       adminSecretNamespace: default
       adminSecretName: storageos-api
     END
    

    StorageOS uses a secret to discover the API endpoint of its workers. adminSecretNamespace must be set to the specified namespace, and the service account needs to have granted permissions to StorageOS pods to have access to that secret.

  7. Create DaemonSet

    StorageOS server pod runs as a daemonset, therefore all nodes in your cluster will have a StorageOS worker. If you want to limit the amount of nodes in the StorageOS cluster or enforce which Kubernetes nodes run storage workloads, you can define spec.nodeSelector based on node tags. For more information check here.

     # Get a cluster token id
     JOIN=$(storageos cluster create)
    
     kubectl create -f - <<END
     kind: DaemonSet
     apiVersion: apps/v1
     metadata:
       name: storageos
       namespace: default
     spec:
       selector:
         matchLabels:
           app: storageos
       template:
         metadata:
           name: storageos
           labels:
             app: storageos
         spec:
           hostPID: true
           hostNetwork: true
           serviceAccountName: storageos
           initContainers:
           - name: enable-lio
             image: storageos/init:0.1
             imagePullPolicy: Always
             volumeMounts:
               - name: kernel-modules
                 mountPath: /lib/modules
                 readOnly: true
               - name: sys
                 mountPath: /sys
                 mountPropagation: Bidirectional
             securityContext:
               privileged: true
               capabilities:
                 add:
                 - SYS_ADMIN
           containers:
           - name: storageos
             image: "storageos/node:1.0.0-rc1"
             imagePullPolicy: IfNotPresent
             args:
             - server
             ports:
             - containerPort: 5705
               name: api
             livenessProbe:
               initialDelaySeconds: 65
               timeoutSeconds: 10
               failureThreshold: 5
               httpGet:
                 path: /v1/health
                 port: api
             readinessProbe:
               initialDelaySeconds: 65
               timeoutSeconds: 10
               failureThreshold: 5
               httpGet:
                 path: /v1/health
                 port: api
             resources:
                 {}
             env:
             - name: HOSTNAME
               valueFrom:
                 fieldRef:
                   fieldPath: spec.nodeName
             - name: JOIN
               value: "$JOIN"
             - name: ADVERTISE_IP
               valueFrom:
                 fieldRef:
                   fieldPath: status.podIP
             - name: NAMESPACE
               value: default
             securityContext:
               privileged: true
               capabilities:
                 add:
                 - SYS_ADMIN
             volumeMounts:
               - name: fuse
                 mountPath: /dev/fuse
               - name: state
                 mountPath: /var/lib/storageos
                 mountPropagation: Bidirectional
               - name: sys
                 mountPath: /sys
           volumes:
           - name: kernel-modules
             hostPath:
               path: /lib/modules
           - name: fuse
             hostPath:
               path: /dev/fuse
           - name: sys
             hostPath:
               path: /sys
           - name: state
             hostPath:
               path: /var/lib/storageos
     END