Cluster Operator examples

Before deploying a StorageOS cluster, create a Secret to define the StorageOS API Username and Password in base64 encoding.

kubectl create -f - <<END
apiVersion: v1
kind: Secret
metadata:
  name: "storageos-api"
  namespace: "default"
  labels:
    app: "storageos"
type: "kubernetes.io/storageos"
data:
  apiUsername: c3RvcmFnZW9z
  apiPassword: c3RvcmFnZW9z

  # CSI Credentials
  csiProvisionUsername: c3RvcmFnZW9z
  csiProvisionPassword: c3RvcmFnZW9z
  csiControllerPublishUsername: c3RvcmFnZW9z
  csiControllerPublishPassword: c3RvcmFnZW9z
  csiNodePublishUsername: c3RvcmFnZW9z
  csiNodePublishPassword: c3RvcmFnZW9z
  csiControllerExpandUsername: c3RvcmFnZW9z
  csiControllerExpandPassword: c3RvcmFnZW9z
END

This example contains a default password, for production installations, use a unique, strong password.

Make sure that the encoding of the credentials doesn’t have special characters such as ‘\n’.

You can define a base64 value by echo -n "mystring" | base64.

Create a cluster-config.yaml according to your needs from the examples below.

kubectl create -f cluster-config.yaml

Note that StorageOS will be deployed in spec.namespace (storageos by default), irrespective of what NameSpace the CR is defined in.

 

Examples

You can checkout all the parameters configurable in the configuration page.

All examples must reference the storageos-api Secret.

spec:
  secretRefName: "storageos-api" # Reference to the Secret created in the previous step
  secretRefNamespace: "storageos-operator"  # Namespace of the Secret

Check out Cluster Definition examples for full CR files.

Installing with an external etcd

spec:
  kvBackend:
    address: '10.43.93.95:2379' # IP of the SVC that exposes ETCD
  # address: '10.42.15.23:2379,10.42.12.22:2379,10.42.13.16:2379' # You can specify individual IPs of the etcd servers

If using Etcd with mTLS, you need to specify the secret that hold the certificates with the following parameters:

spec:
  # External mTLS secured etcd cluster specific properties
  tlsEtcdSecretRefName: "etcd-client-tls"                          # Secret containing etcd client certificates
  tlsEtcdSecretRefNamespace: "storageos-etcd"                      # Namespace of the client certificates secret

Follow the etcd operations page to setup the secret with the Etcd client certificate, client key and CA.

Installing to a subset of nodes

In this case we select nodes that are workers. To make sure that StorageOS doesn’t start in Master nodes.

You can see the labels in the nodes by kubectl get node --show-labels.

spec:
  nodeSelectorTerms:
    - matchExpressions:
      - key: "node-role.kubernetes.io/worker"
        operator: In
        values:
        - "true"

# OpenShift uses "node-role.kubernetes.io/compute=true"
# Rancher uses "node-role.kubernetes.io/worker=true"
# Kops uses "node-role.kubernetes.io/node="

Different provisioners and Kubernetes distributions use node labels differently to specify master vs workers. Node Taints are not enough to make sure StorageOS doesn’t start in a node. The JOIN variable is defined by the operator by selecting all the nodes that match the nodeSelectorTerms.

Enabling CSI

spec:
  csi:
    enable: true
    deploymentStrategy: deployment
    enableProvisionCreds: true
    enableControllerPublishCreds: true
    enableNodePublishCreds: true
    enableControllerExpandCreds: true

The credentials must be defined in the storageos-api Secret

apiVersion: v1
kind: Secret
metadata:
  name: "storageos-api"
  namespace: "storageos-operator"
  labels:
    app: "storageos"
type: "kubernetes.io/storageos"
data:
  # echo -n '<secret>' | base64
  apiUsername: c3RvcmFnZW9z
  apiPassword: c3RvcmFnZW9z

   # Add base64 encoded creds below for CSI credentials.
  csiProvisionUsername: c3RvcmFnZW9z
  csiProvisionPassword: c3RvcmFnZW9z
  csiControllerPublishUsername: c3RvcmFnZW9z
  csiControllerPublishPassword: c3RvcmFnZW9z
  csiNodePublishUsername: c3RvcmFnZW9z
  csiNodePublishPassword: c3RvcmFnZW9z
  csiControllerExpandUsername: c3RvcmFnZW9z
  csiControllerExpandPassword: c3RvcmFnZW9z

Specifying a shared directory for use with kubelet as a container

spec:
  sharedDir: '/var/lib/kubelet/plugins/kubernetes.io~storageos'

Defining pod resource requests and reservations

spec:
  resources:
    requests:
      memory: "512Mi"
  #   cpu: "1"
  # limits:
  #   memory: "4Gi"

Limiting StorageOS can cause malfunction for IO to StorageOS volumes, therefore we do not currently recommend applying upper limits to resources for StorageOS pods.

Specifying custom Tolerations

spec:
  tolerations:
  - key: "key1"
    operator: "Equal"
    value: "value1"
    effect: "EffectToTolerate"
  - key: "key2"
    operator: "Exists"

Custom tolerations specified in the StorageOSCluster definition are added to all StorageOS components; the StorageOS daemonset, CSI helper and scheduler.

In the above example a toleration key1=value1:EffectToTolerate would be tolerated and key2 would be tolerated regardless of the value and effect. For more information about tolerations please see the Kubernetes documentation.