InfluxDB with StorageOS
InfluxDB is a popular open source time series database application optimised for managing datasets consisting of many small measurements. Its advantages include the ability to handle very high write and query loads. Its uses include monitoring, analytics and the recording and analysis of data from sensors.
Before you start, ensure you have StorageOS installed and ready on a Kubernetes cluster. See our guide on how to install StorageOS on Kubernetes for more information.
Deploying InfluxDB on Kubernetes
You can find the latest files in the StorageOS use cases repository
git clone https://github.com/storageos/use-cases.git storageos-usecases cd storageos-usecases
apiVersion: apps/v1 kind: StatefulSet metadata: name: influxdb spec: replicas: 1 selector: matchLabels: app: influxdb serviceName: influxdb ... spec: serviceAccountName: influxdb ... volumeMounts: - mountPath: /var/lib/influxdb name: data ... volumeClaimTemplates: - metadata: name: data spec: accessModes: ["ReadWriteOnce"] storageClassName: "fast" # StorageOS storageClass resources: requests: storage: 20Gi
This excerpt is from the StatefulSet definition. This file contains the VolumeClaimTemplate that will dynamically provision storage, using the StorageOS storage class. Dynamic provisioning occurs as a volumeMount has been declared with the same name as a VolumeClaim.
Create the InfluxDB objects
kubectl create -f ./influxdb
Confirm InfluxDB is up and running.
$ kubectl get pods NAME READY STATUS RESTARTS AGE influxdb-client 1/1 Running 0 1m influxdb-0 1/1 Running 0 1m
Connect to the InfluxDB client pod, then to the InfluxDB server through the service (this reflects the common kubernetes pattern of maintaining a client pod to conveniently inspect a resource interactively). The default user (admin) and password (admin) are defined in the StatefulSet.
$ kubectl exec -it influxdb-client -- bash [email protected]:/# influx -host influxdb-0.influxdb Connected to http://influxdb-0.influxdb:8086 version 1.8.2 InfluxDB shell version: 1.8.2 > auth username: admin password: > show databases name: databases name _internal > CREATE DATABASE weather; > USE weather Using database weather > INSERT temperature,location=London value=26.4 > INSERT temperature,location=London value=24.9 > INSERT temperature,location=London value=22.2 > INSERT temperature,location=London value=14.7 > INSERT temperature,location=London value=19.5 > INSERT temperature,location=Paris value=27.1 > INSERT temperature,location=Paris value=27.5 > INSERT temperature,location=Paris value=21.3 > INSERT temperature,location=Paris value=26.7 > INSERT temperature,location=Paris value=30.0 > SELECT MEAN(*) FROM "temperature" GROUP BY "location" name: temperature tags: location=London time mean_value 0 25.65 name: temperature tags: location=Paris time mean_value 0 26.90
In the above steps we have inserted some time series data on the temperature at two locations, and calculated the mean for both. InfluxDB offers a variety of such aggregations (see the docs here), allowing convenient analysis of time series data.
If you need custom startup options, you can edit or add to the environment
variables within the
In this example of how to perform backups of an InfluxDB database on a
Kubernetes cluster, we write the output backup file
to an Amazon Web Services (AWS) S3 bucket. Other approaches, such as backing
up to internal servers or other StorageOS volumes, are possible. For this
example to run successfully, Base64-encoded AWS credentials and an S3 bucket
name should be inserted into the data field of the
$ echo -n '<your-aws-access-key-id>' | base64 XXXXXXXXXXXX $ echo -n '<your-aws-secret-access-key>' | base64 XXXXXXXXXXXX $ echo -n '<your-aws-default-region>' | base64 XXXXXXXXXXXX $ echo -n '<your-S3-bucket-name>' | base64 XXXXXXXXXXXX
apiVersion: v1 kind: Secret metadata: name: backup-pod-environment type: Opaque data: AWS_ACCESS_KEY_ID: XXXXXXXXXXXX AWS_SECRET_ACCESS_KEY: XXXXXXXXXXXX AWS_DEFAULT_REGION: XXXXXXXXXXXX BUCKET_NAME: XXXXXXXXXXXX DB_NAME: d2VhdGhlcg== DB_HOST: aW5mbHV4ZGItMC5pbmZsdXhkYjo4MDg4
To perform the backup, create the secret and job from the manifest files
kubectl create -f ./influxdb/backup/
Confirm that the backup pod has been created, and the backup performed successfully.
NAME READY STATUS RESTARTS AGE backup-ks976 0/1 Completed 0 1m client 1/1 Running 0 10m influxdb-0 1/1 Running 0 10m mysql-0 1/1 Running 0 10m
The files generated by the backup operation should now be available in you S3 bucket. Re-perform the operation at any time by deleting and re-creating the backup job.
$ kubectl delete -f "influxdb/40-backup-job.yaml" job.batch "backup" deleted $ kubectl create -f "influxdb/40-backup-job.yaml" job.batch/backup created