I was using the Prometheus Helm chart located at stable/prometheus and I noted that my Persistent Volume Claim (PVC) was stuck in pending. Usually this is due to a few things – your storage integration not working or a misconfiguration in the claim itself. For me it was the latter.
[email protected]:~$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE prometheus-alertmanager Pending 6h prometheus-server Pending 6h
Ugh. Pending. When you do have an issue with your storage integration for persistent volumes usually you have a storage class assigned. I note that the helm chart that I do does not define a storage class.
NAME PROVISIONER AGE fast kubernetes.io/vsphere-volume 2d thin-disk kubernetes.io/vsphere-volume 1d [email protected]:~$ kubectl describe sc thin-disk Name: thin-disk IsDefaultClass: No Provisioner: kubernetes.io/vsphere-volume Parameters: diskformat=thin AllowVolumeExpansion:MountOptions: ReclaimPolicy: Delete VolumeBindingMode: Immediate Events:
I can add an annotation to define a storage class as default.
kubectl patch storageclass thin-disk -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}
This here will append the annotation for the storage class to become default. I alternatively could edit the object.
[email protected]:~$ kubetl edit sc thin-disk # apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: annotations: storageclass.kubernetes.io/is-default-class: "true" creationTimestamp: 2018-10-09T23:30:50Z name: thin-disk resourceVersion: "389472" selfLink: /apis/storage.k8s.io/v1/storageclasses/thin-disk uid: 5b9ae080-cc1b-11e8-bfe5-005056a3e23d parameters: diskformat: thin provisioner: kubernetes.io/vsphere-volume reclaimPolicy: Delete volumeBindingMode: Immediate
Now if I look at the object and describe it I note my annotation is there
[email protected]:~$ kubectl describe sc thin-disk Name: thin-disk IsDefaultClass: Yes Annotations: storageclass.kubernetes.io/is-default-class=true Provisioner: kubernetes.io/vsphere-volume Parameters: diskformat=thin AllowVolumeExpansion:MountOptions: ReclaimPolicy: Delete VolumeBindingMode: Immediate Events:
Now when I apply my Helm Chart for Prometheus I get the following against my storage claim.
[email protected]:~$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE prometheus-alertmanager Bound pvc-68bdefa9-cd8d-11e8-bfe5-005056a3e23d 2Gi RWO thin-disk 17m prometheus-server Bound pvc-68c07b96-cd8d-11e8-bfe5-005056a3e23d 8Gi RWO thin-disk 17m
Note: I can modify or annotate option on the Helm chart to provide this SC Value. In this case here I’ve opted for a default to take place.
Now my pods are happily scheduled.