Juju K8S models Ceph backed storage


#1

Hi folks

I saw the example for AWS/EBS backed juju operational storage. Does the same work for my K8S cluster that has a Ceph backed storage pool?


#2

As a guess I tried this and failed:

ubuntu@juju:~$ juju create-storage-pool operator-storage kubernetes storage-class=juju-operator-storage storage-provisioner=csi-rbdplugin
ERROR facade "Storage" not supported for a CAAS model API connection (not supported)

#3

Ah, did a controller upgrade and its started doing something…


#4

Yeah, what you had should work. Since this stuff is in active development, we recommend using the latest 2.5 (edge) snap. There’s a bunch of additional stuff going into the 2.6 snap which we will soon recommend as the version to use if you want the latest and greatest.


#5

Yeah with the upgrade and stuff it allowed me to add it, it didn’t actually create any storage though so i went back to using the no provisioner. I’ll check again tomorrow.


#6

Just wanted to let you know that I wrote a tutorial for how to get k8s charms working on the charmed k8s cluster.

When you don’t configure any storage pool in Juju, it will use the default storage class in Kubernetes. As a result, if you deploy CDK and Ceph, and add a relation between them, then Juju will automatically use the ceph cluster as default storage (since that relation configures ceph-zfs as default).


#7

Awesome thanks!

Just to expand a bit on storage - Juju will use the default storage class so long as the provisioner matches Juju opinionated default for the underlying cloud. So on AWS that would be EBS volumes, on GCE that’s Google PD etc. If the cluster default storage class uses a different provisioner, you need to use the
--storage option when you register the cluster using add-k8s.

You can also still use Juju storage pools is you need to further customise how storage is provisioned.


#8

@dt6gy6 could you share a small snippet on how we hooked up the rbd provisioner in our models? Thanks!


#9

Sorry if this is a silly question, what is ceph-zfs, and where can I find some information on it?


#10

When you add the relation between CDK and Ceph, it creates two ceph-backed storage classes in Kubernetes; one of them is ceph-zfs; this one is labeled as the default in Kubernetes, so Juju uses it by default if you don’t configure something else.

I haven’t looked into it further. The Juju magic works so I don’t question it :wink:


#12

I’m pretty sure we were just able to create the rbd-provisioner and a named storage class, then when we create a storage pool in a k8s model we reference the storage class that uses the rbd provisioner. Nice to hear thats its baked in though!