Writing a Kubernetes charm

In metadata.yaml

storage:
  database:
    type: filesystem
    location: /var/lib/mysql

What are the else storage types besides filesystem?

Juju also supports block storage devices for IAAS models but not k8s models yet. Block device support in Kubernetes itself was only relatively recently added. It will be supported at some point but for now it’s just filesystem storage.

emptyDir in-mem support is not implemented yet but we can look at it for this cycle.
It would be done using standard Juju storage primitives. Juju does have a tmpfs storage type which for k8s models would be mapped to emptyDir.medium=“Memory”

juju deploy mycharm --storage mydata=100M,tmpfs

$ juju deploy cs:~juju/mariadb-k8s abc --storage database=10M,tmpfs ;
ERROR pool “tmpfs” not found

Should some special values be defined in the metadata?

It’s not implemented yet. But it’s something we can try and look at this cycle.

@wallyworld I followed the steps outlined above and thought it all came together pretty smooth. Hit a few bumps along the way (more just my lack of knowledge in k8s) a little googling and I got things working. Documented most of the process here.

Might you have an idea on how this workflow extends itself to getting ingress to the applications deployed in k8s?

Glad it worked, and thanks for documenting your experience. You are right that when things go wrong, you do need some k8s knowledge to debug.

With the jenkins permission issue, I’m hoping that one of the guys with more practical k8s experience than I have can chime in. If there’s something we need to allow for in the k8s charm setup we can add it in. @kwmonroe @cory_fu @knobby any insight?

With regards to ingress to an application, juju expose works - see Exposing Gitlab.
But the exact mechanics of it depend on the underlying cluster setup. In my experience I have found that deploying CDK on AWS with the integrator charm results in a Load Balanced service with a public IP being created out of the box and so the application is exposed automatically. But it appears that’s not the case for you so the juju expose step is necessary.

Also, again depending on the underling k8s setup, you can use advanced app config to configure the deployed application with various k8s specific settings that relate to ingress. eg external ips, load balancer ip etc. Ideally we’ll get to documenting a deployment recipe for various scenarios but for now there’s a bit of k8s operations knowledge needed to get things going in this type of set up.

1 Like

Exactly what I was missing. Thanks @wallyworld

If I need to import a python package in my reactive file, how can I install this with k8s charms? I used to create a wheelhouse.txt file in the charm and it would get pulled into the wheelhouse after charm build but that doesn’t seem to work here.

@sborny,

The wheelhouse.txt file is still used, but due to technical differences in the way k8s charms work vs machine charms, the dependencies are installed into the image at build time rather than being put into a wheelhouse/ directory as source wheels or eggs. You should be able to see your packages in the built charm under the lib/ directory, which is automatically added to the PYTHONPATH by layer:caas-base so it should function seamlessly in the way you would expect from machine charm development.

If you’re having import issues in your charm, please start a new thread (if you don’t mind) so we can dive into debugging it there. Thanks!

1 Like

@cory_fu
Thanks for the write-up. The wheelhouse.txt file does indeed still work. The python lib that I needed didn’t install via the wheelhouse.txt method and needs a pip_install from the charmhelpers library.

I see the various bits of K8S stuff you can overload. One obvious thing that I can’t seem to find in documentation or charm, is, can you overload the command or do you have to use the container provided one?

k8s charms do support configuring the command to run when starting the container plus args, working dir etc:

containers:
  - name: mycharm
    imageDetails:
      imagePath: mycharm
      username: fred
      password: secret
    imagePullPolicy: Always
    command: ["sh", "-c"]
    args: ["doIt", "--debug"]
    workingDir: "/path/to/here"
    ports:
    - containerPort: 80
      protocol: TCP
1 Like

Cool thanks! I’ll give it a test.

Hi @wallyworld this section doesn’t fully describe what’s different. Does the example relate to IaaS or CaaS charms?

Hi, I’m trying to set a persistent volume in my charm for a k8s app. I’m having trouble setting the capacity of the volume requested. In my metadata.yml, I’ve put:

storage: 
    database:
        type: filesystem
        location: /var/opt/mydatabaseapp
        capacity: 8Gi

However the persistent volume is created with 1Gi only. Do you have an example of the right syntax to use?

Edit : found it :slight_smile: it needs to be minimum-size: 8G and not capacity!

1 Like

There is no “capacity” attribute for storage, but there is a “minimum-size”

storage: 
    database:
        type: filesystem
        location: /var/opt/mydatabaseapp
        minimum-size: 8Gi

At deploy time, the user can then (or is expected to in production scenarios) specify the actual storage configuration to be used, eg

$ juju deploy foo --storage database=100G

The default storage provisioned on k8s is a PV but you can also choose to use k8s emptyDir backed by either memory or disk for prototyping or testing etc, eg

$ juju deploy foo --storage database=tmpfs,1G

or

$ juju deploy foo --storage database=rootfs,1G

The above tmpfs and rootfs storage types map to generic Juju storage concepts applicable to any type of deployment, k8s or cloud.

See https://discourse.jujucharms.com/t/persistent-storage-and-kubernetes/1078 for more info.

Admitting some ignorance here, but one thing that has got me stuck a few times is figuring out how to tell Juju about which Docker image I want attached as a resource to local charms during development.

If I run docker pull redis, what commands do I need to execute to convert that image into a resource accepted by the deploy command?

Is it necessary to use juju attach-storage for local charms, e.g.:

  • docker pull redis:5
  • juju deploy /path/to/local/redis-charm-k8s
  • juju attach-resource oci-image=redis

Here’s a tutorial which explains what to do.

See the “Local Charms” section. TL;DR: docker image metadata is modelled as Charm Resources. So the standard --resource semantics are used. eg

juju deploy /path/to/mariadb-k8s-local --resource mysql_image=mariadb:latest
2 Likes