K8S Charms - Annotations/Charm Config Use Case


#1

Playing around with k8s charms a bit I’ve started to notice some places where unique/user defined annotations might be really useful on a per deployment basis.

An example of this could be loadbalancer customization. Imagine CDK on AWS, by default, when an application is configured with an ingress type of LoadBalancer, k8s will give you a classic elb that forwards port 80 to the exposed port of your application. This is something we can modify by using annotations, and something that should be customizable on a per deployment basis.

Find here a charm pod spec template with the top level key service and a child key annotations

service:
  annotations:
    %(annotations)

Find here annotations exposed as a charm config.

And templated into the pod spec in the reactive file here.

In this way, we could allow the user to specify custom annotations at deploy time that are relevant to the user’s environment.

For example, if I wanted to have a custom cert in front of my loadbalancer, and also want an alb instead of a classic elb. I could deploy the charm with annotations config such that the resource would be configured to my custom spec.

For example:

$ juju deploy cs:~jamesbeedy/jenkins-k8s --config jenkins_k8s_config.yaml \
  --config kubernetes-service-type=LoadBalancer \
  --config juju-external-hostname="jenkins-k8s.myexamnple.com" \
  --storage jenkins-home=10G,k8s-ebs

Where jenkins_k8s_config.yaml contains some annotations to get my loadbalancer how I want it.

jenkins-k8s:
  annotations: |
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:<aws-account-id>:certificate/<some-uuid-for-cert>
    service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: false
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: 443
    service.beta.kubernetes.io/aws-load-balancer-type: alb

The above would work great if it was possible to pass yaml as a yaml string in that way :slight_smile:

Which leads me to think of passing in some base64 yaml string and decoding it in the charm … or a json string.

While I feel this is moving in the right direction, and a proper place where charm config could facilitate custom annotations, there is something that isn’t sitting right about passing in a json string or base64 yaml string to be decoded/parsed back into yaml and set to the pod spec.

Anyone else crossed this path yet?