Playing around with k8s charms a bit I’ve started to notice some places where unique/user defined annotations might be really useful on a per deployment basis.
An example of this could be loadbalancer customization. Imagine CDK on AWS, by default, when an application is configured with an ingress type of
LoadBalancer, k8s will give you a classic elb that forwards port 80 to the exposed port of your application. This is something we can modify by using annotations, and something that should be customizable on a per deployment basis.
Find here a charm pod spec template with the top level key
service and a child key
service: annotations: %(annotations)
annotations exposed as a charm config.
And templated into the pod spec in the reactive file here.
In this way, we could allow the user to specify custom annotations at deploy time that are relevant to the user’s environment.
For example, if I wanted to have a custom cert in front of my loadbalancer, and also want an alb instead of a classic elb. I could deploy the charm with annotations config such that the resource would be configured to my custom spec.
$ juju deploy cs:~jamesbeedy/jenkins-k8s --config jenkins_k8s_config.yaml \ --config kubernetes-service-type=LoadBalancer \ --config juju-external-hostname="jenkins-k8s.myexamnple.com" \ --storage jenkins-home=10G,k8s-ebs
jenkins_k8s_config.yaml contains some annotations to get my loadbalancer how I want it.
jenkins-k8s: annotations: | service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:<aws-account-id>:certificate/<some-uuid-for-cert> service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: false service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http service.beta.kubernetes.io/aws-load-balancer-ssl-ports: 443 service.beta.kubernetes.io/aws-load-balancer-type: alb
The above would work great if it was possible to pass yaml as a yaml string in that way
Which leads me to think of passing in some base64 yaml string and decoding it in the charm … or a json string.
While I feel this is moving in the right direction, and a proper place where charm config could facilitate custom annotations, there is something that isn’t sitting right about passing in a json string or base64 yaml string to be decoded/parsed back into yaml and set to the pod spec.
Anyone else crossed this path yet?