Juju, Kubernetes and microk8s


#1

microk8s is awsome

If you want to quickly and easily do some Kubernetes charm development, or hack on Juju’s Kubernetes support, do yourself a favour and snap install microk8s.

I have previously described how to run a Kubernetes demo on AWS. It’s great and all, but the time to bootstrap, deploy a Kubernetes bundle, and run the trust command to set up the necessary permissions means that you are waiting up to say 20 minutes to get started. And often, you’ll need to tear things down and start again as part of the development process. And you need to be online.

With microk8s, you have a local, fully compliant Kubernetes deployment with dynamic persistent volume support, and a running ingres controller.

Ensure microk8s is set up correctly

After installing mictok8s, you’ll want to enable dns and storage.

sudo snap install microk8s --edge --classic
microk8s.enable dns storage

I normally also alias microk8s.kubctl:
sudo snap alias microk8s.kubectl mkubectl

Juju time!

You can just bootstrap a LXD controller without any special config if you want to test with local charms. But if you want to use charms from the staging store, bootstrap like so:

juju bootstrap lxd --config charmstore-url=https://api.staging.jujucharms.com/charmstore

Now, register your microk8s cloud with Juju and add a model:

microk8s.config | juju add-k8s k8stest
juju add-model test k8stest

It is mandatory to set up a storage pool which Juju will use to allocate charm operator storage. Luckily, microk8s’ built in storage support has been enabled and there’s a storage class we can easily use with no extra effort on our part:

juju create-storage-pool operator-storage kubernetes storage-class=microk8s-hostpath

That’s all the setup we need. In the time taken to bootstrap a LXd controller, we’re ready to deploy Kubernetes workloads with Juju.

Deploy mariadb

Let’s deploy mariadb. It requires storage, so we’ll need to create a storage pool in Juju for it to use. But as before, there’s a ready made Kubernetes storage class we can leverage:

juju create-storage-pool mariadb-pv kubernetes storage-class=microk8s-hostpath
juju deploy cs:~wallyworld/mariadb-k8s --storage database=10M,mariadb-pv

juju status

Model  Controller  Cloud/Region  Version      SLA          Timestamp
test   ian         k8stest       2.5-beta1    unsupported  14:56:31+10:00

App      Version  Status  Scale  Charm        Store       Rev  OS          Address        Charm version  Notes
mariadb           active      1  mariadb-k8s  jujucharms    7  kubernetes  10.152.183.94                 

Unit        Workload  Agent  Address    Ports     Message
mariadb/0*  active    idle   10.1.1.32  3306/TCP  Started container

juju storage --filesystem

[Filesystems]
Unit       Storage     Id  Provider id                     Mountpoint      Size   State     Message
mariadb/0  database/0  0   juju-database-0-juju-mariadb-0  /var/lib/mysql  34MiB  attached  Successfully provisioned volume pvc-39a6de3a-b255-11e8-a2de-80fa5b27f2bf

The actual physical storage location is at /var/snap/microk8s/common/default-storage/ so you can easily inspect the files created by both the Juju mariadb operator and the database itself.

ls /var/snap/microk8s/common/default-storage/
test-juju-database-0-juju-mariadb-0-pvc-39a6de3a-b255-11e8-a2de-80fa5b27f2bf  test-mariadb-operator-volume-juju-operator-mariadb-0-pvc-33d08565-b255-11e8-a2de-80fa5b27f2bf

Known Issues

When destroying a model, it’s possible that Kubernetes will leave behind a persistent volume claim, and this seems to prevent the model destruction from completing.

If you want to reset things and “start again”, it’s easy enough just to do:

juju kill-controller <name> -y -t 0
microk8s.reset
juju remove-cloud k8stest

#2

Getting Started
#3

I had to use:

microk8s.enable dns storage

To get storage enabled.


#4

This command gave me an error:

juju create-storage-pool operator-storage kubernetes storage-class=microk8s-hostpath
ERROR storage provider "kubernetes" not found (not found)

#5

You’ll get that error if you attempt to create a kubernetes storage pool on a model that is not a Kubernetes model.


#6

On a fresh machine, I get

/snap/microk8s/251/microk8s-config.wrapper: line 38: netstat: command not found
/snap/microk8s/251/microk8s-config.wrapper: line 39: ifconfig: command not found

So, net-tools need to be installed on a client, for example ‘sudo apt install net-tools’.


#7

@tvansteenburgh @kos.tsakalozos can you provide input here?


#8

@wallyworld, @anastasia-macmood, thank you for spotting this problem.

We need to package net-tools within microk8s. A workaround for now is to ‘sudo apt install net-tools’ as you suggested.

Just added the following issue https://github.com/ubuntu/microk8s/issues/148 . We will get to it soon.

Thank you again.


#9

If you are adding a k8s model to a LXD controller as in this example, and you have UFW enabled, you might get an error like:

ERROR failed to list namespaces: Get http://192.168.1.16:8080/api/v1/namespaces?includeUninitialized=true: dial tcp 192.168.1.16:8080: i/o timeout

Granting access to port 8080 from the LXD bridge subnet solves this. For me it was:

sudo ufw allow from 10.7.95.1/24 to any port 8080

Juju 2.5.0 Beta 1 Release Notes