Kubernetes-master stuck in waiting state

I’m attempting to deploy the charmed-kubernetes bundle and the kubernetes-master application is stuck.

Here are the relevant lines from juju status. The rest is available on the Ubuntu pastebin.

App                    Version  Status   Scale  Charm                  Store       Rev  OS      Notes
...
kubernetes-master      1.15.2   waiting      2  kubernetes-master      jujucharms  700  ubuntu  

Unit                      Workload  Agent      Machine  Public address  Ports           Message
...
kubernetes-master/1*      waiting   idle       6        10.247.4.199    6443/tcp        Waiting for 7 kube-system pods to start
  containerd/1            active    idle                10.247.4.199                    Container runtime available.
  flannel/1               active    idle                10.247.4.199                    Flannel subnet 10.1.34.1/24
kubernetes-master/2       waiting   idle       10       10.247.4.234    6443/tcp        Waiting for 7 kube-system pods to start
  containerd/2            active    idle                10.247.4.234                    Container runtime available.
  flannel/2               active    idle                10.247.4.234                    Flannel subnet 10.1.71.1/24

Any suggestions for prodding the the application so that it can move forward? Perhaps a config setting?

The thing to do here is to figure out what is keeping the pods from coming up. I would either attach a cdk-field-agent report or poke around with kubectl to see what is going on. Something like kubectl get po -A will list all the pods and then pick one that is having trouble like core-dns and kubectl describe -n kube-system core-dns-xxxxx, where xxxxx is replaced with the hash of the pod from the get po command above. That should tell you something about why it isn’t starting. Another thing to check is kubectl logs -n kube-system core-dns-xxxxx.

For collecting debug info, we’re moving away from cdk-field-agent in favor of using juju-crashdump.

sudo snap install juju-crashdump --channel edge
juju-crashdump -a debug-layer -a config