Auth Error - Unable to connect to the server: x509

charm
1

I feel so silly … most likely something simple I’m missing but possibly an extra set of eyes would help here.

Simply put, I deploy juju deploy kubernetes-core, and can’t seem to communicate via kubectl - seems like this was just working for me though, so just not really sure if it’s a personal issue or what.

$ juju deploy kubernetes-core

$ juju status
Model               Controller  Cloud/Region   Version  SLA          Timestamp
k8s-spark-testing0  pdl-aws     aws/us-west-2  2.4.6    unsupported  15:44:32-08:00

App                Version  Status  Scale  Charm              Store       Rev  OS      Notes
easyrsa            3.0.1    active      1  easyrsa            jujucharms  185  ubuntu  
etcd               3.2.10   active      1  etcd               jujucharms  319  ubuntu  
flannel            0.10.0   active      2  flannel            jujucharms  339  ubuntu  
kubernetes-master  1.12.2   active      1  kubernetes-master  jujucharms  522  ubuntu  exposed
kubernetes-worker  1.12.2   active      1  kubernetes-worker  jujucharms  378  ubuntu  exposed

Unit                  Workload  Agent  Machine  Public address  Ports           Message
easyrsa/0*            active    idle   0/lxd/0  252.102.67.123                  Certificate Authority connected.
etcd/0*               active    idle   0        172.31.102.67   2379/tcp        Healthy with 1 known peer
kubernetes-master/0*  active    idle   0        172.31.102.67   6443/tcp        Kubernetes master running.
  flannel/1           active    idle            172.31.102.67                   Flannel subnet 10.1.85.1/24
kubernetes-worker/0*  active    idle   1        172.31.103.148  80/tcp,443/tcp  Kubernetes worker running.
  flannel/0*          active    idle            172.31.103.148                  Flannel subnet 10.1.36.1/24

Entity  Meter status  Message
model   amber         user verification pending  

Machine  State    DNS             Inst id              Series  AZ          Message
0        started  172.31.102.67   i-0dd3916c6aecc21bf  bionic  us-west-2a  running
0/lxd/0  started  252.102.67.123  juju-4b0686-0-lxd-0  bionic  us-west-2a  Container started
1        started  172.31.103.148  i-0741ed969dbe21360  bionic  us-west-2b  running


$ rm -rf ~/.kube/

$ mkdir -p ~/.kube

$ juju scp kubernetes-master/0:config ~/.kube/config

$ sudo snap install kubectl --classic
snap "kubectl" is already installed, see 'snap help refresh'

$ kubectl cluster-info

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Unable to connect to the server: x509: certificate has expired or is not yet valid

$ kubectl cluster-info dump
Unable to connect to the server: x509: certificate has expired or is not yet valid

$ cat ~/.kube/config 
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURQakNDQWlhZ0F3SUJBZ0lKQUx0WmZMR2d1NWlFTUEwR0NTcUdTSWIzRFFFQkN3VUFNQmt4RnpBVkJnTlYKQkFNTURqSTFNaTR4TURJdU5qY3VNVEl6TUI0WERURTRNVEV5TnpJeU1qVXdORm9YRFRJNE1URXlOREl5TWpVdwpORm93R1RFWE1CVUdBMVVFQXd3T01qVXlMakV3TWk0Mk55NHhNak13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBCkE0SUJEd0F3Z2dFS0FvSUJBUURXVmZrWFJGcFJpM0F4cU9oSmZzdDNkRE5SQmY4aHhxaWRJVUJLNmJiME5EeVQKSlQwcWNPZHJUWjVtai85d3ViMDhsZWN2d0lmblBDYlBGRGtIaUIzRm1FelgyR1psYzBkTTNsckZUMUx4NkJORApmRHVvK2xablgzbGhPQ1J3VUJVQlIvQWxwZnFuSld5RkVjVzEvKzVnWlhWNGt5c3RxL0hpNW5WZ1ZNMTdlaEtjCmh2SDdHT2wrcEVKNDRyZCt6MEE0dkZZN3Q4N3pQUURnMVZRWFR3UUR4WVFoQzcwdnUzemEzMWluRFdzcEdUa2UKNTY4Kzh1aGpIOHNYcUhJWTN6ZjJXaldiRjRhU3o3VWtOQ3FsNnlOOHRNSEluMjJlbDh4Lzd2cS9pV0k5NWowSQprbTFkZ0VkeVZPMldTNUhRZW1tcU9qbWNVTElkU2Fub2p3cVRhSzlYQWdNQkFBR2pnWWd3Z1lVd0hRWURWUjBPCkJCWUVGSDZiWTBTbytncFgyT1hVUDhndFNtY2lzcFd4TUVrR0ExVWRJd1JDTUVDQUZINmJZMFNvK2dwWDJPWFUKUDhndFNtY2lzcFd4b1Iya0d6QVpNUmN3RlFZRFZRUUREQTR5TlRJdU1UQXlMalkzTGpFeU00SUpBTHRaZkxHZwp1NWlFTUF3R0ExVWRFd1FGTUFNQkFmOHdDd1lEVlIwUEJBUURBZ0VHTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCCkFRQS9mcVRHc3dVOHhDN3RNaVFCTWRvSksrOHR4RjAwQ2x3ZTEvTHNBUEdlcU1UZGxqN0F6TFBmVHhkMGpCb2YKQVhHdlBTaWlTOUpJMGx2VUUwVkZCZmJsS1RpT2grYnUyMmNtdFJVa1ExRDh0dEFPdzFCRGM5ZFVwM0VkNlR6eQpjNWozbkRYaitUQ1pycGM5bWxnbmdOOGxNWkNRc3ZRMW9UU0Jvd0VnTHJ2Sjd2V1RuVGRSOGJ4V0d5MUhpWG1LCnFYZ0ViN0VGc0c5R3NybGRxM2lFMlBocHFTMU5wazRmRjBOekQ0Qkt6MVZ5MERpRzNLMUp4aHVScHZ3cGRwWHQKcDJBZjg3am1vN0FSM2lIRG8yejFtSmNZUWhGZ3gxcmsyQ2Z5ZGhIU0IwUjBuQTZNR0NRNTcxTFhvV3VPUS9SKwp4M05Dd0RCbk00cnYwUTBBc0tNV0NQSHcKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==
    server: https://172.31.102.67:6443
  name: juju-cluster
contexts:
- context:
    cluster: juju-cluster
    user: admin
  name: juju-context
current-context: juju-context
kind: Config
preferences: {}
users:
- name: admin
  user:
    password: TVgpmFsH6FDYeumw7JWO8B4pyHL7XPI0
    username: admin
2

Running from another machine everything checks out …

$ juju scp kubernetes-master/0:config ~/.kube/config

[pdl-aws:k8s-spark-testing0]
jamesbeedy@ip-192-168-1-196:~$ kubectl cluster-info
Kubernetes master is running at https://172.31.102.67:6443
Heapster is running at https://172.31.102.67:6443/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://172.31.102.67:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://172.31.102.67:6443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
Grafana is running at https://172.31.102.67:6443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
InfluxDB is running at https://172.31.102.67:6443/api/v1/namespaces/kube-system/services/monitoring-influxdb:http/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

I wonder what is getting in the way of the initial machine being able to auth against the k8s? … either way looks like this is user error somehow on my end, and works fine from my other box.

After a reboot my initial box can now successfully communicate with the k8s master. I wonder if I had some time skew or something going on…