Preface
My initial efforts in getting CephFS PVCs to work with CDK were cut short due to the containers not being able to reach the storage on the underlying l2 network when attempting to mount the CephFS.
Problem
Cannot enable the containers deployed on Flannel or Calico backed CDK to communicate with anything (storage) on the underlying l2 network.
Proposed Solution
Implement linux-bridge networking in order to get the containers on a local l2 network where they can reach the storage.
This can be done by writing a linux-bridge network plugin charm that relates to the Kubernetes charms on the cni interface.
I’m thinking this would allow us implement linux-bridge networking with CDK on bare metal and enable our containers to reach our storage.
The issues I foresee are such that the linux-bridge method isn’t really a plugin, but more or less a “way” to get containers on the l2.
I suspect there is a good amount of work needed to get linux-bridge method networking implemented as a useable solution. I’m sure there is a better path forward here that I am just not privy to.
People have suggested kube-router and/or using a hardware vendor solution where we can do l3 <-> l2 routing at the switch level. I will be exploring these options in the coming weeks.
Thoughts?
