Cross Model Relations and Openstack


#1

It seems practical to deploy parts of Openstack into separate models.

In theory, given Juju cross model relations AND given the Openstack charms, deploying Openstack in a cross model context seems reasonable.

Despite what I knew to be true, I have given deploying the Openstack components in a CMR fashion at multiple points in time; 1) Juju 2.3 (the dawn of CMR) - just about everything in the deploy seemed broken, 2) Juju 2.4 - far less brokeness, but many things still borked, 3) Juju 2.5 (congrats on the release today!) - juju status green across all models (first time an Openstack deploy has actually settled using CMR), some functionality is broken/missing.

My hope is that @rr-pdl and myself alongside some insight of others, can help get to the bottom of the issues that remain in CMR and thus also in CMR Openstack.

@rr-pdl and myself have attempted to create a POC of CMR Openstack using 3 separate models:

$ juju models
Controller: maas-controller

Model                Cloud/Region  Status     Machines  Cores  Access  Last connection
ryan/ceph-storage    pdl-maas      available        12    336  admin   3 hours ago
ryan/pdl-openstack*  pdl-maas      available        17    424  admin   16 minutes ago
ryan/utilities       pdl-maas      available         6    104  admin   2 hours ago

$ for model in ryan/ceph-storage ryan/utilities ryan/pdl-openstack; do
    juju status -m $model --relations;
done

https://paste.ubuntu.com/p/gvdPq62Nmf/

Although we are applying the same relations between the same components we would be in a non-CMR style deploy, we are seeing missing ceph config where we would expect to see it amongst other things.

Possibly we can use the space below to enumerate the list of issues we are seeing in the cross model context and their associated logs and resources.

Thanks!


Openstack Charms and Juju Cross Model Relationship
#2

I’ve used conjure-up to deploy Openstack with nova-lxd to compare cinder and ceph configuration files on cinder:

The monitor and auth portions of the configuration are not being set in the CMR deploy.

ceph.conf:

[cinder-ceph] section of configuration is not being added

cinder.conf:


#3

@rr-pdl I’m thinking, possibly https://jujucharms.com/ceph-proxy/ might be an answer here. So long as the only relations still giving us grief are the ceph-mon:client and ceph-mon:radosgw relations, possibly we can just sub in the ceph-proxy and be off to the races!


#4

Are these issues that the charms aren’t getting something over a CMR vs a typical relation? I’m not seeing if this is a bug in the Juju CMR code or something that the charms need tweaking to better support the CMR variant (which honestly is still probably a Juju bug as charms shouldn’t need to worry too much about CMR vs not)


#5

Just FYI, @james-page mentioned the other day in #openstack-charms that the OpenStack charms were largely untested with CMR so your milage may vary.


#6

The relations that didn’t work in a cross model context were the relations to ceph-mon:client. We were able to get a decoupled storage/control plane working using the ceph-proxy charm.


#7

The ceph-mon charm uses the remote unit names in relation keys when generating responses to broker requests to setup pools and access to the ceph cluster.

In a cross-model relation the remote unit name on the offer side is obfuscated so the ceph-mon charm generates a broker response message with a unit name that does not match the unit names on the consuming side. This means the relation never completes as the consuming side and offer side keys don’t match.

This is a pattern that’s used in a few places in charms - I think MySQL has a similar pattern to indicate when access for a specific remote unit has been setup.

As CMR’s obfuscate the unit names, we need to move towards something unit-name agnostic; I’m leaning towards each consuming unit providing a fixed nonce of data which is then used in the broker responses instead of the unit name but that’s going to need some careful thinking to get right.


#8

To answer Rick’s question, charms don’t need to be CMR aware, but they need to not assume they can rely on knowing the remote unit name as James explains.