Can't remove relation on CMR?

I was trying to use CMR to let a monitoring model scrape data from ceph-mon. It turns out ceph-mon doesn’t support CMR on the client relation, but now I can’t seem to remove the offer.

Steps I’ve taken so far.

From the Ceph controller

juju offer ceph-mon:client
Application "ceph-mon" endpoints [client] available at "admin/prod.ceph-mon"

From the consuming controller

juju add-relation ceph-exporter prod-maas:admin/prod.ceph-mon

This part worked as expected, and that’s when I found the limitation on the relation handling of ceph-mon. To reverse this I then:

From the consuming controller

juju remove-relation ceph-exporter ceph-mon

This worked and removed the relation, the charm recognized this and set relation flags (states) as expected. However, after this I tried to re-add it to debug further and that’s when I noticed issues.

The relation never fired a new hooks for the ceph-exporter. The relation was also not added to the list of relations. Checking back on the offer model I see that it still think the connection exists.

From the offer controller

Offer     Application  Charm     Rev  Connected  Endpoint           Interface    Role
ceph-mon  ceph-mon     ceph-mon  42   1/1        client             ceph-client  provider

I’ve tried removing the relation (from the consumer), then removing the SAAS from the Offer and it says there is still a relation. I then tried removing with --force which doesn’t error but doesn’t remove the relation either.

As far as I can tell the ceph-mon has never had the ‘departed’ hook fired and the offer model doesn’t seem to recognize the removal of the relation at all. Am I doing something wrong or is there an issue with CMRs here?

Both controllers are juju 2.6.6 as are the models.

Sorry to hear about the issue you’re having @chris.sanders. Thanks for taking the time to report the issue.

We’ve spent some time this cycle eliminating similar bugs. It’s unfortunate to see that there are still some cases that are causing issues.

So what are my options, should I open a LP bug to work through it? I don’t want to remove the application or model as I’m using these it’s not a test system.

Yes, filing a bug report would be a useful next step. I’ll try to look into this myself today/tomorrow and see if I can provide any recommendations.

Thanks tim, I’ve opened Bug #1840850 “CMR fails to remove from offer model” : Bugs : juju to track it as a bug. I’m happy to comment here or there if I can gather information to help understand what’s going on.

Discourse uses my personal email, which I’m generally more responsive to but I’m happy to use whichever is better for you.

Thanks for taking the time to post that Chris. I know it can be tedious, but it makes triage etc significantly simpler.

I left a question on the bug, best to correspond there so as not to double up.