Removing a model in JAAS takes forever?

1

I’m trying to remove a model in jaas:

erik@juju-dev:~$ juju destroy-model testing
WARNING! This command will destroy the “testing” model.
This includes all machines, applications, data and other resources.

Continue [y/N]? y
Destroying model
Waiting on model to be removed…
Waiting on model to be removed…

…

Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
Waiting on model to be removed…
ERROR timeout after 30m0s timeout

Yeah, you get the idea. 30 minutes later, the model is still not removed and ERROR.

2

This incident is better bugged up than be on discourse.

There are a few vital pieces of information missing here - what version of Juju is running? was this model upgraded? what cloud is it on? what workload did this model have? did it use any additional features like storage, CMR, etc.? what other recent commands where used on this model? are you using the same client or have you changed clients since this model’s creation/previous usage?

All of these need to be provided in the bug as well as the logs.
We cannot really address issues like this without a reproducible scenarios since we are not able to observe this behavior in any of our CI nor running models.

3

@anastasia-macmood Thanx for chipping in. I’m sorry for not providing more context, but I have managed to reproduce this again where I have more context.

I’m using an AWS account provided by my company, which has a feature that automatically “shuts down” my nodes during nights at some schedule unless i tag the resources.

When I run “juju destroy-model”, the machines gets removed, but the model ends up in “Status: destroying” and timeouts.

If I’m creating another model, without any of the machines having being “automatically turned off”. Then the model gets removed successfully.

I’m using

  • juju version: 2.5.2-bionic-amd64
  • Cloud: AWS (aws/eu-west-1)
  • No additional features added.

Is this a bug or a known “feature” related to AWS perhaps? I’m happy to send in a bug as soon as I’m sure its not the expected behaviour for some reason.

4

@erik-lonroth,

This is great and well explained. Yes, please bug it up and provide these steps as well as any logs that you can access.

Thank you \o/ We’ll look into it.

5

Juju doesn’t remove machines that are currently down, unless you have specified ‘–force’. It is waiting for the machine agent on the machine to acknowledge that it is shutting down. Otherwise we can’t tell the difference between a temporary network failure/the machine will be coming back, and the machine is fully gone.

We could potentially treat ‘juju destroy-model’ as applying a --force to all machines, since we know we will be cleaning them up/shutting them down via the Provider.

1 Like