Machine-0.log getting spammed with (AuthFailure)


#1

My controller machine-0.log is getting hit pretty hard with the same set of events scrolling by [0].

It seems I have some models stuck in destroying [1]. Not sure if this may be part of the log spam issue (like possibly a model that is stuck in destroying cannot destroy because it has invalid creds or something), its difficult for me to tell from the log event information what the origin is other then that it is an AuthFailure. Possibly I should temporarily turn up the verbosity to get more insight? Possibly someone else has some insight?

Maybe there is a way we can display a message in juju status if the credentials are invalid?

On another note, how do people feel about a --force flag for juju destroy-model command?

[0] https://paste.ubuntu.com/p/xzVjyxqjt7/
[1] https://paste.ubuntu.com/p/NYncQxhQkB/


#2

We really want a --force flag for the juju destroy-model command. Most of our long-running controller have models stuck in the destroying phase.


#3

That’s on the todo list for 19.04 along with juju remove-application --continue-on-errors and other aids in getting rid of units as well.


#4

There is work under way that should address the log spamming issue. In 2.5+, once we’d get invalid credential message from the cloud, Juju will stop communicating with it until the credential that a model uses is updated.

There are also plans to surface invalid credential message in status as well as other model related commands like ‘show-model’, ‘models’, etc.

At this stage, to get unstuck, you need to update the credential that these destroying models use. This will allow model destruction to proceed.