MAAS: Instance not found

help-needed

#1

Hi all,
We are using VMs from several MaaS PoDs in one of our juju models to host some units. Due to hard disk limitations we had to move a machine from one PoD to another. As there is no straight forward way for MaaS to do that, we deployed a new machine (same as the obsolete one), changed through virsh the virtual disk and brought the new machine up again with the exact same state as the obsolete.

The problem is that juju complains it cannot find the instance for a specific unit. We didn’t want to go with remove-unit/add-unit due to some customizations we have on this machine, not scripted yet.

I searched through the juju db and configuration files and I have found only one reference of the machine id in mongodb instanceData . I have changed it, carefully to match exaclty with MaaS and restarted both the controllers and the agent on the machine.

Still no luck and the agent on the machine continuously logs the following:

2019-09-19 10:50:36 ERROR juju.worker.dependency engine.go:663 “machiner” manifold worker returned unexpected error: cannot update observed network config: cannot get network interfaces of “wg8gx4”: instance “wg8gx4” not found (not found)

The id “wg8gx4” is the correct id of the new machine in MaaS. Does anyone have any insight of the code to shred some light ?!?!


#2

In MAAS, machines have an AgentName field.

This is used by Juju so that machines are created/queried/destroyed in the context of a model. I would hazard a guess that when the machine was created anew in MAAS, it was not created with this field, and so Juju does not find it when querying.

I am not sure how to go about setting this to the correct value in MAAS, but perhaps the folks over at https://discourse.maas.io/ can be of some help.


#3

I think you might find there’s other information related to the machine in other documents that have the old id. You might try a db dump and see if you can find any other instances of that machine id, but unfortunately this isn’t greatly supported to hack the db to move things around sans-juju.


#4

Good catch! I have changed the agent_name in the table maasserver_node from “empty” to the model uuid and have gone one step beyond. The log now know the juju machine id:

2019-09-20 13:48:08 ERROR juju.worker.dependency engine.go:663 "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot get network interfaces of "wg8gx4": instance "wg8gx4" not found (not found)
2019-09-20 13:50:16 ERROR juju.worker.dependency engine.go:663 "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot set link-layer devices to machine "3": cannot change ProviderID of link layer device "ens4"

Still trying to figure the new error with the ProviderID…


#5

Well it seems that it was a little bit trickier to figure…
You have to retreive the ids of all interfaces of the “new” server from the table maasserver_interface and with those ids handy, update the juju mongodb collection linklayerdevices.

So far so good :slight_smile: