Hi,
With a normal and correct deployment of Openstack with ceph as storage backend you are able to utilize the snapshot features of ceph to basically instantly copy a Glance image and boot the instance with nova.
This requires that cephx is correct and the user client.nova-compute has “rwx” access to the glance pool.
I am also using RAW images.
However i have verified that my config is correct but still, it does not work. I have verified against a separate staging cluster.
Example of my ceph auth
client.cinder-ceph
key: foobar1==
caps: [mon] allow r; allow command "osd blacklist"
caps: [osd] allow rwx
client.glance
key: foobar2==
caps: [mon] allow r; allow command "osd blacklist"
caps: [osd] allow rwx
client.nova-compute
key: foobar3==
caps: [mon] allow r; allow command "osd blacklist"
caps: [osd] allow rwx
It is the same for our Production and Staging environments. But still it does not work in production.
I get these errors from nova-compute.log:
2019-09-25 12:24:34.452 87448 DEBUG nova.virt.libvirt.imagebackend [req-bdf268b7-ce68-4a0d-94e8-a040d4d182c7 dcfc6807c683426697a698b2b1db2aa8 809617a6db2641a9b219a32548a9a75e - 2eb5f30ff25e4a7fae1e6a059a6a8587 2eb5f30ff25e4a7fae1e6a059a6a8587] Image locations are: [{'url': 'rbd://b%273bc43fec-6042-11e9-9012-00163e6d2e65%27/glance/106b5da6-e82c-49f3-9ce7-bdba47d5fec4/snap', 'metadata': {}}, {'url': 'rbd://b%273bc43fec-6042-11e9-9012-00163e6d2e65%27/glance/106b5da6-e82c-49f3-9ce7-bdba47d5fec4/snap', 'metadata': {}}] clone /usr/lib/python3/dist-packages/nova/virt/libvirt/imagebackend.py:916
2019-09-25 12:24:34.472 87448 DEBUG nova.virt.libvirt.storage.rbd_utils [req-bdf268b7-ce68-4a0d-94e8-a040d4d182c7 dcfc6807c683426697a698b2b1db2aa8 809617a6db2641a9b219a32548a9a75e - 2eb5f30ff25e4a7fae1e6a059a6a8587 2eb5f30ff25e4a7fae1e6a059a6a8587] rbd://b%273bc43fec-6042-11e9-9012-00163e6d2e65%27/glance/106b5da6-e82c-49f3-9ce7-bdba47d5fec4/snap is in a different ceph cluster is_cloneable /usr/lib/python3/dist-packages/nova/virt/libvirt/storage/rbd_utils.py:209
2019-09-25 12:24:34.487 87448 DEBUG nova.virt.libvirt.storage.rbd_utils [req-bdf268b7-ce68-4a0d-94e8-a040d4d182c7 dcfc6807c683426697a698b2b1db2aa8 809617a6db2641a9b219a32548a9a75e - 2eb5f30ff25e4a7fae1e6a059a6a8587 2eb5f30ff25e4a7fae1e6a059a6a8587] rbd://b%273bc43fec-6042-11e9-9012-00163e6d2e65%27/glance/106b5da6-e82c-49f3-9ce7-bdba47d5fec4/snap is in a different ceph cluster is_cloneable /usr/lib/python3/dist-packages/nova/virt/libvirt/storage/rbd_utils.py:209
Please help