Unable to create loadbalancers with cs:octavia charm

1

I followed this blog post to try and deploy Octavia on Openstack created with Juju and Maas.

Everything seems to be working but when I try to create a load balancer the amphora instance fails. I’ve tried creating them manually with the Openstack CLI, via the Openstack dashboard and via a Kubernetes cluster that was deployed on Openstack with Juju. The problem may be with Openstack but I thought I’d start here first.

I thought maybe the issue was with creating the instance in Nova so I watched the logs there and everything seems fine.

Nova Log - https://gist.github.com/2stacks/152918ec7f7d60c61ba17d5809211c97

The only place I see an indication of an issue is in the Octavia worker logs. There seems to be a clear indication that the controller can’t reach the newly created instance over the lb-mgmt-net.

2019-08-24 16:23:37.528 675855 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='fc00:8f36:122:78d8:f816:3eff:fe35:60e1', port=9443): Max retries exceeded with url: /0.5/info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fbc2549d048>: Failed to establish a new connection: [Errno 111] Connection refused',))
2019-08-24 16:23:42.585 675855 ERROR octavia.amphorae.drivers.haproxy.exceptions [-] Amphora agent returned unexpected result code 404 with response {'error': '404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.', 'http_code': 404}
2019-08-24 16:23:43.013 675855 WARNING octavia.controller.worker.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (1858607d-f808-494f-8ead-8e4bc012a3e2) transitioned into state 'FAILURE' from state 'RUNNING'

Octavia Log - https://gist.github.com/2stacks/736d6b36f97cec0dc05025a7b198bdb2

I verified that all of the necessary networks, security groups, flavors etc. that are created when running

juju run-action --wait octavia/0 configure-resources

have been created in Openstack as expected. I also double check that the Octavia controller can reach other resources on the IPv6 lb-mgmt-net.

root@juju-9e01a5-0-lxd-10:/var/log/octavia# ping fc00:8f36:122:78d8::1
PING fc00:8f36:122:78d8::1(fc00:8f36:122:78d8::1) 56 data bytes
64 bytes from fc00:8f36:122:78d8::1: icmp_seq=1 ttl=64 time=1.69 ms
64 bytes from fc00:8f36:122:78d8::1: icmp_seq=2 ttl=64 time=0.181 ms

root@juju-9e01a5-0-lxd-10:/var/log/octavia# ping fc00:8f36:122:78d8:f816:3eff:fe05:884c
PING fc00:8f36:122:78d8:f816:3eff:fe05:884c(fc00:8f36:122:78d8:f816:3eff:fe05:884c) 56 data bytes
64 bytes from fc00:8f36:122:78d8:f816:3eff:fe05:884c: icmp_seq=1 ttl=64 time=0.525 ms
64 bytes from fc00:8f36:122:78d8:f816:3eff:fe05:884c: icmp_seq=2 ttl=64 time=0.180 ms

I’m pretty much at a loss for where to look next. Any advice will be greatly appreciated.

2

Well I solved the problem so I’m posting what I found in case anyone else comes across the same issue.

I made the mistake of using the daily pre-built amphora images from http://tarballs.openstack.org/octavia/test-images/. I tried both bionic and xenial thinking that was a good test. Out of desperation I tried them all, found that the fedora image actually worked so decided to look in to it further.

I decided to build my own image following instructions from https://github.com/openstack/octavia/tree/stable/stein/diskimage-create and making sure I used the correct branch for the version of Openstack that Juju deployed (currently stable/stein).

That branch builds a xenial amphora image by default which works as expected.

4 Likes
3

Hello, @stacks,

Thank you for using the OpenStack charms, and great to hear you resolved your issue!

Since the incarnation of the blog post you referenced we have further developed the image part of the charmed Octavia experience.

Take a look at the updated Amphora image section of the Octavia LBaaS appendix of the OpenStack Charms Deployment Guide.

Cheers!

1 Like
4

Thank you for the follow up. I’ll use this process for creating the images going forward.