The Octavia Loadbalancer
Preface
Octavia is a highly-available and scalable open source load balancing solution designed to work with OpenStack. Octavia handles load balancing services by managing and configuring a fleet of virtual machines – also known as amphorae – in its project. These amphorae run a HAproxy.
Alternatively, the OVN Load Balancer Driver is available, which works without Amphora instances. Details can be found in the section Using the OVN Load Balancer Driver.
Using the OVN Load Balancer Driver
In addition to the classic amphora provider, our platform also supports the ovn provider as an alternative.
Instead of launching Amphora VMs, the ovn provider uses the native load balancing features of OVN. This eliminates the overhead of dedicated Amphora instances, resulting in faster creation times and lower resource consumption.
The OVN provider does not support Layer 7 features (L7 policies, TLS termination, insert headers, etc.). It is intended for simple TCP/UDP load balancing at Layer 4.
The OVN provider cannot currently be configured via Horizon.
First Steps
To use Octavia, the client first needs to be installed on your system. Instructions for the installation can be found here of our guide.
Creating an Octavia-Ladbalancer
Here we specify the subnet ID in which the load balancer should be created.
$ openstack loadbalancer create --name Example-LB --vip-subnet-id 32259126-dd37-44d5-922c-99d68ee870cd [--provider ovn]
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| admin_state_up | True |
| created_at | 2019-05-01T09:00:00 |
| description | |
| flavor_id | |
| id | e94827f0-f94d-40c7-a7fd-b91bf2676177 |
| listeners | |
| name | Beispiel-LB |
| operating_status | OFFLINE |
| pools | |
| project_id | b15cde70d85749689e08106f973bb002 |
| provider | amphora |
| provisioning_status | PENDING_CREATE |
| updated_at | None |
| vip_address | 10.0.0.10 |
| vip_network_id | f2a8f00e-204b-4c37-9d19-1d5c8e4efbf6 |
| vip_port_id | 37fc5b34-ee07-49c8-b054-a8d591a9679f |
| vip_qos_policy_id | None |
| vip_subnet_id | 32259126-dd37-44d5-922c-99d68ee870cd |
+---------------------+--------------------------------------+
By default, the amphora provider is used.
With the amphora provider, Octavia now spawns the amphora instances in the background.
$ openstack loadbalancer list
+--------------------------------------+-------------+----------------------------------+--------------+---------------------+----------+
| id | name | project_id | vip_address | provisioning_status | provider |
+--------------------------------------+-------------+----------------------------------+--------------+---------------------+----------+
| e94827f0-f94d-40c7-a7fd-b91bf2676177 | Beispiel-LB | b15cde70d85749689e08106f973bb002 | 10.0.0.10 | PENDING_CREATE | amphora |
+--------------------------------------+-------------+----------------------------------+--------------+---------------------+----------+
Once the provisioning_status is ACTIVE, the process has completed successfully.
$ openstack loadbalancer list
+--------------------------------------+-------------+----------------------------------+--------------+---------------------+----------+
| id | name | project_id | vip_address | provisioning_status | provider |
+--------------------------------------+-------------+----------------------------------+--------------+---------------------+----------+
| e94827f0-f94d-40c7-a7fd-b91bf2676177 | Beispiel-LB | b15cde70d85749689e08106f973bb002 | 10.0.0.10 | ACTIVE | amphora |
+--------------------------------------+-------------+----------------------------------+--------------+---------------------+----------+
Create LB listener
In our example, we want to create a listener for HTTP on port 80. A listener here - as in other LB solutions - refers to the port of the front end.
$ openstack loadbalancer listener create --name Beispiel-listener --protocol HTTP --protocol-port 80 Beispiel-LB
+-----------------------------+--------------------------------------+
| Field | Value |
+-----------------------------+--------------------------------------+
| admin_state_up | True |
| connection_limit | -1 |
| created_at | 2019-05-01T09:00:00 |
| default_pool_id | None |
| default_tls_container_ref | None |
| description | |
| id | 0a3312d1-8cf7-41a8-8d24-181246468cd7 |
| insert_headers | None |
| l7policies | |
| loadbalancers | e94827f0-f94d-40c7-a7fd-b91bf2676177 |
| name | Beispiel-listener |
| operating_status | OFFLINE |
| project_id | b15cde70d85749689e08106f973bb002 |
| protocol | HTTP |
| protocol_port | 80 |
| provisioning_status | PENDING_CREATE |
| sni_container_refs | [] |
| timeout_client_data | 50000 |
| timeout_member_connect | 5000 |
| timeout_member_data | 50000 |
| timeout_tcp_inspect | 0 |
| updated_at | None |
| client_ca_tls_container_ref | |
| client_authentication | |
| client_crl_container_ref | |
+-----------------------------+--------------------------------------+
As already mentioned, the OVN provider only supports L4 protocols, i.e. TCP / UDP.
Once the admin_state_up is true the loadbalancer has been successfully created. The Octavia loadbalancer can be further configured at this point.
$ openstack loadbalancer listener list
+--------------------------------------+-----------------+-------------------+----------------------------------+----------+---------------+----------------+
| id | default_pool_id | name | project_id | protocol | protocol_port | admin_state_up |
+--------------------------------------+-----------------+-------------------+----------------------------------+----------+---------------+----------------+
| 0a3312d1-8cf7-41a8-8d24-181246468cd7 | None | Beispiel-listener | b15cde70d85749689e08106f973bb002 | HTTP | 80 | True |
+--------------------------------------+-----------------+-------------------+----------------------------------+----------+---------------+----------------+
Create LB pool
The LB-pool here refers to a collection of all objects (listeners, members, etc.) - comparable to a pool of public IP addresses from which one can be assigned. A pool for our example is created as follows:
$ openstack loadbalancer pool create --name Beispiel-pool --lb-algorithm ROUND_ROBIN --listener Beispiel-listener --protocol HTTP
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| admin_state_up | True |
| created_at | 2019-05-01T09:00:00 |
| description | |
| healthmonitor_id | |
| id | 4053e88e-c2b5-47c6-987e-4387d837c88d |
| lb_algorithm | ROUND_ROBIN |
| listeners | 0a3312d1-8cf7-41a8-8d24-181246468cd7 |
| loadbalancers | e94827f0-f94d-40c7-a7fd-b91bf2676177 |
| members | |
| name | Beispiel-pool |
| operating_status | OFFLINE |
| project_id | b15cde70d85749689e08106f973bb002 |
| protocol | HTTP |
| provisioning_status | PENDING_CREATE |
| session_persistence | None |
| updated_at | None |
| tls_container_ref | |
| ca_tls_container_ref | |
| crl_container_ref | |
| tls_enabled | |
+----------------------+--------------------------------------+
It should be noted that with openstack loadbalancer pool create --help all possible settings can be displayed. The most common settings and their choices:
--protocol: {TCP,HTTP,HTTPS,PROXY,PROXYV2,UDP,SCTP}
--lb-algorithm {SOURCE_IP,ROUND_ROBIN,LEAST_CONNECTIONS,SOURCE_IP_PORT}
As already mentioned, the OVN provider only supports L4 protocols, i.e., TCP/UDP. Furthermore, the lb-algorithm SOURCE_IP_PORT must be selected for the OVN provider.
The pool has been successfully created when the provisioning_status has reached the status ACTIVE.
$ openstack loadbalancer pool list
+--------------------------------------+---------------+----------------------------------+---------------------+----------+--------------+----------------+
| id | name | project_id | provisioning_status | protocol | lb_algorithm | admin_state_up |
+--------------------------------------+---------------+----------------------------------+---------------------+----------+--------------+----------------+
| 4053e88e-c2b5-47c6-987e-4387d837c88d | Beispiel-pool | b15cde70d85749689e08106f973bb002 | ACTIVE | HTTP | ROUND_ROBIN | True |
+--------------------------------------+---------------+----------------------------------+---------------------+----------+--------------+----------------+
Create the LB member
For our loadbalancer to know which backends it is allowed to forward to, we still need a member, which we define as follows:
$ openstack loadbalancer member create --subnet-id 32259126-dd37-44d5-922c-99d68ee870cd --address 10.0.0.11 --protocol-port 80 Beispiel-pool
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| address | 10.0.0.11 |
| admin_state_up | True |
| created_at | 2019-05-01T09:00:00 |
| id | 703e27e0-e7fe-474b-b32d-68f9a8aeef07 |
| name | |
| operating_status | NO_MONITOR |
| project_id | b15cde70d85749689e08106f973bb002 |
| protocol_port | 80 |
| provisioning_status | PENDING_CREATE |
| subnet_id | 32259126-dd37-44d5-922c-99d68ee870cd |
| updated_at | None |
| weight | 1 |
| monitor_port | None |
| monitor_address | None |
| backup | False |
+---------------------+--------------------------------------+
and
$ openstack loadbalancer member create --subnet-id 32259126-dd37-44d5-922c-99d68ee870cd --address 10.0.0.12 --protocol-port 80 Beispiel-pool
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| address | 10.0.0.12 |
| admin_state_up | True |
| created_at | 2019-05-01T09:00:00 |
| id | 2add1e17-73a6-4002-82af-538a3374e5dc |
| name | |
| operating_status | NO_MONITOR |
| project_id | b15cde70d85749689e08106f973bb002 |
| protocol_port | 80 |
| provisioning_status | PENDING_CREATE |
| subnet_id | 32259126-dd37-44d5-922c-99d68ee870cd |
| updated_at | None |
| weight | 1 |
| monitor_port | None |
| monitor_address | None |
| backup | False |
+---------------------+--------------------------------------+
It should be noted here that the two IP’s from 10.0.0. * already exist, listening on port 80 (web server), and deliver a simple website with the information about their service name. Assuming these web servers in the following example are Ubuntu/Debian and you have root permissions, you could quickly create a simple web page with:
root@BeispielInstanz1:~# apt-get update; apt-get -y install apache2; echo "you hit: you hit: webserver1" > /var/www/html/index.html
root@BeispielInstanz2:~# apt-get update; apt-get -y install apache2; echo "you hit: you hit: webserver2" > /var/www/html/index.html
We can check the status of created members as follows:
$ openstack loadbalancer member list Beispiel-pool
+--------------------------------------+------+----------------------------------+---------------------+--------------+---------------+------------------+--------+
| id | name | project_id | provisioning_status | address | protocol_port | operating_status | weight |
+--------------------------------------+------+----------------------------------+---------------------+--------------+---------------+------------------+--------+
| 703e27e0-e7fe-474b-b32d-68f9a8aeef07 | | b15cde70d85749689e08106f973bb002 | ACTIVE | 10.0.0.11 | 80 | NO_MONITOR | 1 |
| 2add1e17-73a6-4002-82af-538a3374e5dc | | b15cde70d85749689e08106f973bb002 | ACTIVE | 10.0.0.12 | 80 | NO_MONITOR | 1 |
+--------------------------------------+------+----------------------------------+---------------------+--------------+---------------+------------------+--------+
Now the “internal” construct of the loadbalancer is configured.
We now have:
- 2
members who provide the actual service via port 80 and between which the loadbalancing takes place, - a
poolfor thismember, - a
listenerwhich listens on port TCP/80 and makes aROUND_ROBINto the two endpoints and - a loadbalancer, which we used to combine all components.
The operating_status NO_MONITOR is corrected under healthmonitor.
Create and configure the floating IP
In order to be able to use the loadbalancer outside of our example network, we must reserve a floating IP and then link it to the vip_port_id of the example LB.
Using the following command we can create a Floating IP from the provider network:
$ openstack floating ip create provider
+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2019-05-01T09:00:00Z |
| description | |
| dns_domain | None |
| dns_name | None |
| fixed_ip_address | None |
| floating_ip_address | 185.116.247.133 |
| floating_network_id | 54258498-a513-47da-9369-1a644e4be692 |
| id | 46c0e8cf-783d-44a0-8256-79f8ae0be7fe |
| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'b15cde70d85749689e08106f973bb002', 'name': 'beispiel-tenant', 'domain_name': None}), 'cloud': '', 'region_name': 'fra', 'zone': None}) |
| name | 185.116.247.133 |
| port_details | None |
| port_id | None |
| project_id | b15cde70d85749689e08106f973bb002 |
| qos_policy_id | None |
| revision_number | 0 |
| router_id | None |
| status | DOWN |
| subnet_id | None |
| tags | [] |
| updated_at | 2019-05-01T09:00:00Z |
+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
In the next step we need the vip_port_id of the loadbalancer. Accomplished with the following command:
$ openstack loadbalancer show Beispiel-LB -f value -c vip_port_id
37fc5b34-ee07-49c8-b054-a8d591a9679f
With the following command we can now assign the public IP address to the loadbalancer. The LB (and thus also the endpoints behind it) can now be reached from the Internet.
openstack floating ip set --port 37fc5b34-ee07-49c8-b054-a8d591a9679f 185.116.247.133
We are now ready to test our loadbalancer deployment. With the following command we query our loadbalancer via port TCP/80 and then get a corresponding response from the single member:
$ for ((i=1;i<=10;i++)); do curl http://185.116.247.133; sleep 1; done
you hit: webserver1
you hit: webserver2
you hit: webserver1
you hit: webserver2
you hit: webserver1
you hit: webserver2
you hit: webserver1
... (usw.)
Create a healthmonitor
With the following command we create a monitor that, in the event of a failure of one of the backends, can remove a faulty backend from the load distribution, thus allowing the website or application to continue to be delivered cleanly.
$ openstack loadbalancer healthmonitor create --delay 5 --max-retries 2 --timeout 10 --type HTTP --name Beispielmonitor --url-path / Beispiel-pool
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| project_id | b15cde70d85749689e08106f973bb002 |
| name | Beispielmonitor |
| admin_state_up | True |
| pools | 4053e88e-c2b5-47c6-987e-4387d837c88d |
| created_at | 2019-05-01T09:00:00 |
| provisioning_status | PENDING_CREATE |
| updated_at | None |
| delay | 5 |
| expected_codes | 200 |
| max_retries | 2 |
| http_method | GET |
| timeout | 10 |
| max_retries_down | 3 |
| url_path | / |
| type | HTTP |
| id | 368f9baa-708c-4e7f-ace1-02de15598c5d |
| operating_status | OFFLINE |
| http_version | |
| domain_name | |
+---------------------+--------------------------------------+
As already mentioned, the OVN provider only supports L4 protocols, i.e. TCP / UDP.
In this example, the monitor removes the failing backend from the pool if the integrity check (–type HTTP, –url-path / ) fails every two five-second intervals(–delay 5, –max-retries 2, –timeout 10).
Once the server is restored and responds to TCP/80 again, it will be added back to the pool.
A manual failover can be enforced if status code of the web server is not equal to “200” or in the event that there is no response from the web server at all.
$ openstack loadbalancer healthmonitor show Beispielmonitor
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| project_id | b15cde70d85749689e08106f973bb002 |
| name | Beispielmonitor |
| admin_state_up | True |
| pools | 4053e88e-c2b5-47c6-987e-4387d837c88d |
| created_at | 2019-05-01T09:00:00 |
| provisioning_status | ACTIVE |
| updated_at | 2019-05-01T09:00:00 |
| delay | 5 |
| expected_codes | 200 |
| max_retries | 2 |
| http_method | GET |
| timeout | 10 |
| max_retries_down | 3 |
| url_path | / |
| type | HTTP |
| id | 368f9baa-708c-4e7f-ace1-02de15598c5d |
| operating_status | ONLINE |
| http_version | |
| domain_name | |
+---------------------+--------------------------------------+
From our deployment example, the result should look something like this:
you hit: webserver1
Mi 22 Mai 2019 17:09:39 CEST
you hit: webserver2
Mi 22 Mai 2019 17:09:40 CEST
you hit: webserver1
Mi 22 Mai 2019 17:09:41 CEST
you hit: webserver2
Mi 22 Mai 2019 17:09:42 CEST
you hit: webserver1
Mi 22 Mai 2019 17:09:43 CEST - up to now both webservers were online, but now the webserver2 went offline.
you hit: webserver1
Mi 22 Mai 2019 17:09:44 CEST - still expected hit by ROUND_ROBIN
you hit: webserver1
Mi 22 Mai 2019 17:09:50 CEST - first retry to webserver2 fails
you hit: webserver1
Mi 22 Mai 2019 17:09:56 CEST - second retry to webserver2 fails
you hit: webserver1
Mi 22 Mai 2019 17:10:01 CEST - the backend (webserver2) was taken out of the pool.
you hit: webserver1
Mi 22 Mai 2019 17:10:02 CEST
you hit: webserver1
Mi 22 Mai 2019 17:10:03 CEST
you hit: webserver1
Mi 22 Mai 2019 17:10:04 CEST
you hit: webserver1
Mi 22 Mai 2019 17:10:05 CEST
you hit: webserver1
Mi 22 Mai 2019 17:10:06 CEST
Monitoring with Prometheus
The Octavia amphora driver provides a Prometheus endpoint. This allows you to collect metrics from Octavia load balancers.
To add a Prometheus endpoint on an existing Octavia load balancer, create a listener with the protocol PROMETHEUS. This will enable the endpoint as /metrics on the listener. The listener supports all the features of an Octavia load balancer, such as allowed_cidrs, but it does not support attaching pools or L7 policies. All metrics are identified by the Octavia object ID (UUID) of the resources.
The OVN provider does not support the Prometheus endpoint.
Note: Currently, UDP and SCTP metrics are not reported via Prometheus endpoints when using the amphora provider.
For example, to create a Prometheus endpoint on port 8088 for load balancer lb1, run the following command:
$ openstack loadbalancer listener create --name stats-listener --protocol PROMETHEUS --protocol-port 8088 lb1
+-----------------------------+--------------------------------------+
| Field | Value |
+-----------------------------+--------------------------------------+
| admin_state_up | True |
| connection_limit | -1 |
| created_at | 2021-10-03T01:44:25 |
| default_pool_id | None |
| default_tls_container_ref | None |
| description | |
| id | fb57d764-470a-4b6b-8820-627452f55b96 |
| insert_headers | None |
| l7policies | |
| loadbalancers | b081ed89-f6f8-48cb-a498-5e12705e2cf9 |
| name | stats-listener |
| operating_status | OFFLINE |
| project_id | 4c1caeee063747f8878f007d1a323b2f |
| protocol | PROMETHEUS |
| protocol_port | 8088 |
| provisioning_status | PENDING_CREATE |
| sni_container_refs | [] |
| timeout_client_data | 50000 |
| timeout_member_connect | 5000 |
| timeout_member_data | 50000 |
| timeout_tcp_inspect | 0 |
| updated_at | None |
| client_ca_tls_container_ref | None |
| client_authentication | NONE |
| client_crl_container_ref | None |
| allowed_cidrs | None |
| tls_ciphers | None |
| tls_versions | None |
| alpn_protocols | None |
| tags | |
+-----------------------------+--------------------------------------+
Once the PROMETHEUS listener is ACTIVE, you can configure Prometheus to collect metrics from the load balancer by updating the prometheus.yml file.
[scrape_configs]
- job_name: 'Octavia LB1'
static_configs:
- targets: ['192.0.2.10:8088']