Benefits of load balancing RabbitMQ
The key benefits of load balancing RabbitMQ are:
- High Availability (HA): Load balancing, often in conjunction with a RabbitMQ cluster and mirrored/quorum queues, ensures that client connections are distributed across multiple nodes. If a single node fails, the load balancer automatically directs traffic to the remaining healthy nodes. This prevents a single point of failure and allows the messaging system to continue processing messages without interruption, providing robust fault tolerance and high availability.
- Scalability: Load balancing distributes the incoming connections and message-processing workload evenly across all available RabbitMQ nodes and consumers. This prevents any single broker or consumer from becoming a bottleneck, allowing the system to handle a much higher volume of messages and overall traffic. It facilitates horizontal scaling—you can simply add more nodes to the cluster behind the load balancer to increase capacity and throughput as the application grows.
- Optimal performance: By distributing connections and messages, load balancing ensures that the CPU, memory, and network resources of all nodes in the cluster are used efficiently. This balanced utilization prevents resource exhaustion on any one server, which helps to maintain low latency, predictable performance, and a consistent quality of service for the messaging system.
About RabbitMQ
RabbitMQ is an open source message broker. It uses a publish-subscribe model to route data from publishers to consumers. It is scalable and can be load balanced, acting as a reliable and highly available intermediary.
It has support for management and monitoring and has a range of tools and plugins available.
Why Loadbalancer.org for RabbitMQ?
Loadbalancer’s intuitive Enterprise Application Delivery Controller (ADC) is designed to save time and money with a clever, not complex, WebUI.
Easily configure, deploy, manage, and maintain our Enterprise load balancer, reducing complexity and the risk of human error. For a difference you can see in just minutes.
And with WAF and GSLB included straight out-of-the-box, there’s no hidden costs, so the prices you see on our website are fully transparent.
More on what’s possible with Loadbalancer.org.
How to load balance RabbitMQ
The load balancer can be deployed in four fundamental ways: Layer 4 DR mode, Layer 4 NAT mode, Layer 4 SNAT mode, and Layer 7 Reverse Proxy (Layer 7 SNAT mode).
For RabbitMQ, Layer 4 DR mode is recommended. This mode offers the best possible performance since replies go directly from the Real Servers to the client, not via the load balancer. It’s also relatively simple to implement.
Layer 7 Reverse Proxy is also supported however.
Virtual service (VIP) requirements
To provide load balancing and HA for RabbitMQ, one virtual service is required.
The virtual service must be set to listen on the same port as the RabbitMQ service, which listens on port 5672 by default.
Server feedback agent
It may be useful to adjust how much traffic is passed to the RabbitMQ servers depending on their CPU load.
This can be done by installing the Loadbalancer.org server feedback agent on each RabbitMQ server and then reconfiguring the Virtual Service to make use of the agent. The feedback agent is available for both Linux and Windows servers.
Load balancing deployment concept
About Layer 4 DR mode
Layer 4 DR (Direct Routing) mode is a very high performance solution that requires little change to your existing infrastructure. The image below shows an example network diagram for this mode.
DR mode works by changing the destination MAC address of the incoming packet to match the selected Real Server on the fly which is very fast. When the packet reaches the Real Server it expects the Real Server to own the Virtual Services IP address (VIP). This means that each Real Server (and the load balanced application) must respond to both the Real Server’s own IP address and the VIP. The Real Server should not respond to ARP requests for the VIP. Only the load balancer should do this. Configuring the Real Server in this way is referred to as “Solving the ARP Problem”. For more information please refer to DR Mode Considerations. On average, DR mode is 8 times quicker than NAT mode for HTTP and much faster for other applications such as Remote Desktop Services, streaming media and FTP. The load balancer must have an interface in the same subnet as the Real Servers to ensure layer 2 connectivity which is required for DR mode to operate. The VIP can be brought up on the same subnet as the Real Servers or on a different subnet provided that the load balancer has an interface in that subnet. Port translation is not possible with DR mode, e.g. VIP:80 → RIP:8080 is not supported. DR mode is transparent, i.e. the Real Server will see the source IP address of the client.
