Do load balancers flood?

Question

I am reading about load balancing.

I understand the idea that load balancers transfer the load among several slave servers of any given app. However very few literature that I can find talks about what happens when the load balancers themselves start struggling with the huge amount of requests, to the point that the "simple" task of load balancing (distribute requests among slaves) becomes an impossible undertaking.

Take for example this picture where you see 3 Load Balancers (LB) and some slave servers.

Figure 1: Clients know one IP to which they connect, one load balancer is behind that IP and will have to handle all those requests, thus that first load balancer is the bottleneck (and the internet connection).

What happens when the first load balancer starts struggling? If I add a new load balancer to side with the first one, I must add even another one so that the clients only need to know one IP. So the dilema continues: I still have only one load balancer receiving all my requests...!

Figure 2: I added one load balancer, but for having clients to know just one IP I had to add another one to centralize the incoming connections, thus ending up with the same bottleneck.

Moreover, my internet connection will also reach its limit of clients it can handle so I probably will want to have my load balancers in remote places to avoid flooding my internet connection. However if I distribute my load balancers, and want to keep my clients knowing just one single IP they have to connect, I still need to have one central load balancer behind that IP carrying all the traffic once again...

How do real world companies like Google and Facebook handle these issues? Can this be done without giving the clients multiple IPs and expect them to choose one at random avoiding every client to connect to the same load balancer, thus flooding us?

Answer 1

您的问题听起来并不特定于 AWS，因此这是一个通用答案（AWS 中的弹性 LB 会根据流量自动缩放）：

你是对的，你可以用传入的请求数量压倒负载均衡器。如果你在标准构建机器上部署 LB，你可能首先耗尽/过载网络堆栈，包括最大打开连接数和处理率的传入连接。

作为第一步，您将微调 LB 机器的网络堆栈。如果这仍然不能为您提供所需的吞吐量，那么市场上有专门的负载平衡器设备，它们是从头开始构建并高度优化的，可以处理大量传入连接并将它们路由到多台服务器。这些示例是 F5 和 netscaler

您还可以以帮助您将流量拆分到不同子域的方式设计您的应用程序，从而减少 1 LB 必须处理的请求数量。

也可以实现循环 DNS，在那里您将有 1 个 DNS 入口点指向多个面向 LB 的客户端，而不是您所描述的只有一个。

Answer 2

像 Netscaler 和类似的高级负载均衡器也使用 DNS 进行 GSLB 而不是简单的 DNS-RR（解释进一步扩展）

如果您要连接到 ie service.domain.com，则让负载均衡器成为该区域的权威 DNS，并将所有负载均衡器添加为有效名称服务器。

当客户端查找“service.domain.com”时，您的任何负载均衡器都会回答 DNS 请求并回复您客户端的正确数据中心的 IP。然后，您可以根据客户端的地理位置、客户端 dns 服务器和 netscaler 之间的延迟进一步对 DNS 请求进行负载均衡器回复，或者您可以根据不同的数据中心负载进行回答。

在每个数据中心，您通常会在集群中设置一个或多个节点。您可以使用这样的设计进行相当高的扩展。