A load balancer is a device or piece of software that sits in front of your servers and spreads incoming traffic across them, so no single server takes the full load. When one server gets busy, slow, or fails completely, the load balancer quietly sends visitors to the healthy ones instead. The result is a website or app that stays fast under traffic spikes and stays online when a server dies.
If you run anything that real people depend on, a database-backed app, an online store, an API, load balancing is one of the first pieces of infrastructure you will meet. This guide explains how it works, the types and algorithms you will actually encounter, and how to tell when your site needs one.
How Load Balancing Works
Think of a supermarket with ten checkout counters and one member of staff at the entrance. Instead of letting every shopper pile into the first queue, that person points each shopper to the shortest line. If a till breaks down, nobody is sent there. That person is the load balancer, and the tills are your servers.
Here is what happens on every request:
- A visitor opens your site. Their browser sends a request to your domain name, for example
example.com. - DNS points to the load balancer, not a server. Your domain resolves to the load balancer's IP address, so all traffic arrives there first.
- The load balancer picks a backend server. It uses a routing rule, called an algorithm, to choose one server from the pool. Round robin simply takes them in turn; other algorithms pick the least busy one.
- The request is forwarded and the answer returned. The chosen server processes the request and sends its response back through the load balancer to the visitor. The visitor never sees the backend server's address.
- Health checks run in the background. Every few seconds the load balancer pings each server, usually with a simple HTTP request. A server that stops answering is pulled out of rotation immediately, and added back automatically once it recovers.
Because the load balancer owns the public-facing address, you can add or remove servers behind it without visitors ever noticing. That is what makes scaling and maintenance possible without downtime: spin up a new server, add it to the pool, and traffic starts flowing to it within seconds.
Types of Load Balancers
Layer 4 load balancing (transport layer)
A Layer 4 load balancer routes traffic using only network information: the source and destination IP addresses and the port number. It never looks inside the request itself, so it is extremely fast and works for any TCP or UDP service, not just websites. The trade-off is that it cannot make smart decisions based on what the visitor actually asked for.
Layer 7 load balancing (application layer)
A Layer 7 load balancer understands HTTP. It can read the URL, headers, and cookies, and route accordingly: send /api/ requests to the API servers, image requests to a static-file server, and logged-in users to a specific backend. It can also terminate SSL, compress responses, and cache content. This is the type most websites use, and it is what Nginx, HAProxy, and cloud load balancers like AWS Application Load Balancer do.
Hardware, software, and cloud load balancers
Hardware load balancers are dedicated appliances, fast and expensive, mostly found in large enterprises. Software load balancers such as Nginx and HAProxy run on ordinary servers or virtual machines and handle the vast majority of real-world setups. Cloud load balancers are managed services, AWS Elastic Load Balancing, Google Cloud Load Balancing, Azure Load Balancer, where the provider runs the balancer and you pay per hour and per gigabyte. For most small and mid-size projects, a software load balancer is the right starting point.
Global load balancing (GSLB)
When your users are spread across continents, a single load balancer in one region is not enough. Global load balancing uses DNS to send each visitor to the nearest healthy data centre, for example Europe for a visitor in Berlin and Asia for a visitor in Mumbai. This is how large services keep latency low worldwide.
Load Balancing Algorithms
The algorithm is the rule the load balancer uses to pick a server. The common ones are:
- Round robin: requests go to each server in turn, one after another. Simple and fair when all servers are equally powerful.
- Weighted round robin: the same idea, but stronger servers get a bigger share of the traffic. Useful when your pool mixes old and new hardware.
- Least connections: each new request goes to the server with the fewest active connections right now. This adapts well when some requests are long and slow and others are quick.
- Least response time: the server that has been answering fastest recently gets the next request. Good for latency-sensitive apps.
- IP hash: the visitor's IP address is hashed to always pick the same server. This keeps a user's session on one machine, which is handy if your app stores session data in memory, though it can unbalance the load.
- Random: a server is picked at random. Surprisingly effective at scale, and some systems use a smarter variant that picks two servers at random and sends the request to the less busy one.
For a first setup, round robin or least connections will serve you well. Reach for weighted or IP-hash variants only when you have a concrete reason, such as uneven servers or session stickiness.
A Real Example: Load Balancing with Nginx
Nginx can act as a load balancer with a few lines of configuration. The upstream block defines the pool of backend servers, and proxy_pass sends traffic to it:
upstream backend {
least_conn;
server 10.0.0.11;
server 10.0.0.12;
server 10.0.0.13;
}
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://backend;
}
}
With this in place, Nginx sends each request to the backend with the fewest active connections, and if one of the three servers stops responding, Nginx marks it down and stops sending it traffic. You can test the whole setup by stopping one backend and confirming the site still answers, which is the single most useful check you can run on any load balancer.
Benefits and Real-World Use Cases
- High availability: if one server crashes, users are routed to the rest. A single server is a single point of failure; a balanced pool is not.
- Scalability: need more capacity for a sale or a traffic spike? Add servers to the pool instead of rebuilding one giant machine.
- Zero-downtime deploys: take one server out of rotation, deploy the new code, test it, and put it back. Repeat for the rest. Users never see a maintenance page.
- Better performance under load: traffic spreads evenly, so response times stay stable instead of collapsing when one server saturates.
- Geographic distribution: global load balancing sends users to the nearest region, cutting latency.
- SSL offloading: the load balancer handles encryption, freeing backend servers to do application work.
Typical users are e-commerce stores, SaaS products, APIs with spiky traffic, and any site where a few minutes of downtime costs real money or trust.
Load Balancer vs Reverse Proxy: What Is the Difference?
This is the most common point of confusion. A reverse proxy sits in front of your servers and forwards requests to them, hiding the backends from the internet. It can also cache responses, terminate SSL, and compress content. A load balancer sits in front of your servers and distributes requests between them.
The difference is the job: a reverse proxy's job is forwarding and hiding, a load balancer's job is distributing. In practice the two overlap heavily. Most load balancers, including Nginx and HAProxy, are also reverse proxies, and most reverse proxies can distribute traffic across several backends. If you set up Nginx as a reverse proxy with an upstream block, you have both at once, which is why people use the two terms almost interchangeably.
Common Misconceptions
1. Load balancing makes my site faster
Not by itself. A load balancer adds capacity and resilience, but it does not speed up a single request. If your app takes three seconds to answer because of a slow database query, balancing just spreads those three-second answers across more servers. Fix the slow query first.
2. I need a load balancer from day one
Almost never. A single well-sized server with caching handles far more traffic than most new sites ever see. Add a load balancer when one server genuinely cannot keep up, or when downtime starts costing you. Premature infrastructure is just a second server to maintain.
3. It fixes a crashing application
It hides crashes, it does not fix them. Health checks route around a dead server, which is valuable, but if every server runs the same buggy code, they will all fail the same way. Load balancing is a safety net, not a bug fix.
4. Sticky sessions are harmless
Sending each user to the same server every time sounds convenient, and sometimes it is necessary. But it can quietly unbalance your pool: a few heavy users pinned to one server can overload it while the others sit idle. Prefer stateless app design, with sessions stored in a shared store like Redis, so any server can serve any user.
Do You Need a Load Balancer?
Ask three questions. Does a single server handle your traffic with comfortable headroom? If yes, you do not need one yet. Does a few minutes of downtime cost you money, users, or trust? If yes, a load balancer plus a second server is the cheapest insurance you can buy. Do you deploy often and want to do it without maintenance windows? If yes, balancing makes that routine instead of risky.
When the answer to the second or third question is yes, start simple: two backend servers and Nginx or HAProxy in front. Measure, then scale the pool when the numbers say so.
Conclusion
Load balancing is a simple idea with an outsized payoff: put a smart traffic director in front of several servers, and your site survives traffic spikes and server failures that would take a single machine down. Understand the two layers, pick an algorithm that matches your traffic, and test failure by actually killing a backend. That one test tells you more than any dashboard.
If you need help designing a highly available setup, from a first Nginx load balancer to a multi-region deployment, XpertBees builds and manages server infrastructure for businesses of every size.



