‫In this lecture,
‫we're going to learn about the first
‫and one of the most important
‫software architecture building blocks,
‫which is used in pretty much 100%
‫of all real-life, large-scale systems.
‫This building block is called a load balancer.
‫After getting some motivation for using a load balancer,
‫we will learn what quality attributes
‫this building block can provide to our system.
‫And finally, we will learn about
‫the different types of load balancing solutions
‫and how to use those solutions
‫in architecting a real-life, large-scale system.
‫So let's start with an introduction to load balancers.
‫As the name suggests, the basic role of a load balancer
‫is to balance the traffic load
‫among a group of servers in our system.
‫If we remember from the previous lectures,
‫the best way to achieve high availability
‫and horizontal scalability
‫is running multiple identical instances of our application
‫on multiple computers.
‫However, without a load balancer,
‫the client application
‫that may run on our customer's computers
‫will have to know the addresses of those computers,
‫as well as the number of the application instances.
‫This tightly couples the client application
‫to our system's internal implementation
‫and makes it very hard for us to make any changes.
‫So while the main purpose of a load balancer
‫is to balance the load among a group of servers
‫to make sure that no individual server
‫is overloaded as an added feature,
‫most load balancing solutions also provide an abstraction
‫between the client application and our group of servers.
‫This abstraction makes our entire system
‫look like a single server,
‫capable of immense computing power and a lot of memory.
‫Now, different load balancing solutions
‫offer different levels of abstraction,
‫which we will see very soon
‫when we talk about different types of load balancers.
‫Now, let's talk about what specific quality attributes
‫a load balancer can provide to our system.
‫The first quality attribute we get from a load balancer
‫is high scalability.
‫As we already mentioned, by hiding a group of servers
‫behind a load balancer, we can scale our system horizontally
‫both up and down by adding additional servers
‫when the load on our system increases
‫and remove unnecessary servers
‫when the load on our system decreases to save money.
‫In a cloud environment
‫where we can easily rent more hardware on demand,
‫we can use auto-scaling policies
‫to intelligently add or remove servers
‫based on different criteria,
‫like the number of requests per second,
‫network bandwidth, and so on.
‫The next quality attribute
‫the load balancer provides us with is high availability.
‫Most load balancers can easily be configured
‫to stop sending traffic to servers that cannot be reached.
‫By having this monitoring feature,
‫load balancers can intelligently balance the load
‫only among healthy servers
‫while ignoring the ones that are considered to be dead
‫or excessively slow.
‫Now, let's see how load balancers
‫affect the system's performance.
‫When it comes to performance,
‫load balancers may add a little bit of latency
‫and increase their response time to the user,
‫but it's generally an acceptable price to pay
‫for an increased performance in terms of throughput.
‫Since the load balancer allows us
‫to theoretically have as many backend servers as we like,
‫of course, with some reasonable limitations,
‫the number of requests or tasks
‫that we can perform per unit of time
‫is much larger than the throughput we could get
‫from a single server.
‫Another important quality attribute
‫that the load balancer helps us achieve is maintainability.
‫Since we can easily add or remove servers to the rotation,
‫we can take down individual servers one-by-one
‫to prefer maintenance
‫or upgrade the application version
‫without any disruption to the client.
‫And when the maintenance on that server is complete,
‫we can add it back to the load balancer
‫and take down the next server.
‫This way we can have a rolling release,
‫while still keeping our SLA in terms of availability.
‫Now, finally, let's talk about
‫a few different types of load balancers
‫that we can choose from.
‫One of the most basic load balancers
‫can be achieved through DNS.
‫The Domain Name System
‫is part of the internet infrastructure
‫that maps human-friendly URLs, like amazon.com,
‫netflix.com, or apple.com
‫to IP addresses that can be used by network routers
‫to route requests to individual computers on the web.
‫It's essentially the phone book of the internet.
‫So when a user or a client application
‫wants to communicate with our system,
‫the user sends a DNS query to the DNS server
‫and the DNS responds with an IP address
‫that corresponds to our domain name.
‫Then, the client application can use that IP address
‫to send a request directly to the server.
‫However, a single DNS record doesn't have to be mapped
‫to a single IP address
‫and can be easily configured
‫to return a list of IP addresses
‫corresponding to different servers.
‫Most DNS servers are implemented in such a way
‫that they return the list of addresses for each domain
‫in a different order on each client request
‫and by convention, most client applications
‫simply pick the first address in the list
‫that uses the resolved IP address for a particular domain.
‫This way, the domain naming system
‫essentially balances the load on our servers
‫by simply rotating this list in a round-robin fashion.
‫Now, although this way of providing
‫load balancing capability is super simple and cheap,
‫as it essentially comes for free
‫by purchasing a domain name,
‫it has a few drawbacks.
‫The main drawback is that DNS
‫doesn't monitor the health of our servers.
‫In other words, if one of our servers stops responding,
‫the Domain Name System will not know about it
‫and will continue referring clients
‫to that particular server.
‫This list of IP addresses changes only so often
‫and is based on the time to live
‫that was configured for that particular DNS record.
‫Additionally, this list of addresses
‫that a particular domain name is mapped to
‫can be cached in different locations,
‫such as the client's computer.
‫That makes the time
‫between a particular server going down
‫and the point that the requests
‫are no longer sent to that server even longer.
‫Another drawback of DNS-based load balancing
‫is that the load balancing strategy
‫is always just as simple as round-robin,
‫which doesn't take into account
‫the fact that some of our application instances
‫may be running on more powerful servers than others,
‫nor can it detect that one of our servers
‫may be more overloaded than the others.
‫The third drawback of the DNS-based load balancing
‫is the declined application
‫gets the direct IP addresses of all our servers.
‫This exposes some implementation details of our system
‫and more importantly, makes our system less secure.
‫The reason for that is that there is nothing
‫that prevents a malicious client application
‫from just picking one IP address
‫and send requests only to that particular server
‫which, of course, would overload it more than others.
‫To address all those drawbacks,
‫there are two load balancing solutions
‫that are a lot more powerful and intelligent.
‫Those two types of solutions are hardware load balancers
‫and software load balancers.
‫The only difference
‫between those two types of load balancers
‫is that hardware load balancers run on dedicated devices
‫designed and optimized specifically for load balancing,
‫while software load balancers are just programs
‫that can run on any general-purpose computer
‫and perform the load balancing function.
‫In the case of software and hardware load balancers,
‫in contrast to DNS load balancing,
‫all the communication between the client
‫and our group of servers is done through the load balancer.
‫In other words, the individual IP addresses,
‫as well as the number of servers
‫we have behind the load balancer
‫are not exposed to the users,
‫which makes our system a lot more secure.
‫Another feature of hardware and software load balancers
‫is that they can actively monitor the health of our servers
‫and send them periodic health checks
‫to actively detect if one of our servers
‫became unresponsive.
‫Finally, both hardware and software load balancers
‫can balance the load among our servers
‫a lot more intelligently,
‫taking to account the different types of hardware
‫our application instances are running on,
‫the current load on each server,
‫the number of open connections, and so on.
‫The nice thing about software and hardware load balancers
‫is in addition to balancing requests from external users,
‫they can also be used inside our system
‫to create an abstraction between different services.
‫For example, if we have an online store system,
‫we can separate at the service
‫that responds directly to client requests
‫and serves the front end to the client browsers
‫from the fulfillment service and the billing service.
‫Each such service can be deployed independently
‫as multiple application instances
‫running on a group of servers.
‫And those services communicate with each other
‫through a load balancer.
‫This way, we can scale each such service
‫completely independently
‫and transparently to the other service.
‫While software and hardware load balancers
‫are superior to DNS in terms of load balancing, monitoring,
‫failure recovery and security,
‫they are usually collocated with the group of servers
‫they balance the load on.
‫The reason for that is if we put the load balancer
‫too far from the actual servers,
‫we're adding a lot of extra latency
‫since all the communication,
‫both to the servers and back to the client,
‫has to go through the load balancer.
‫So if we run system in multiple geographical locations,
‫which are commonly referred to as data centers,
‫then having only one load balancer
‫for both groups of servers will sacrifice the performance
‫for at least one of those locations.
‫Additionally, load balancers, on their own,
‫do not solve the DNS resolution problem,
‫so we would still need some kind of DNS solution
‫to map human readable domain names to an IP address.
‫For that, there is a fourth load balancing solution,
‫which is called Global Server Load Balancer,
‫or GSLB in short.
‫A GSLB is somewhat of a hybrid between a DNS service
‫and the hardware or software load balancer.
‫A GSLB solution typically can provide a DNS service
‫just like any other DNS server that we're familiar with.
‫However, in addition,
‫it also can make more intelligent routing decisions.
‫On one hand, the GSLB can figure out the user's location
‫based on the origin IP inside the incoming request.
‫On the other hand,
‫a GSLB service has similar monitoring capabilities
‫to a typical software or hardware load balancer.
‫So at any given moment, it knows the location
‫and state of each server that we register with our GSLB.
‫In a typical large-scale system deployment,
‫those servers are load balancers
‫located in different data centers
‫in different geographical locations.
‫So when a user sends a DNS query to the GSLB,
‫the GSLB may return just the address
‫of the most nearby load balancer.
‫From that point on, the user will use that IP address
‫to communicate directly
‫with our system in that data center
‫through a collocated software or hardware load balancer.
‫The cool part about GSLBs
‫is that most GSLBs can be configured to route traffic
‫based on a variety of strategies
‫and not just by physical location.
‫Since they're in constant communication
‫with our data centers,
‫they can be configured to route users
‫based on their current traffic
‫or CPU load on each data center
‫or based on the best estimated response time
‫or bandwidth between the user
‫and that particular data center.
‫Thanks to this great feature,
‫we can provide the best performance possible for each user,
‫regardless of their geographical location.
‫Additionally, GSLBs play a very important role
‫in disaster recovery situations.
‫If there's a natural disaster
‫or a power outage in one of our data centers,
‫the users can be easily routed to different locations,
‫which provides us with higher availability.
‫Finally, to prevent a load balancer
‫from being a single point of failure in each region,
‫we can place multiple load balancers
‫and register all their addresses
‫with the GSLB's DNS service or any other DNS service.
‫So the client applications
‫can get a list of all our load balancers
‫and either send a request to the first one in the list
‫or pick one themselves randomly.
‫We learned a lot in this lecture,
‫so let's quickly summarize it.
‫In this lecture,
‫we learned about a very important
‫software architecture building block,
‫the load balancer.
‫We learned about four load balancing solutions,
‫which are DNS load balancing,
‫hardware load balancing,
‫software load balancing,
‫and Global Server Load Balancing.
‫Later, we talked about the different quality attributes
‫that the load balancer provides to our system.
‫And finally, we saw how we can combine
‫all those different solutions
‫to architect a large-scale system
‫that can provide high performance and high availability,
‫and scale to millions of users
‫located in different geographical locations.
‫I will see you soon in the next lecture.