‫Hey, welcome back.
‫In this lecture, we're going to talk about one
‫of the most important quality attributes,
‫"The scalability of the system."
‫As always, we'll start with some motivation
‫and a precise definition for scalability
‫and later we'll learn
‫about the three scalability dimensions.
‫So let's start with some motivation.
‫Generally, the load or the traffic on our system
‫never stays the same and it can follow different patterns.
‫We can have a seasonal traffic pattern
‫such as in the case of retail websites.
‫In this case, we can have a lot of traffic during holidays
‫or sales and lower traffic in between those events.
‫Similarly, if we're a search engine,
‫those spikes would represent a global event
‫where a lot of users suddenly open their browsers
‫and start looking for news regarding this event.
‫We can have completely different traffic patterns
‫even on a daily basis.
‫For example, if we're a news website
‫or some kind of tool that people use at work,
‫we could have different load on our system
‫during the weekdays, comparing to the weekends.
‫And the traffic we have during the days
‫would look very different
‫than the one we have during the night.
‫Generally in the long run, if our business is doing well,
‫we should expect to see more users using our system
‫which should result in an increase in traffic and load.
‫Now, in the previous lecture we talked about performance
‫and one of the considerations was finding the point
‫where our system starts degrading
‫and cannot handle the additional load
‫with the same level of performance.
‫We call that point in a graph,
‫the performance degradation point.
‫Now let's focus on the response time only for a moment
‫and compare two different designs of the same system
‫based on how they perform under different amount of load.
‫Now, let's say that both team one and team two
‫get the same budget, both in terms of time and money
‫to upgrade their systems so they can take more load.
‫So after the upgrades, the graphs look like this.
‫From those graphs, we can see that design number two
‫can perform much better for a higher load
‫than design number one for the same budget and time.
‫So informally, we can say that design number two
‫is a lot more scalable than design number one.
‫Because for the same amount of effort and cost,
‫it was able to achieve much better results.
‫So now that we got some intuition for scalability,
‫let's define it in a more formal way.
‫Scalability is the measure of the system's ability
‫to handle a growing amount of work
‫in an easy and cost effective way
‫by adding resources to the system.
‫So to clarify this definition,
‫if we plot the amount of work our system can perform
‫as the function of our system resources,
‫we have a few options.
‫The most optimistic scenario we can have
‫is full linear scalability.
‫For example, if our system is currently in this spot
‫and we double the amount of resources,
‫we would get double the amount of work
‫or we can take double the amount of load
‫before our system degrades.
‫However in practice,
‫linear scalability is very hard to achieve.
‫So we would have either of the following options
‫where the last option is the least desired
‫as adding more resources to the system
‫will actually give us worse performance.
‫And that can happen because of the overhead involved
‫in managing and coordinating between those resources.
‫Generally, we can scale our system
‫on three orthogonal dimensions.
‫We can scale our system up
‫which is also referred to as vertical scalability.
‫We can scale our system out
‫which is also called horizontal scalability.
‫And on the third dimension,
‫we have the team or organizational scalability.
‫So let's start talking about the first dimension
‫which is vertical scalability.
‫Let's assume we have a service running on a single computer
‫which is taking traffic from many users.
‫As the number of users grows,
‫at some point our system cannot take any more load
‫and reaches the point of degradation.
‫One way we can solve it is simply upgrading
‫our service to a faster and newer machine
‫that has a faster CPU, more memory,
‫and maybe even a network card
‫that can handle higher bandwidth.
‫Similarly, we can upgrade the database
‫by putting it on a stronger hardware
‫with more storage capacity.
‫This type of upgrade is called scaling up
‫or is commonly referred to as vertical scaling.
‫Essentially, vertical scalability means adding more
‫resources or upgrading the existing resources
‫on a single computer.
‫All with the purpose to allow our system
‫to handle higher traffic or load.
‫Now, let's talk about the second type of scalability.
‫Horizontal Scalability, in contrast to vertical scalability,
‫instead of upgrading the existing hardware
‫we can simply add more units of the resources
‫that we already have, like running our service
‫on multiple computers instead of on a single computer.
‫This allows us to spread the load among a group of machines
‫keeping each individual machine
‫far from its degradation point.
‫Similarly, we can scale our database by running it
‫on multiple machines instead of on a single machine.
‫This will release the performance pressure
‫from any individual computer.
‫So more formally, horizontal scalability
‫means adding more resources
‫in a form of new instances running on different machines
‫all with the same intent to allow our system
‫to handle higher traffic or load.
‫Now, before we move on to the third scalability dimension,
‫let's compare vertical scalability
‫to horizontal scalability.
‫The pros of scaling the system vertically
‫is that pretty much any application can benefit from it
‫without any additional code changes.
‫For example, if the bottleneck that brings our application
‫to its degradation point is the high CPU utilization,
‫then upgrading it to a faster CPU will solve that problem
‫without any special changes.
‫Also with vertical scalability,
‫the migration is fairly simple.
‫When the traffic is high,
‫we can simply migrate our application to a stronger machine.
‫And when the traffic is low,
‫we can move our application instance
‫to a weaker machine and save money.
‫This is particularly easy if we're renting our hardware
‫from a Cloud service provider.
‫The major downside of scaling our system vertically,
‫is that there's a limit
‫on how much we can upgrade our hardware.
‫And when we're building a system on an internet scale,
‫we reach that limit very fast
‫and there's nowhere to go beyond that point.
‫The second, and probably even bigger issue
‫with scaling our system vertically only
‫is that we're locking ourselves
‫to an inherently centralized system.
‫And a centralized system cannot provide us
‫with high availability and full tolerance
‫which are two extremely important quality attributes
‫which we're going to discuss in the following lectures.
‫Now in comparison, when we scale our system horizontally,
‫there is virtually no limit to our ability to scale
‫and that's because we can easily add more
‫and more machines as needed.
‫And we can also remove them just as easily
‫when the load on our system goes down.
‫And finally, when we scale our system horizontally
‫if we design the system correctly,
‫our system can provide us with high availability
‫and fault tolerance out of the box.
‫On the other hand, not every application
‫can be easily ported to run
‫as multiple instances on different computers.
‫So potentially we would need
‫to make significant changes to our code.
‫However, once we do make those changes once,
‫then adding or removing resources
‫shouldn't require any changes.
‫Finally, the biggest disadvantage of horizontal scalability
‫in running a group of instances of our application
‫is the increased complexity and coordination overhead.
‫We will talk about some of those challenges
‫and how to solve them during the course.
‫So now that we covered both vertical
‫and horizontal scalability
‫and compared them to each other,
‫let's talk about the last but definitely not least
‫scalability dimension which is team
‫or organizational scalability.
‫If we look back at the definition of scalability,
‫and we look at this not from a client's perspective
‫but from a developer's perspective,
‫the work that we need to do on the system
‫is adding new features, testing, fixing bugs,
‫deploying releases, and so on.
‫So to increase that type of work for our system
‫the resources that we can add is more engineers.
‫Now, let's assume that we have a group of engineers
‫working on one big monolithic code base.
‫It turns out that if we plot the productivity
‫as the function of the number of engineers in the team,
‫before we reach a certain point,
‫the more engineers we have
‫in the team, the more work we get done
‫which means our productivity increases.
‫However, at some point the more engineers we add to the team
‫the less productivity we actually get.
‫Intuitively, if a team of many engineers work
‫on the same big code base,
‫we can think of many reasons
‫why we see such a degradation and productivity
‫as we hire more engineers to the team.
‫For example, now meetings become a lot more frequent
‫and a lot more crowded
‫which makes them a lot less productive.
‫Also because there are a lot
‫of developers working on many features simultaneously,
‫Code merge conflicts are inevitable
‫which of course adds a lot of overhead.
‫Essentially what happens,
‫is everyone starts stepping on each other toes.
‫Also as the code base grows with the number of developers,
‫each new developer who joins the team
‫has a lot more to learn before they become productive.
‫Also testing becomes very hard and slow.
‫The reason for that, is there is basically no isolation.
‫So every minor change can potentially break something.
‫So every time we make any change to the code base,
‫we need to test everything.
‫And finally releases also become very risky
‫because each release includes a lot of new changes
‫that came from many engineers.
‫As a response to that, a lot of companies start making
‫releases even less frequent
‫which actually makes things even worse
‫because now every release contains even more changes.
‫So believe it or not, our software architecture
‫impacts not only the system performance
‫but also the ability to scale our team
‫or increase the engineering velocity.
‫For example, if we split our code base into separate modules
‫or abstract, the way pieces into separate libraries
‫then separate groups of developers can work
‫on each individual module independently
‫and not interfere with each other as much.
‫However, even with the separation
‫into modules and libraries,
‫all those pieces are still part of the same service
‫which means that there's still very tightly coupled
‫especially when it comes
‫to releasing new versions to production.
‫So the next step, is to separate our code base
‫into separate services.
‫Each service has its own code base,
‫its own stack and its own release schedule.
‫And the services are communicating with each other
‫through loosely coupled protocols over the network.
‫This breakdown into multiple services allows
‫for much better engineering productivity
‫and helps us scale our organization.
‫Of course breaking our monolithic code base
‫into multiple services, doesn't come for free.
‫But we'll talk about it in much greater detail
‫when we get to the topic or of architectural patterns.
‫As a final note, as this chart suggests
‫vertical scalability, horizontal scalability,
‫and team scalability are orthogonal to each other
‫which means we can scale our system on either one dimension,
‫two dimensions or all three dimensions.
‫In this lecture, we talked about
‫a very important quality attribute of the system
‫which is scalability.
‫We first got some motivation based on the fact
‫that the traffic and the load on our system
‫is never constant and it can increase over time
‫as well as on a seasonal or sporadic pattern.
‫After that, we define scalability more formally
‫as the measure of system's ability
‫to handle a growing amount of work
‫in an easy and cost effective way
‫by adding more resources to the system.
‫Later, we went ahead and learned about
‫the three orthogonal ways we can scale our system
‫in terms of performance and productivity.
‫The first way was the vertical scalability
‫which we achieved by adding resources
‫or upgrading the existing resources on a single computer.
‫After that, we talked about the horizontal scalability
‫which can be achieved by adding more resources
‫in a form of new instances running on different computers.
‫And finally, we talked about
‫the third scalability dimension
‫which is theme or organizational scalability.
‫Unlike the first two that are focused mainly on performance,
‫this scalability allows our company
‫to keep increasing its productivity
‫while hiring more engineers into the team.
‫I will see you soon in the next lecture.