‫Welcome back.
‫Now after capturing all of our functional and non-functional requirements and clearly defining our API,
‫we are in a perfect spot to start defining our software architecture.
‫So in this lecture we will translate those requirements and API into an architecture diagram that takes
‫care of all the features we need to deliver to our clients.
‫So as a reminder, this is the full list of functional requirements we captured in the previous lecture.
‫So in this first stage, we will ignore the required quality attributes and focus only on addressing
‫those functional requirements.
‫First, we know this is a web based forum, and all the interactions with our system come from the user's
‫web browser.
‫So we need a web app service to serve all the static and dynamic content to the user's devices.
‫Now let's take the first requirement to allow users to sign up and log in.
‫Since we're designing for scale, we'll follow a service or microservice architecture style and create
‫a user service behind that service.
‫We will have a database to store the user's account information.
‫This can be a simple SQL database where the user's data will likely be stored in a table along with
‫a secure representation of their passwords.
‫This takes care of the first functional requirement and all the API calls to sign up and log in will
‫go to the user service.
‫Now let's move on to the second requirement, which is the ability to create a post.
‫For that, we will create another service called the Post Service and we'll store the posts in a database.
‫In this case, we can use a SQL or no SQL database.
‫But since we are architecting for scale and want to keep the structure of a post open ended, we'll
‫go with a no SQL database.
‫This way we can store the topic tags as an array together with the post itself.
‫In addition, we need a place to store the images in that post.
‫So we'll add an object store or a blob store for that purpose.
‫So when a user creates a new post, they can first upload their image.
‫Then the UI will reference that image URL in the post body markdown, which will be sent to the post
‫service and stored in the database.
‫Similarly, when a user wants to view a post, the request will go to the post service.
‫Now, once the post is loaded to the web browser on the user's device, the browser will send another
‫request directly to the object store to load the actual image.
‫We can send that request directly to the object store because the images are publicly available and
‫the object store already exposes a rest API.
‫Finally, the delete posts request will simply go to the post service which will delete that post from
‫the database.
‫Given that the logged in user ID matches the ID of the user who created the post.
‫This fully fulfills the second functional requirement and partially fulfills the fifth functional requirement.
‫Now let's move on to the next requirement, which is allowing users to comment on existing posts.
‫Now, one option is to keep our microservices architecture granular and create a new dedicated microservice
‫for comments and store them in their own database.
‫However, comments are very similar in structure to posts, except they're a lot simpler.
‫So to keep things simple, let's combine them into one microservice making loading a post with its comments
‫easier since they are in the same database.
‫To store the comments, we would need to create a new collection where each record will have the following
‫fields.
‫A unique comment id the comment text, a reference to the user ID who wrote the comment and the post
‫ID to which the comment belongs.
‫To keep comments in the same order as posted, we can ensure that the comment IDs are monotonically
‫increasing or in addition, we can also add a comment timestamp.
‫So now all the API calls to add a comment to a post, getting a list of comments for a post or deleting
‫an existing comment will go to this service.
‫This satisfies the third, the fourth and the fifth functional requirements.
‫Now let's talk about voting or more accurately, upvoting or downvoting a post or a comment.
‫At first glance, this problem may seem very simple.
‫All we need to do is maintain a counter for each post or comment and allow users to increment or decrement
‫this counter concurrently.
‫But if we stop and think about it for a moment, this simple approach is not enough.
‫We need to ensure that each user can upvote or downvote a given post or comment only once, and if we
‫only maintain a counter, we won't be able to enforce it.
‫Additionally, in our last functional requirement, we need to be able to count the votes for each post
‫in a 24 hour window.
‫And that is a sliding window, meaning every second that passes, this 24 hour window moves forward,
‫leaving behind all their votes.
‫So alternatively we can create a separate voting microservice with a separate database.
‫This database will have a collection of votes where each vote contains the votes, user ID, the post
‫ID or comment ID, and the vote where a plus one can be an up vote and a minus one can be a down vote.
‫Additionally, we will store a timestamp of when the user casts or updates that vote.
‫This timestamp will help us narrow down the relevant votes for the feed of popular posts.
‫Of course, counting this may be an issue, but we will take care of that later when we talk about performance.
‫So now whenever a user wants to vote, that request will go to that microservice.
‫And if that user already voted for that entity, that vote will get rejected.
‫Now let's address the last functional requirement, which is the newsfeed that shows a user the most
‫popular posts based on their voting score in the last 24 hours.
‫We need to make a few observations to design this part of the system.
‫First, the information about the popularity of each post is stored in the voting service database,
‫but the posts themselves are stored in the post and comment service database.
‫So to efficiently join the data from both microservices, we will use a variation of the architecture
‫pattern.
‫Using this pattern, we pull data from both microservices by a new microservice, which we will call
‫the ranking service.
‫The service will have a dedicated read optimized database to store the posts in a sorted and ready to
‫be served way.
‫The second observation is that most posts are not active in the last 24 hours and we don't need to care
‫about those posts.
‫So we need to sort only a small number of posts.
‫The next observation, which we already briefly mentioned, is this 24 hour window is constantly moving
‫but continuously recalculating and ranking the posts is a very expensive operation.
‫At the same time, there is no strict requirement to keep this list accurately sorted all the time.
‫So all of this screams batch processing.
‫So combining the batch processing and the access pattern, the ranking service will run maybe once in
‫ten minutes, 30 minutes or even maybe once an hour.
‫And every time it runs, it will request the voting service to give it all the new votes that happened
‫in the last 24 hours.
‫The ranking service will then sum up the upvotes and downvotes per post and sort the posts by popularity.
‫Then it will pull the content of those posts from the posts and comments, service and store the sorted
‫list of posts in its database.
‫And now whenever a user opens our website, that request will go to the ranking service and send back
‫the top 20 most popular posts.
‫And if the user wants the next 20, it would be another get request with an offset which would load
‫the next batch of posts from the database.
‫And that's it.
‫We are done addressing all the functional requirements for the system and we also mapped every API call
‫to a path.
‫It would take within our architecture diagram.
‫So in the next and final step, we will address all the non-functional requirements and ensure that
‫our system has all the desired quality attributes such as scalability, performance, high availability
‫and durability.
‫I'll see you all in the next lecture.