‫Welcome back.
‫In the previous lecture, we finished a crucial part of our design phase by capturing our system's functional
‫requirements.
‫We use the sequence diagram to ensure we hadn't missed any requirements in the diagram.
‫We visualized the interactions between the merchants in our system, the users or potential buyers in
‫our system, and the users that place products in their shopping cart and are now checking out by placing
‫an order with our system.
‫We also captured our nonfunctional requirements that dictate our quality attributes for the merchants
‫and the users.
‫So with that, let's go ahead and start addressing the functional requirements with an architecture
‫diagram, starting with the merchant experience.
‫As a reminder, this is the list of requirements we have for the merchants.
‫So let's start with the first requirement, which is providing a web based product management system
‫for creating new products and making product and inventory updates.
‫For that, we will create a product management service to provide the merchants with a web based UI.
‫The service will allow a new merchant to register and for an existing merchant to log in to our system
‫to store the information about the merchants.
‫The service will have a SQL database where we securely store the merchants information and their credentials.
‫We will create a product service to allow a logged in merchant to add a new product or update an existing
‫product.
‫The service will store all the information about the product, such as the product title description,
‫price, additional attributes and links to images in its own No SQL database.
‫We choose a no SQL document store because different types of products can have different optional attributes.
‫Now we also need to store the product images somewhere, so we'll add an object store for that purpose.
‫Now, besides the product service, we need a place to store, read and update our inventory for each
‫product.
‫For that we will create the inventory service.
‫That service will store the current inventory for each product ID in a high performance key value store.
‫So now a merchant can log in at any time and both view and update their product inventory.
‫Now let's skip the second functional requirement for the merchants to the end of the design because
‫it will need data from other services that we haven't yet created.
‫Moving on to the first functional requirement for users, we now need to create a storefront.
‫For the store front.
‫We have two separate UI's, one in the browser and another on a mobile app for the browser.
‫We need to create a storefront web service to serve and render HTML pages and return them to the user.
‫We don't need to serve any pages for the mobile app users, but we do need a service that can take API
‫calls and send them to the appropriate services.
‫This is the perfect use case for an API gateway.
‫Now all the requests will go to the API gateway regardless of their origin and based on the type of
‫request or type of device that the request originated from, we send the appropriate request to the
‫relevant service.
‫Now, when a user opens our storefront web app or mobile app, we need to show them a list of products
‫and allow them to search for products.
‫We could send all those requests to the product service.
‫However, we have conflicting requirements for the merchants and the users regarding products.
‫Merchants would mainly interact with the products data to update the product information, which is
‫a write intensive workload.
‫On the other hand, users will only read product data, which means it's a read intensive workload.
‫For merchants, we need to guarantee consistency so they can always see the most up to date product
‫information or none.
‫On the other hand, it's okay if users don't see the most up to date product data as long as they can
‫see something.
‫So we prefer availability over consistency for our database configuration.
‫Finally, we don't want to have any contention between merchants updating the products and the millions
‫of users reading product data.
‫So this is the perfect place to use the secure rest pattern.
‫For that, we will create a separate service called the Product Search Service, which will receive
‫product updates from the product service through a message broker.
‫The service will store the product information in a read optimized database that favors availability
‫over consistency.
‫We will also use a no SQL database technology with an efficient text search engine.
‫This way, if we create an index on the products title description and categories, we will ensure very
‫fast responses to product search queries coming from the user.
‫Now, every time a storefront user either loads our home page or performs a product search, that request
‫will go to the product search service, which already has the data organized and the index for such
‫queries.
‫And to load the product images, the front end or mobile app code will make the appropriate calls to
‫our object store.
‫So that satisfies all the functional requirements for the user storefront.
‫Now let's move on to the second functional requirement for the users, which is a store checkout.
‫Now, since the shopping cart is 100% client side, we don't need to care about the products in the
‫shopping cart until the user navigates to the checkout page on our store website or mobile app.
‫Once they do, we want to give them a complete breakdown of their payments due, including taxes that
‫they will pay once they submit the payment form.
‫So first, let's add a new taxi service with a dedicated database with all the tax related data.
‫The service will calculate the taxes based on the product price, its type, and the user's location,
‫which can be derived from the user's IP address.
‫Notice that since now we do care about consistency, we are going directly to the source of truth for
‫the products pricing data, which is the product service.
‫Now, to take care of the user's order, we need to add the following services the order service, payment
‫service and shipping service.
‫Once the user fills out their shipping and payment information, our request will go to our order service.
‫Now, at this point, there are many ways we can go about the checkout process.
‫One way to do this is to hold the connection to the user's device until the inventory is updated, the
‫payment is confirmed and the shipping is scheduled.
‫However, this approach has one major issue and that is the dependency on a timely response from our
‫external services.
‫This poses both a performance and availability issue.
‫Since we don't have direct control over those companies, so the approach we will take is to send the
‫user confirmation as soon as the order service receives a response only from the inventory service and
‫we will perform the billing and the shipping asynchronously one after the other.
‫The downside of this approach, of course, is if there is a payment issue or the product cannot be
‫shipped to the given address, the user will find out about it only after the fact via an email or a
‫push notification.
‫But that is a tradeoff that we need to make.
‫Now we need to think about how to store the order data.
‫One option is to store the order in a dedicated database and override that record with every update
‫we make to the order.
‫However, this will not allow us to have a complete audit about what and when each update to the order
‫happened.
‫So instead we will use the event sourcing pattern.
‫By using this pattern, we get the following benefits.
‫First, we can now add a notification service that can subscribe to that same topic and for certain
‫updates to the order.
‫It can send an email or a push notification to the user's device.
‫The second benefit is order recovery.
‫Let's assume that the order service successfully billed the user.
‫However, before updating the shipping service, the Order service instance crashed.
‫To deal with that situation.
‫We can add another service called Orders Recovery since every update is stored within the message broker.
‫Along with its timestamp, the Orders Recovery Service can subscribe to the topic and watch every currently
‫in flight order.
‫And if it detects an active order that hasn't been sent to the shipping service within a certain reasonable
‫timeframe, it can send a message to the order service to continue processing the order from where it
‫left off.
‫Now, to keep the user fully up to date with their order event after the billing is complete, we can
‫also expose an API endpoint to the delivery system we are using.
‫This way they can send us updates which we can propagate to the user using the notification service
‫until the final delivery.
‫And that's it.
‫Our checkout user experience is now complete.
‫Now, let's go ahead and address the last functional requirement, which is providing analytics for
‫our merchants.
‫As a reminder, we need to provide the merchants with analytics about the current number of users viewing
‫their products page, as well as historical and projected data about their products performance.
‫To provide that data, we need to collect many data points from several services.
‫For example, to get data about their product page visits.
‫We need to get data from the product search service and to get data about their product sales.
‫We need to get data from the order service.
‫Additionally, we need to have real time data to show the merchant the current number of visitors to
‫their product page.
‫On the other hand, to provide historical and projected trends, we need to fuse multiple sources of
‫data aggregated over time.
‫So this is the perfect use case for the LAMDA architecture.
‫To achieve this.
‫Anytime a product is viewed by a user, the product search service will send an event to a special topic
‫or channel in a message broker.
‫Similarly, the order service will publish an event on another topic whenever an order purchase is complete.
‫On the receiving end of those events will have an analytics lambda architecture with two processing
‫layers.
‫The speed layer for the real time data and the batch layer that runs on a schedule.
‫Now, whenever the merchant wants to know how many users are currently visiting their product page,
‫the Product management service will use the speed layer.
‫And whenever the merchant wants to see the views and historical data and projected trends, it will
‫query the data inside the batch layer.
‫And that's it.
‫This addressed all of our functional requirements and already fulfills most of our nonfunctional requirements.
‫So in the next and final lecture, we will wrap up our architecture and address all the nonfunctional
‫requirements that the current diagram doesn't consider.
‫But before we do that, let's quickly recap some key points we already accomplished in this lecture.
‫To support hundreds of merchants selling thousands of products on our platform and millions of users
‫shopping on our platform.
‫We use the microservices architecture that allows us to distribute the responsibilities for each experience
‫across different codebases and engineering teams.
‫When faced with conflicting consistency, performance and workload requirements between merchants and
‫storefront users, we use the six pattern separating the product service from the products search service.
‫Additionally, we chose a no SQL database with a dedicated text search engine to support efficient product
‫search for the user's checkout experience.
‫We made a tradeoff between synchronous and asynchronous billing and scheduling the order shipment and
‫chose the latter for better performance and higher availability.
‫We also use the event sourcing pattern to store our order status events for failure, recovery, auditing
‫and better organizational scalability.
‫Since the notification service could simply subscribe to those events without needing the order service
‫to know about it.
‫And finally, we use the Lambda architecture for the merchant Analytics.
‫This provided the merchants with both real time data and a batch processed, historical and projected
‫view coming from multiple sources of events.
‫I'll see you all in the next lecture.