Design Facebook | System Design

Designing a system as complex and globally impactful as Facebook requires careful consideration of various factors, including scalability, reliability, and performance. This article explores the key components and architectural decisions involved in designing Facebook, focusing on its core functionalities and the challenges of supporting billions of users worldwide.

1. Requirements Gathering for Facebook System Design

Functional Requirements for Facebook System Design

• User Authentication: Implement secure methods for users to log in, such as using OAuth 2.0 or JWT.
• Profile Management: Allow users to personalize their profiles with information like a bio, location, work/education details, and more.
• News Feed: Use algorithms to deliver personalized content to users based on their interests and interactions.
• Posting: Support different types of posts, such as text, photos, videos, events, and polls.
• Social Interactions: Enable features like tagging friends, reacting to posts with emojis, and sharing content to other platforms.
• Notifications: Implement real-time push notifications, so users can receive updates instantly.
• Search: Use indexing and search algorithms to provide fast and accurate search results.
• Privacy Settings: Offer users granular control over their privacy settings, allowing them to choose who can see their posts, friend lists, and profile.

Non-functional Requirements for Facebook System Design

• Security: Ensure that user authentication methods are secure and protect user data from unauthorized access.
• Performance: Ensure that the system can handle a large number of concurrent users and deliver content quickly.
• Scalability: Design the system to easily scale as the user base grows.
• Usability: Ensure that the user interface is intuitive and easy to use, even for users who are not tech-savvy.
• Reliability: Ensure that the system is reliable and available, with minimal downtime or disruptions.
• Real-time: Ensure that notifications are delivered to users in real time without significant delay.
• Accuracy: Ensure that search results are accurate and relevant to the user’s query.
• Customizability: Allow users to customize their experience, such as choosing preferences for their news feed or notifications.

2. Capacity Estimation for Facebook System Design

1. User Base

Facebook has over 2.8 billion monthly active users as of January 2022. Plan for scalability to accommodate potential future growth in user numbers.

2. Traffic Estimation

• Understanding Traffic Patterns
  • To estimate traffic capacity, it is important to analyze historical data to identify peak usage times.
  • For example, let’s say we look at the data from December 2023 and find that there were approximately 16.6 billion visits during that month, with an average of around 536 million visits per day.
  • This information can help us understand the typical traffic patterns and identify any trends.
• Predicting Future Traffic
  • To forecast future traffic, we can employ predictive modeling techniques.
  • This involves analyzing historical data, considering user growth rates, and taking into account upcoming events.
  • For instance, we can incorporate factors like seasonal variations and major events that may impact user activity.
  • By doing so, we can make informed predictions about future traffic levels.
• Handling Peak Loads
  • During special events and holidays, it is important to plan server capacity to handle spikes in traffic.
  • For example, when there are major news events or holidays, the traffic may surge beyond the average daily visits.
  • In such cases, additional server resources may be required to ensure smooth performance and user experience.
• Scalability
  • To accommodate traffic fluctuations, it is crucial to design the system to scale horizontally.
  • This means implementing load balancing techniques and auto-scaling mechanisms that can dynamically adjust to varying loads while maintaining optimal performance.
  • Think of it like a highway system that can add more lanes during rush hour to handle increased traffic.

3. Storage Estimation

• 1. Managing Data Growth
  • When estimating storage capacity, we need to consider the rate at which data is generated.
  • For example, Facebook generates a staggering 4 petabytes of data per day, which is equivalent to around 4 million gigabytes.
  • This daily data generation rate is important for determining storage requirements.
• 2. Current Storage Capacity
  • To plan for storage needs, it is essential to assess the current storage capacity.
  • In the case of Facebook, their storage system called “The Hive” can hold about 300 petabytes of data. By understanding the current capacity, we can determine if there is a need for future expansion to accommodate growing data volumes.
• 3. Redundancy and Disaster Recovery:
  • To mitigate the risk of data loss, it is crucial to implement data replication techniques.
  • This involves replicating data across multiple servers or data centers.
  • By doing so, we ensure redundancy and availability in case of hardware failures or disasters. Think of it like making copies of important documents and keeping them in different secure locations.
• 4. Improving Performance and Scalability
  • To manage growing volumes of user-generated content, it is important to use scalable storage solutions.
  • One approach is to use sharding techniques, which involve distributing data across multiple servers. This improves performance and scalability by partitioning data into smaller chunks and distributing them across servers.
  • It’s like dividing a large task into smaller parts and assigning them to different team members for faster completion.
• 5. Continuous Monitoring and Planning
  • Regularly monitoring storage usage and growth trends is essential to anticipate future storage needs.
  • As user-generated content continues to grow over time, it is important to plan for additional storage capacity.
  • By staying proactive and planning ahead, we can ensure that the storage infrastructure can handle the increasing demands.

3. HLD (High-Level Design) for Facebook System Design

1. Web Browser and Mobile App

These are the two primary interfaces through which users interact with the Facebook of Things platform. Users can access the platform either through a web browser on their desktop/laptop or through a dedicated mobile app on their smartphones or tablets.

2. Authentication

Before accessing any features or services, users are required to authenticate themselves. This step ensures that only authorized users can access their accounts and the associated functionalities.

3. Core Services

• User Profile Service: This service manages and stores user profile information, including personal details, preferences, and settings.
• News Feed Service: Responsible for aggregating and delivering personalized news feeds to users based on their interests, social connections, and activity.
• Messaging Service: Facilitates real-time communication between users through text, voice, and video messaging features.

4. Data Storage

• User Data: Stores various types of user-generated data, such as profile information, posts, comments, likes, and shares.
• Media Storage: Handles the storage of multimedia content uploaded by users, including photos, videos, and audio files.
• Message Database: Stores chat history and message conversations exchanged between users.

5. Compute

Content Delivery Network (CDN): The CDN optimizes the delivery of static and dynamic content, such as images, videos, and other media files, to users across different geographical locations. It helps improve the performance and reliability of content delivery by caching data on edge servers located closer to users.

4. LLD (Low-Level Design) for Facebook System Design

1. User Interaction

Users interact with the platform through the Mobile App and Web Frontend.

2. Load Balancer (Nginx)

• Nginx, serving as the Main Load Balancer, efficiently distributes incoming traffic from the Mobile App and Web Frontend across various servers to ensure optimal performance and reliability.
• Nginx also connects with specific services, including the Authentication Service, User Profile Service, Like Service, Comment Service, Content Delivery Service, and News Feed Service, to manage incoming requests and distribute them accordingly.

3. Authentication Service

Handles user authentication and verification of login credentials. Connected to the Authentication Database for storing user authentication data.

4. User Profile Service

Manages user profiles, preferences, and account-related information. Connected to the User Profile Database for storing user profile data.

5. Like Service

Records user likes on posts or content. Connected to the Like Database for storing like actions.

6. Comment Service

Manages user comments on posts or content. Connected to the Comment Database for storing comments.

7. Content Delivery Service

Delivers content efficiently and reliably to users. Retrieves content from the Content Delivery Database for serving to users. Connected to the Content Cache for optimizing content delivery performance.

8. News Feed Service

Aggregates and delivers personalized content to users. Retrieves and organizes content from the Content Delivery Database. Connected to the News Feed Cache for quick retrieval and responsiveness of personalized news feed content.

9. Databases

• Authentication Database: Stores user authentication data, including usernames, passwords, and session tokens.
• User Profile Database: Stores user profiles, preferences, and account-related information. This can be stored in a NoSQL database like MongoDB or Cassandra.
• Like Database: Stores records of user likes on posts or content, including the relationships between users and liked content.
• Comment Database: Stores user comments on posts or content, including metadata such as timestamps and user identifiers.
• Content Delivery Database: Stores content items and metadata for efficient delivery to users, including media files, text content, and associated information.

10. Message Queues

• Content Update Queue:
  • Facilitates timely updates across the platform by managing the flow of content updates.
  • Connected to the Content Delivery Service for processing and distribution of content updates.
• User Activity Queue:
  • Manages user activity data, such as likes, comments, and posts.
  • Connected to relevant services like the Like Service, Comment Service, and others for processing user interactions and updates.

11. Caching

• Caching mechanisms, such as the Content Cache and News Feed Cache, optimize performance by storing frequently accessed data.
• The Content Cache reduces the need to fetch content from the Content Delivery Database repeatedly, improving response times for users.
• The News Feed Cache ensures quick retrieval and responsiveness for personalized news feed content.
• The caching system works in tandem with the Load Balancer and Services to enhance overall platform performance and user experience.

5. Database Design for Facebook System Design

For designing a Facebook-like application, you can follow a database design that incorporates both relational and NoSQL databases, along with caching mechanisms.

1. Relational Database:

• User Profiles: Store user information such as name, email, password hash, date of birth, etc.
• Friendships: Store relationships between users. This could be a table with columns like “user_id”, “friend_id”, and “status” to represent friend requests, accepted friendships, etc.
• Notifications: Store notifications for users such as friend requests, likes, comments, etc. This table could have columns like “user_id”, “notification_type”, “created_at”, etc.

2. Non-Relational Database(No SQL):

• Posts: Store user posts along with associated metadata like timestamps, location, privacy settings, etc. This can be stored in a NoSQL database like MongoDB or Cassandra.
• Comments: Store comments on posts with information about the commenter, timestamp, etc.
• Reactions: Store reactions (likes, loves, etc.) on posts.

3. Caching:

• Use Redis or similar caching mechanisms to store frequently accessed data such as user sessions, frequently accessed user profiles, recent posts, etc. This helps reduce the load on the main databases and improves performance.

6. Scalability for Facebook System Design

• Auto-scaling: Configure the system to automatically adjust server capacity based on the workload, ensuring that there are enough resources to handle increasing user demand. It’s like having a flexible workforce that can grow or shrink depending on the number of customers.
• Database Sharding: Distribute data across multiple servers to improve performance and handle larger amounts of data. It’s like dividing a large task among multiple people to get it done faster.
• Content Delivery Network (CDN): Cache and serve static assets like images and videos from servers located around the world, reducing the load on your servers and improving content delivery speed. It’s like having local warehouses in different countries to deliver products faster to customers.

7. APIs Used for Facebook System Design

1. User Service API

• Authentication: Allows users to log in and obtain authentication tokens.
• Profile Management: Enables users to create, update, and delete their profiles.
• Friend Requests: Provides endpoints for sending, accepting, rejecting, and managing friend requests between users.

2. Feed Service API

• Personalized Feeds: Exposes endpoints to fetch personalized news feeds based on user preferences and interactions.
• Feed Management: Allows users to customize their feed preferences and interact with feed items (like, comment, share, etc.).

3. Notification Service API

• Real-time Notifications: Provides endpoints for sending and receiving real-time notifications about user interactions such as likes, comments, friend requests, etc.
• Notification Settings: Allows users to manage their notification preferences.

4. Search Service API

• Search Functionality: Exposes endpoints for performing searches across user profiles, posts, and pages.
• Search Filters: Allows users to filter search results based on various criteria such as relevance, time, popularity, etc.

5. Messaging Service API

• Real-time Messaging: Provides endpoints for sending and receiving real-time messages between users.
• Message Management: Enables users to manage their messages, including sending, deleting, archiving, etc.

8. Components Details in Facebook System Design

Facebook design consists of several key components that work together to create a seamless and engaging user experience. These components include the layout, color scheme, typography, icons, and imagery.

1. Load Balancer

A load balancer is like a traffic cop that directs incoming visitors to different servers to make sure they are evenly distributed and not overwhelmed. It’s like having multiple cash registers open at a store to prevent long lines and delays. Some popular load balancing technologies include Nginx, HAProxy, and AWS Elastic Load Balancer (ELB).

2. API Gateway

An API gateway is like a front door that provides a single entry point for client applications to access backend services and APIs.

• It handles important tasks like routing requests to the right place, authenticating users, and limiting the number of requests per second.
• Think of it as a receptionist who directs visitors to the right offices and makes sure they follow the rules. Popular API gateway technologies include Kong, Amazon API Gateway, and Apigee.

3. Content Delivery Network (CDN)

A CDN is like having storage units spread out in different neighborhoods, so you can quickly access your belongings without traveling far.

• It caches and delivers content, such as images and videos, from servers located closer to users’ geographical locations.
• This reduces the time it takes to load content and improves the overall experience. Popular CDN technologies include Cloudflare, Akamai, and Amazon CloudFront.

4. Database Sharding

Database sharding is like dividing a large puzzle into smaller pieces and having multiple people work on different sections simultaneously.

• It horizontally partitions data across multiple database instances or shards to distribute the workload and improve scalability.
• This allows for faster data retrieval and processing. Popular technologies for database sharding include built-in features in MySQL and PostgreSQL, as well as Vitess.

5. Data Replication

Data replication is like making copies of important documents and storing them in different locations to ensure you always have a backup.

• It copies data across multiple servers or data centers to ensure redundancy and high availability. This means that even if one server fails, the data is still accessible from another location.
• Popular technologies for data replication include MySQL Replication, PostgreSQL Streaming Replication, and Apache Cassandra.

6. Caching Solutions

Caching solutions are like having a cheat sheet or a quick reference guide that you can consult instead of going through a long process every time.

• They store frequently accessed data in memory to speed up data retrieval and reduce the load on the database.
• This improves overall performance and user experience. Popular caching technologies include Redis and Memcached.

7. Messaging Queue

A messaging queue is like a postal service that allows different components of a system to communicate with each other asynchronously. It enables scalability and loose coupling by decoupling the sender and receiver of messages.

• It’s like leaving a note for someone and knowing that they will get it when they have time.
• Popular messaging queue technologies include Apache Kafka and RabbitMQ.

8. Monitoring and Logging

Monitoring and logging are like having surveillance cameras and a journal to keep track of what’s happening in your house. They track system performance, detect anomalies, and provide insights into application behavior.

• This helps with troubleshooting and optimizing the system.
• Popular monitoring and logging technologies include Prometheus, Grafana, ELK Stack (Elasticsearch, Logstash, Kibana), and AWS CloudWatch.

9. Security Components

Security components are like security guards and alarm systems that protect your home from different security threats. They implement measures such as firewalls, intrusion detection systems (IDS), and web application firewalls (WAF) to prevent unauthorized access and attacks. Popular security technologies include AWS WAF, ModSecurity, and Snort.

10. Frontend Technologies

HTML/CSS, JavaScript, React.js, React Native.

11. Backend Technologies

Node.js, Express.js, Python/Django.

12. Databases

MySQL, PostgreSQL, MongoDB, Cassandra.