Comparison – Centralized, Decentralized and Distributed Systems

In this article, we will try to understand and compare different aspects of centralized, decentralized, and distributed systems. 

1. CENTRALIZED SYSTEMS:

We start with centralized systems because they are the most intuitive and easy to understand and define. 

Centralized systems are systems that use client/server architecture where one or more client nodes are directly connected to a central server. This is the most commonly used type of system in many organizations where a client sends a request to a company server and receives the response. 

Figure – Centralized system visualization 

Characteristics of Centralized Systems

• Run on a single computer system and do not interact with another computer system.
• A modern, general-purpose computer system consists of one to a few processors and a number of device controllers that are connected through a common bus that provides access to shared memory.
• The processors have local cache memories that store local copies of parts of the memory, to speed up access to data.
• Centralized systems have specialized device controllers responsible for managing various hardware components, such as disk drives and audio devices
• A typical single-user system is a desktop unit used by a single person, usually with only one processor and one or two hard disks, and usually only one person using the machine at a time.
• A typical multiuser system, has more disks and more memory and may have multiple processors. It serves a large number of users who are connected to the system remotely.
• Database systems designed for use by single users usually do not provide many of the facilities that a multiuser database provides.
• Centralized systems can have multiple processors, and fine-grained parallelism is a characteristic that may be present in some centralized systems to improve performance

Example –
Wikipedia. Consider a massive server to which we send our requests and the server responds with the article that we requested. Suppose we enter the search term ‘junk food’ in the Wikipedia search bar. This search term is sent as a request to the Wikipedia servers (mostly located in Virginia, U.S.A) which then responds back with the articles based on relevance. In this situation, we are the client node, Wikipedia servers are the central server. 

Characteristics of Centralized System –

• Presence of a global clock: As the entire system consists of a central node(a server/ a master) and many client nodes(a computer/ a slave), all client nodes sync up with the global clock(the clock of the central node). 
• One single central unit: One single central unit which serves/coordinates all the other nodes in the system. 
• Dependent failure of components: Central node failure causes the entire system to fail. This makes sense because when the server is down, no other entity is there to send/receive responses/requests. 
  

Scaling –
Only vertical scaling on the central server is possible. Horizontal scaling will contradict the single central unit characteristic of this system of a single central entity. 

Components of Centralized System –
Components of Centralized System are, 

• Node (Computer, Mobile, etc.). 
• Server. 
• Communication link (Cables, Wi-Fi, etc.). 
  

Architecture of Centralized System –
Client-Server architecture. The central node that serves the other nodes in the system is the server node and all the other nodes are the client nodes. 

Limitations of Centralized System –

• Can’t scale up vertically after a certain limit – After a limit, even if you increase the hardware and software capabilities of the server node, the performance will not increase appreciably leading to a cost/benefit ratio < 1. 
  
• Bottlenecks can appear when the traffic spikes – as the server can only have a finite number of open ports to which can listen to connections from client nodes. So, when high traffic occurs like a shopping sale, the server can essentially suffer a Denial-of-Service attack or Distributed Denial-of-Service attack. 
  

Advantages of Centralized System –

• Easy to physically secure. It is easy to secure and service the server and client nodes by virtue of their location
• Smooth and elegant personal experience – A client has a dedicated system which he uses(for example, a personal computer) and the company has a similar system which can be modified to suit custom needs
• Dedicated resources (memory, CPU cores, etc)
• More cost-efficient for small systems up to a certain limit – As the central systems take fewer funds to set up, they have an edge when small systems have to be built
• Quick updates are possible – Only one machine to update.
• Easy detachment of a node from the system. Just remove the connection of the client node from the server and voila! Node detached.
• Centralized control: In a centralized system, the central authority has complete control over the system, which can lead to better coordination and decision-making.
• Easier to manage: As there is only one central node to manage, it is easier to maintain and manage the system.
• Lower latency: Centralized systems can provide lower latency compared to distributed systems as there is no delay in communication between different nodes.
• Higher performance: Centralized systems can achieve higher performance as the resources can be optimized for specific tasks.
• Simpler implementation: Centralized systems are easier to implement as they require less complex algorithms and protocols.

Disadvantages of Centralized System –

• Highly dependent on the network connectivity – The system can fail if the nodes lose connectivity as there is only one central node.
• No graceful degradation of the system – abrupt failure of the entire system
• Less possibility of data backup. If the server node fails and there is no backup, you lose the data straight away
• Difficult server maintenance – There is only one server node and due to availability reasons, it is inefficient and unprofessional to take the server down for maintenance. So, updates have to be done on-the-fly(hot updates) which is difficult and the system could break.
• Single point of failure: Centralized systems have a single point of failure, which can cause the entire system to fail if the central node goes down.
• Lack of transparency: Centralized systems lack transparency as the central authority has complete control over the system, which can lead to issues like censorship and bias.
• Security risks: Centralized systems are more vulnerable to security risks as the central authority has complete access to all the data.
• Limited scalability: Centralized systems have limited scalability as the central node can only handle a limited number of clients at a time.
• Limited innovation: Centralized systems can stifle innovation as the central authority has complete control over the system, which can limit the scope for experimentation and creativity.

Applications of Centralized System –

• Application development – Very easy to set up a central server and send client requests. Modern technology these days do come with default test servers which can be launched with a couple of commands. For example, Express server, Django server.
• Data analysis – Easy to do data analysis when all the data is in one place and available for analysis
• Personal computing

Use Cases –

• Centralized databases – all the data in one server for use.
• Single-player games like Need For Speed, GTA Vice City – an entire game in one system(commonly, a Personal Computer)
• Application development by deploying test servers leading to easy debugging, easy deployment, easy simulation
• Personal Computers

Organizations Using –
National Informatics Center (India), IBM 

2. DECENTRALIZED SYSTEMS:

These are other types of systems that have been gaining a lot of popularity, primarily because of the massive hype of Bitcoin. Now many organizations are trying to find the application of such systems. 

In decentralized systems, every node makes its own decision. The final behavior of the system is the aggregate of the decisions of the individual nodes. Note that there is no single entity that receives and responds to the request. 

Figure – Decentralized system visualization 

Example –
Bitcoin. Let’s take Bitcoin for example because it is the most popular use case of decentralized systems. No single entity/organization owns the bitcoin network. The network is a sum of all the nodes who talk to each other for maintaining the amount of bitcoin every account holder has. 

Characteristics of Decentralized System –

• Lack of a global clock: Every node is independent of each other and hence, has different clocks that they run and follow.
• Multiple central units (Computers/Nodes/Servers): More than one central unit which can listen for connections from other nodes
• Dependent failure of components: one central node failure causes a part of the system to fail; not the whole system

Scaling –
Vertical scaling is possible. Each node can add resources(hardware, software) to itself to increase the performance leading to an increase in the performance of the entire system. 

Components –
Components of Decentralized System are, 

• Node (Computer, Mobile, etc.)
• Communication link (Cables, Wi-Fi, etc.)

Architecture of Decentralized System –

• peer-to-peer architecture – all nodes are peers of each other. No one node has supremacy over other nodes
• master-slave architecture – One node can become a master by voting and help in coordinating of a part of the system but this does not mean the node has supremacy over the other node which it is coordinating

Limitations of Decentralized System –

• May lead to the problem of coordination at the enterprise level – When every node is the owner of its own behavior, its difficult to achieve collective tasks
• Not suitable for small systems – Not beneficial to build and operate small decentralized systems because of the low cost/benefit ratio
• No way to regulate a node on the system – no superior node overseeing the behavior of subordinate nodes

Advantages of Decentralized System –

• Minimal problem of performance bottlenecks occurring – The entire load gets balanced on all the nodes; leading to minimal to no bottleneck situations
• High availability – Some nodes(computers, mobiles, servers) are always available/online for work, leading to high availability
• More autonomy and control over resources – As each node controls its own behavior, it has better autonomy leading to more control over resources.
• Improved fault tolerance: Decentralized systems are designed to be fault tolerant, meaning that if one or more nodes fail, the system can still continue to function. This is because the workload is distributed across multiple nodes, rather than relying on a single point of failure.
• Increased transparency: Decentralized systems often have a transparent and open structure, which allows for greater accountability and trust. Each node has access to the same information, making it more difficult to manipulate or corrupt the data.
• Greater security: Decentralized systems can be more secure than centralized systems because there is no single point of failure or vulnerability that can be exploited by attackers. Data is distributed across multiple nodes, making it more difficult to hack or compromise.
• Improved scalability: Decentralized systems can be more scalable than centralized systems because adding new nodes to the network can help to distribute the workload and increase capacity. This can be particularly useful for large, complex systems that need to be able to handle high volumes of traffic or data.

Disadvantages of Decentralized System –

• Difficult to achieve global big tasks – No chain of command to command others to perform certain tasks
• No regulatory oversight
• Difficult to know which node failed – Each node must be pinged for availability checking and partitioning of work has to be done to actually find out which node failed by checking the expected output with what the node generated
• Difficult to know which node responded – When a request is served by a decentralized system, the request is actually served by one of the nodes in the system but it is actually difficult to find out which node indeed served the request.
• Security challenges: Decentralized systems can be vulnerable to security threats such as DDoS attacks, sybil attacks, and 51% attacks. These attacks can compromise the integrity and security of the network, leading to data breaches and loss of trust.
• Lack of scalability: Decentralized systems can face scalability issues as the number of nodes increases. This is because each node needs to maintain a copy of the entire database, which can be difficult to manage as the database grows.
• Inefficient resource utilization: Decentralized systems can suffer from inefficient resource utilization as some nodes may have spare computing resources while others may be overloaded. This can lead to a waste of resources and decreased performance.
• Lack of standardization: Decentralized systems lack standardization, which can make it difficult to integrate with other systems and can lead to interoperability issues. This can be a major challenge for organizations that need to work with multiple decentralized systems.
• Slow transaction processing: Decentralized systems can be slower in processing transactions compared to centralized systems. This is because each transaction needs to be validated by multiple nodes, which can take time.

Applications of Decentralized System –

• Private networks – peer nodes joined with each other to make a private network.
• Cryptocurrency – Nodes joined to become a part of a system in which digital currency is exchanged without any trace and location of who sent what to whom. However, in bitcoin, we can see the public address and amount of bitcoin transferred, but those public addresses are mutable and hence difficult to trace.

Use Cases –

• Blockchain
• Decentralized databases – Entire databases split into parts and distributed to different nodes for storage and use. For example, records with names starting from ‘A’ to ‘K’ in one node, ‘L’ to ‘N’ in the second node, and ‘O’ to ‘Z’ in the third node
• Cryptocurrency

Organizations Using –
Bitcoin, Tor network 

3. DISTRIBUTED SYSTEMS:

A distributed system is a collection of computer programs that utilize computational resources across multiple, separate computation nodes to achieve a common, shared goal. Also known as distributed computing or distributed databases, it relies on separate nodes to communicate and synchronize over a common network. These nodes typically represent separate physical hardware devices but can also represent separate software processes, or other recursive encapsulated systems. Distributed systems aim to remove bottlenecks or central points of failure from a system.

Example :
Google search system. Each request is worked upon by hundreds of computers that crawl the web and return the relevant results. To the user, Google appears to be one system, but it actually is multiple computers working together to accomplish one single task (return the results to the search query). 

Characteristics of Distributed System:

• Resource sharing: A distributed system can share hardware, software, or data but they do not share main memory or disks.
• Simultaneous processing: Multiple machines can process the same function simultaneously
• Scalability: The computing and processing capacity can scale up as needed when extended to additional machines
• Error detection: Failures can be more easily detected
• Transparency: A node can access and communicate with other nodes in the system
• The computers in a distributed system communicate with one another through various communication media, such as high-speed private networks or the Internet.
• Data is spread over multiple machines .
• The computers in a distributed system are referred to by a number of different names, such as sites or nodes.
• Network interconnects the machines.
• Data is shared by users on multiple machines.

Types of Distributed Systems:

Homogeneous distributed databases:

• Same software/schema on all sites, data may be partitioned among sites.
• Goal: provide a view of a single database, hiding details of distribution.

Heterogeneous distributed databases:

• Different software/schema on different sites.
• Goal: integrate existing databases to provide useful functionality.

Components of Distributed System :

The components of Distributed System are, 

• Node (Computer, Mobile, etc.)
• A communication link (Cables, Wi-Fi, etc.)

The architecture of Distributed System –

• peer-to-peer – all nodes are peers of each other and work towards a common goal
• client-server – some nodes become server nodes for the role of coordinator, arbiter, etc.
• n-tier architecture – different parts of an application are distributed in different nodes of the systems and these nodes work together to function as an application for the user/client

Limitations of Distributed System –

• Difficult to design and debug algorithms for the system. These algorithms are difficult because of the absence of a common clock; so no temporal ordering of commands/logs can take place. Nodes can have different latencies which have to be kept in mind while designing such algorithms. The complexity increases with the increase in the number of nodes. Visit this link for more information
• No common clock causes difficulty in the temporal ordering of events/transactions
• Difficult for a node to get the global view of the system and hence take informed decisions based on the state of other nodes in the system

Advantages of Distributed System –

• Low latency than a centralized system – Distributed systems have low latency because of high geographical spread, hence leading to less time to get a response.
• Scalability: Distributed systems are highly scalable as they can be easily expanded by adding new nodes to the network. This allows the system to handle a large amount of data and traffic without compromising on performance.
• Fault tolerance: Distributed systems are fault-tolerant, meaning they can continue to function even if some nodes fail or go offline. This is because the workload is distributed among multiple nodes, so the system can continue to operate even if some nodes are down.
• Increased reliability: With multiple nodes in the network, distributed systems are more reliable than centralized systems. Even if one node fails, there are still other nodes that can provide the required service.
• Cost-effective: Distributed systems are often more cost-effective than centralized systems, as they can be built using off-the-shelf hardware and software components. This makes them a more affordable option for many organizations.
• Improved performance: Distributed systems can achieve higher performance by using parallel processing techniques, where tasks are split among multiple nodes in the network, and each node processes its share of the workload simultaneously. This can significantly reduce processing time and improve overall system performance.
• Autonomy: Each site is able to retain a degree of control over data stored locally.
• Sharing data: Users at one site are able to access the data residing at some other sites.

Disadvantages of Distributed System –

• Difficult to achieve consensus
• The conventional way of logging events by absolute time they occur is not possible here.
• Network complexity: Distributed systems require a complex network infrastructure to operate, which can be difficult to set up and maintain. Network outages and bottlenecks can cause system failures and data loss.
• Security challenges: Distributed systems can be vulnerable to security threats such as cyber-attacks, data breaches, and unauthorized access. It can be challenging to implement robust security measures across all nodes and ensure data privacy and integrity.
• Synchronization issues: Data consistency and synchronization across all nodes can be a challenge, especially when multiple users are accessing the system concurrently. Ensuring data consistency and avoiding data conflicts requires complex algorithms and protocols.
• High development and maintenance costs: Distributed systems require specialized skills and expertise to design, develop, and maintain. This can lead to high development and maintenance costs, especially for large-scale systems.
• Limited scalability: Distributed systems can have limited scalability due to network constraints, synchronization challenges, and other technical limitations. Scaling up the system requires significant investment and can be challenging to implement.

Applications of Distributed System –

• Cluster computing – a technique in which many computers are coupled together to work so that they achieve global goals. The computer cluster acts as if they were a single computer
• Grid computing – All the resources are pooled together for sharing in this kind of computing turning the systems into a powerful supercomputer; essentially.

Use Cases –

• SOA-based systems
• Multiplayer online games

Organizations Using –
Apple, Google, Facebook.