Hadoop: It is a framework that stores Big Data in distributed systems and then processes it parallelly. Four main components of Hadoop are Hadoop Distributed File System(HDFS), Yarn, MapReduce, and libraries. It involves not only large data but a mixture of structured, semi-structured, and unstructured information. Amazon, IBM, Microsoft, Cloudera, ScienceSoft, Pivotal, Hortonworks are some of the companies using Hadoop technology.
SQL: Structured Query Language is a domain-specific language used in computing and to handle data management in relational database management systems, it also processes data streams in relational data stream management systems. In nutshell, SQL is a standard Database language that is used for creating, storing and extracting data from relational databases such as MySQL, Oracle, SQL Server, etc.
Below is a table of differences between Hadoop and SQL:
| Feature | Hadoop | SQL |
|---|
| Technology | Modern | Traditional |
| Volume | Usually in PetaBytes | Usually in GigaBytes |
| Operations | Storage, processing, retrieval and pattern extraction from data | Storage, processing, retrieval and pattern mining of data |
| Fault Tolerance | Hadoop is highly fault tolerant | SQL has good fault tolerance |
| Storage | Stores data in the form of key-value pairs, tables, hash map etc in distributed systems. | Stores structured data in tabular format with fixed schema in cloud |
| Scaling | Linear | Non linear |
| Providers | Cloudera, Horton work, AWS etc. provides Hadoop systems. | Well-known industry leaders in SQL systems are Microsoft, SAP, Oracle etc. |
| Data Access | Batch oriented data access | Interactive and batch oriented data access |
| Cost | It is open source and systems can be cost effectively scaled | It is licensed and costs a fortune to buy a SQL server, moreover if system runs out of storage additional charges also emerge |
| Time | Statements are executed very quickly | SQL syntax is slow when executed in millions of rows |
| Optimization | It stores data in HDFS and process though Map Reduce with huge optimization techniques. | It does not have any advanced optimization techniques |
| Structure | Dynamic schema, capable of storing and processing log data, real-time data, images, videos, sensor data etc.(both structured and unstructured) | Static Schema, capable of storing data(fixed schema) in tabular format only(structured) |
| Data Update | Write data once, read data multiple times | Read and Write data multiple times |
| Integrity | Low | High |
| Interaction | Hadoop uses JDBC(Java Database Connectivity) to communicate with SQL systems to send and receive data | SQL systems can read and write data to Hadoop systems |
| Hardware | Uses commodity hardware | Uses propriety hardware |
| Training | Learning Hadoop for entry-level as well as seasoned profession is moderately hard | Learning SQL is easy for even entry-level professionals |