# Getting Started with Hadoop

# What is Hadoop

Apache Hadoop is an **open-source software**, **Scalable**, and **Fault-tolerant framework** written in **Java** which is used for **storing** and **processing Big Data** in a **distributed manner** on large clusters of computers.

## Components of Hadoop

Hadoop consists of three main components

1. Hadoop Distributed File System (HDFS) - It is a Storage Layer
    
2. Map Reduce - It is a Processing Layer
    
3. YARN - It is a Resource Management Layer
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682250977135/cb2810fa-6331-46c8-a036-576c71c34b97.png align="center")

## Hadoop Daemons

To support these components Hadoop provides you 5 daemon services

1. NameNode
    
2. SecondaryNameNode
    
3. DataNode
    
4. ResourceManager
    
5. NodeManager
    

# History of Hadoop

The Hadoop was started by Doug Cutting and Mike Cafarella in 2002. Its origin was the Google File System paper, published by Google.

![](https://miro.medium.com/v2/resize:fit:1050/0*23y41Btkok6WPBPc.png align="left")

Let’s focus on the history of Hadoop in the following steps: -

* In 2002, Doug Cutting and Mike Cafarella started to work on a project, **Apache Nutch.** It is an open source web crawler software project.
    
* While working on Apache Nutch, they were dealing with big data. To store that data they have to spend a lot of costs which becomes the consequence of that project. This problem becomes one of the important reason for the emergence of Hadoop.
    
* In 2003, Google introduced a file system known as GFS (Google file system). It is a proprietary distributed file system developed to provide efficient access to data.
    
* In 2004, Google released a white paper on Map Reduce. This technique simplifies the data processing on large clusters.
    
* In 2005, Doug Cutting and Mike Cafarella introduced a new file system known as NDFS (Nutch Distributed File System). This file system also includes Map reduce.
    
* In 2006, Doug Cutting quit Google and joined Yahoo. On the basis of the Nutch project, Dough Cutting introduces a new project Hadoop with a file system known as HDFS (Hadoop Distributed File System). Hadoop first version 0.1.0 released in this year.
    
* Doug Cutting gave named his project Hadoop after his son’s toy elephant.
    
* In 2007, Yahoo runs two clusters of 1000 machines.
    
* In 2008, Hadoop became the fastest system to sort 1 terabyte of data on a 900 node cluster within 209 seconds.
    
* In 2013, Hadoop 2.2 was released.