> For the complete documentation index, see [llms.txt](https://jairanjankumar.gitbook.io/cheatsheet/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://jairanjankumar.gitbook.io/cheatsheet/yarn.md).

# YARN

YARN

is resource manager for hadoop.

* It's a framework to provide **computational resources** for **execution engine**.
  * Here computational resources means -> CPU, Memory, Disk I/O, Network Bandwidth
  * execution engine example - Spark, Map Reduce, Storm, Solr, Tez

Component of YARN, Below are 2 daemon running on cluster.

* Resource Manager - One per Cluster - master service
  * deployed on high availability configuration.
* Node Manager - One per data Node - Slave service
  * Node manager is responsible to launching and monitoring container. - Container is linux control groups that limits / isolates the resource usage (CPU, memory, disk I/O, network, etc.) to process.

job is submitted -> **resource manager** -> resource manager finds one node manager and ask to start one **container** -> its first container and called **Application Master** -> Now Application Master takes responsibility of executing job, and behave differently for different executing framework -> (next steps for **Map reduce**) -> Now Application Master will ask Resource Manager for more containers so it can start map and reduce tasks -> Once container are located, Application Master ask Node Manager to launch containers and execute the task -> Task directly report its status to Application Master -> Once all Tasks are completeed, All containers including Application Master perform necessary clean-up and terminate.

**Spark** on YARN