The practice of removing nodes from a Hadoop cluster is very common. Hardware might have failed, or machines might need to be upgraded. In this recipe, we will show you the steps to safely remove a worker node from a Hadoop cluster.
It is assumed that your cluster is up and running and you have configured the following properties in mapred-site.xml
:
<property> <name>dfs.hosts.exclude</name> <value>/path/to/hadoop/dfs_excludes</value> <final>true</final> </property> <property> <name>mapred.hosts.exclude</name> <value>/path/to/hadoop/mapred_excludes </value> <final>true</final> </property>
In addition, there should be two files located in the Hadoop home
folder on the head node: dfs_excludes
and mapred_excludes
.
Perform the following steps to decommission a node in the Hadoop cluster:
dfs_excludes
and mapred_excludes
files on the head node:$ vi /path/to/hadoop/dfs_excludes worker1 $ vi /path/to/hadoop/mapred_excludes worker1
$ hadoop dfsadmin –refreshNodes
$ hadoop mradmin –refreshNodes
$ hadoop dfsadmin -report
First, we added the hostname of the node we wanted to decommission to the dfs_excludes
and
mapred_excludes
files we created in a previous recipe. Next, we issued the hadoop dfsadmin –refreshNodes
command to notify the NameNode to disconnect from all of the hosts listed in the dfs_excludes
file. Similarly, we issued the
hadoop mradmin –refreshNodes
command to notify the JobTracker to stop using the TaskTrackers on the nodes listed in the mapred_excludes
file.