In any Isilon cluster there is a requirement to have a minimum of 3 nodes for each type of node that is within the cluster. This rule begs the question of how to remove the last 3 nodes of a node type from a cluster.
What are my options for evacuating the data on the 3 nodes that will be removed from the cluster?
- SmartPools – A file pool policy can be created to move all data to the other nodes in the cluster. Simply create a new file pool policy that moves everything under /ifs to the tier/pool that will remain in the cluster.
Note: If all of your pools are in a single tier you will need to remove one of the pools from the tier before you can create the SmartPool file policy. Pools can safely be removed from a tier at any time.
- SmartFail – This method will not work. The first node you SmartFail will start to evacuate it’s data to the other 2 nodes remaining in the cluster that are the same type. Eventually those 2 nodes will fill up and the SmartFail job will hang.
What do I do when the SmartPool job has finished moving all of my data to the nodes which will remain in the cluster?
Simply SmartFail all 3 of the nodes you would like to remove from the cluster 1 by 1. They will probably take roughly 30 minutes to SmartFail when they are empty.
Does it matter which order I SmartFail the nodes?
If one of the nodes you are SmartFailing is the 1st node in the cluster then perform the SmartFail on that node last. The 1st node in an Isilon cluster has extra roles such as serving the Smart Connect Zones and hosting the cluster IP. SmartFailing the 1st node last ensures that you only need to re-assign those roles once.
Will I corrupt my data or will it cause an interruption to service at any time during the SmartFail process?