Removing the last 3 nodes of a node type from an Isilon cluster

isilon_speech_cloudInfo

In any Isilon cluster there is a requirement to have a minimum of 3 nodes for each type of node that is within the cluster. This rule begs the question of how to remove the last 3 nodes of a node type from a cluster.

What are my options for evacuating the data on the 3 nodes that will be removed from the cluster?

  1. SmartPools – A file pool policy can be created to move all data to the other nodes in the cluster. Simply create a new file pool policy that moves everything under /ifs to the tier/pool that will remain in the cluster.

Note: If all of your pools are in a single tier you will need to remove one of the pools from the tier before you can create the SmartPool file policy. Pools can safely be removed from a tier at any time.

  1. SmartFail – This method will not work. The first node you SmartFail will start to evacuate it’s data to the other 2 nodes remaining in the cluster that are the same type. Eventually those 2 nodes will fill up and the SmartFail job will hang.

What do I do when the SmartPool job has finished moving all of my data to the nodes which will remain in the cluster?

Simply SmartFail all 3 of the nodes you would like to remove from the cluster 1 by 1. They will probably take roughly 30 minutes to SmartFail when they are empty.

 

 

Does it matter which order I SmartFail the nodes?

If one of the nodes you are SmartFailing is the 1st node in the cluster then perform the SmartFail on that node last. The 1st node in an Isilon cluster has extra roles such as serving the Smart Connect Zones and hosting the cluster IP. SmartFailing the 1st node last ensures that you only need to re-assign those roles once.

Will I corrupt my data or will it cause an interruption to service at any time during the SmartFail process?

No.

3 thoughts to “Removing the last 3 nodes of a node type from an Isilon cluster”

    1. Hi Jorge – I would recommend against trying to perform this operation without a SmartPools license. When I was draining my 3 nodes I tried to do it without SmartPools and failed. I’m not saying there isn’t a way to do it, I just haven’t found it yet. I would recommend reaching out to Isilon support, or maybe you could ask for a SmartPools trial license 🙂

    2. Late reply to an old thread, but hopefully someone will find this helpful. Even without a SmartPools license, a default file pool policy exists. If you are migrating to a new pool of >3 nodes you can edit the default policy and select the new node pool as the storage target. Data will be migrated in the background. Once the old nodes are empty and you’ve moved networking to the new nodes, you can then retire the old nodes one at a time without any impact. We’ve used this process in the past without issue.

Leave a Reply

Your email address will not be published. Required fields are marked *