KB450198 – Rebooting Ceph Storage Nodes

Last modified: June 14, 2021
You are here:

Rebooting Ceph Storage Nodes

Scope/Description:

This guide will detail the process of going  through a rolling reboot of each OSD/GW node. This will need to be done after an update to the kernel.

Prerequisites:

A Cluster in a Healthy State

Steps:

To reboot the Ceph Storage nodes, follow this process:

  1. Select the first Ceph Storage node to reboot and log into it.

  2. Disable Ceph Storage cluster rebalancing temporarily:

    ceph osd set noout
    ceph osd set nobackfill
    ceph osd set norecover
  3. Reboot the node:

    reboot
  4. Wait until the node boots.

  5. Log into the node and check the cluster status:

    ceph -s

    Verify all OSDs, MON and MGR services have started and rejoined cluster

  6. Log out of the node, reboot the next node, and check its status. Repeat this process until you have rebooted all Ceph storage nodes.

  7. When complete, enable cluster rebalancing again:

    ceph osd unset noout
    ceph osd unset nobackfill
    ceph osd unset norecover

    
    

Verification:

Perform a final status check to make sure the cluster reports HEALTH_OK:

ceph -s

Troubleshooting:

If the cluster is above 90 percent full there may be some client IO issues when rebooting a node. It is better to do this process after hours in that case.


Was this article helpful?
Dislike 0
Views: 585
Unboxing Racking Storage Drives Cable Setup Power UPS Sizing Remote Access