KB450424 – Ceph Backfill & Recovery

Last modified: June 10, 2021
You are here:
Estimated reading time: 1 min

Ceph Backfill & Recovery

Scope/Description:

This article describes what ceph defines as recovery and backfill and how to adjust the thresholds to limit or increase recovery throughput.

  • Ceph defines “recovery” as moving pg’s when osd’s crashes and comes back online.
    • A more detailed explanation can be found here
  • Ceph defines “backfill” as moving pg’s when adding/removing osds
    • A more detailed explanation can be found here

Prerequisites:

A standing Ceph Cluster

Steps:

In Ceph Dashboard: 

  • Go to the Cluster Tab and click ‘Configuration”.

  • In the search bar search search for the setting you wish to change, select the setting and click “Save”, see below.  (You might have to uncheck the “basic” search option that is defaulted on to see these options)
Low Default High Custom
osd_max_backfills 1 1 4
osd_recovery_max_active 1 3 4
osd_recovery_max_single_start 1 1 4
osd_recovery_sleep 0.5 0 0

  • Now edit the value to the setting you wish, add the variable to the global parameter, then click “Save”.

  • Setting these variable will take effect immediately cluster wide and will not require a restart.

 

In the Terminal:

  • There are the four config options for controlling recovery/backfill
ceph config set osd osd_max_backfills <int>
ceph config set osd osd_recovery_max_active <int>
ceph config set osd osd_recovery_max_single_start <int>
ceph config set osd osd_recovery_sleep <float>
Low Default High Custom
osd_max_backfills 1 1 4
osd_recovery_max_active 1 3 4
osd_recovery_max_single_start 1 1 4
osd_recovery_sleep 0.5 0 0

Verification:

If you change the commands run in the steps above from “set” to “show”, it will output the current integer value it is set to. (i.e.,ceph config show osd.0 osd_recovery_max_active)

Troubleshooting:

Was this article helpful?
Dislike 0
Views: 43
Unboxing Racking Storage Drives Cable Setup Power UPS Sizing Remote Access