Scope/Description
- This article describes what ceph defines as recovery and backfill and how to adjust the thresholds to limit or increase recovery throughput.
Prerequisites
- Ceph Cluster Configured
Steps
- Ceph defines recovery as moving PGs when OSDs crash and come back online. A more detailed explanation can be found here.
- Ceph defines backfill as moving PGs when adding/removing OSDs. A more detailed explanation can be found here.
Using the Ceph Dashboard
- Go to the Cluster Tab and click Configuration.
- In the search bar search search for the setting you wish to change, select the setting and click Save, see below. (You might have to uncheck the basic search option that is defaulted on to see these options)
| Low | Default | High | Custom | |
| osd_max_backfills | 1 | 1 | 4 | – |
| osd_recovery_max_active | 1 | 3 | 4 | – |
| osd_recovery_max_hdd | 1 | 3 | 4 | – |
| osd_recovery_max_ssd | 1 | 10 | 12 | – |
| osd_recovery_max_single_start | 1 | 1 | 4 | – |
| osd_recovery_sleep | 0.5 | 0 | 0 | – |
- Now edit the value to the setting you wish, add the variable to the global parameter, then click Save.
- Setting these variable will take effect immediately cluster wide and will not require a restart.
Using the Terminal
- There are the four config options for controlling recovery/backfill.
ceph config set osd osd_max_backfills <int>
ceph config set osd osd_recovery_max_active <int>
ceph config set osd osd_recovery_max_single_start <int>
ceph config set osd osd_recovery_sleep <float>
| Low | Default | High | Custom | |
| osd_max_backfills | 1 | 1 | 4 | – |
| osd_recovery_max_active | 1 | 3 | 4 | – |
| osd_recovery_max_hdd | 1 | 3 | 4 | – |
| osd_recovery_max_ssd | 1 | 10 | 12 | – |
| osd_recovery_max_single_start | 1 | 1 | 4 | – |
| osd_recovery_sleep | 0.5 | 0 | 0 | – |
Verification
If you change the commands run in the steps above from “set” to “show“, it will output the current integer value it is set to. (i.e.,ceph config show osd.0 osd_recovery_max_active)
ceph config show osd.0 osd_recovery_max_active
Troubleshooting
Views: 6026