Placement Group Sizing
Sizing placement group(PG) is very important. You can always increase PG size later but you can never decrease.
Keep in mind increasing the # of PGs will cause all data to begin a migration – this will put a strain on your cluster performance until it is complete.
It is best to follow:
Size correctly first with pg_calculator. https://ceph.com/pgcalc/
If you’re unsure what to choose for the correct # of PG, start with 64 PG per pool and increase the PG count at a later date. Ideally, before you put data on the pool.
Off hand rule of thumb is as follows:
Less than 5 OSDs set pg_num to 128 Between 5 and 10 OSDs set pg_num to 512 Between 10 and 50 OSDs set pg_num to 1024 If you have more than 50 OSDs, you need to understand the tradeoffs and how to calculate the pg_num value by yourself For calculating pg_num value by yourself please take help of pgcalc tool
If you’re using multiple device classes (ssd,hdd,nvme), treat each device class as if it was a separate cluster when it comes to PG calcs. (i.e. If the pool is going to live on the ssd device class – the # of OSDs would equal only the number of ssds in the cluster and the percentage of data would be 100%)