Scope/Description

This article will go through the steps required to add journals from SSDs to your OSDs (HDD)

Prerequisites

• Stable ceph cluster
• OSDs on Hard Drives
• SSDs added to the each OSD node within the cluster

Steps

• Create LVM devices on SSD

pvcreate /dev/1-6

vgcreate ceph-db-0 /dev/1-6

lvcreate -l <extents> -n osd-db-0 ceph-db-0

extents = # of 4MiB blocks

extents = 7680 = 30GiB

• Assign Permissions

chown -h ceph:ceph /dev/ceph-0/osd-db-0

chown -R ceph:ceph $(realpath /dev/ceph-0/osd-db-0)

• Set the noout flag, then stop the osd you wish to add an SSD LVM to.

ceph osd set noout

systemctl stop ceph-osd@2

• Create new block.db device.

CEPH_ARGS="--bluestore-block-db-size 32212254720" ceph-bluestore-tool bluefs-bdev-new-db --path /var/lib/ceph/osd/ceph-2/ --dev-target /dev/ceph-db-0/osd-db-0

• Migrate the existing block.db data to a new device.

ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-2/ --devs-source /var/lib/ceph/osd/ceph-2/block --dev-target /var/lib/ceph/osd/ceph-2/block.db bluefs-bdev-migrate

• Create LVM tags on the new block.db and existing block.

./add-lv-tags /dev/ceph-a493b896-e6e0-449f-9b06-c2daeb69cb92/osd-block-47b0a1d9-43e6-4ffe-925b-855548699a0b /dev/ceph-0/osd-db-0

• Unmount the OSD directory and run the following commands:

umount /var/lib/ceph/osd-2/

ceph-volume lvm activate --all

• Start the OSD again, and unset the noout flag.

systemctl start ceph-osd@2

ceph osd unset noout

• Repeat steps for all OSD’s.

Verification

Run “ceph-volume lvm list” and find the OSD you just did to confirm it now reports having a [DB] device attached to it.