Did you know  45Drives offers free  public and private  webinars ? Click here to learn more  & register! Build & Price

KB450403 – Adding SSD Journals to OSDs

You are here:

Scope/Description

This article will go through the steps required to add journals from SSDs to your OSDs (HDD)

Prerequisites

  • Stable ceph cluster
  • OSDs on Hard Drives
  • SSDs added to the each OSD node within the cluster

Steps

  • Create LVM devices on SSD
pvcreate /dev/1-6

vgcreate ceph-db-0 /dev/1-6

lvcreate -l <extents> -n osd-db-0 ceph-db-0

extents = # of 4MiB blocks

extents = 7680 = 30GiB
  • Assign Permissions
chown -h ceph:ceph /dev/ceph-0/osd-db-0

chown -R ceph:ceph $(realpath /dev/ceph-0/osd-db-0)
  • Set the noout flag, then stop the osd you wish to add an SSD LVM to.
ceph osd set noout

systemctl stop ceph-osd@2
  • Create new block.db device.
CEPH_ARGS="--bluestore-block-db-size 32212254720" ceph-bluestore-tool bluefs-bdev-new-db --path /var/lib/ceph/osd/ceph-2/ --dev-target /dev/ceph-db-0/osd-db-0
  • Migrate the existing block.db data to a new device.
ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-2/ --devs-source /var/lib/ceph/osd/ceph-2/block --dev-target /var/lib/ceph/osd/ceph-2/block.db bluefs-bdev-migrate
  • Create LVM tags on the new block.db and existing block.
./add-lv-tags /dev/ceph-a493b896-e6e0-449f-9b06-c2daeb69cb92/osd-block-47b0a1d9-43e6-4ffe-925b-855548699a0b /dev/ceph-0/osd-db-0
  • Unmount the OSD directory and run the following commands:
umount /var/lib/ceph/osd-2/

ceph-volume lvm activate --all
  • Start the OSD again, and unset the noout flag.
systemctl start ceph-osd@2

ceph osd unset noout
  • Repeat steps for all OSD’s.

Verification

Run “ceph-volume lvm list” and find the OSD you just did to confirm it now reports having a [DB] device attached to it.

 

Was this article helpful?
Dislike 2
Views: 2126
Unboxing Racking Storage Drives Cable Setup Power UPS Sizing Remote Access