Scope/Description

• This article will walk through how to map a Rados Block Device (RBD) image to a client machine.
• This article will also walk through creating a XFS filesystem and mounting it to a directory.

Prerequisites

• Ceph Cluster
• RBD Pool created
• RBD image(s) created
• Ceph Common package installed on client(s)

Steps

Copying Ceph Conf and Keyring to the client machines

• Copy the ceph.conf and admin keyring from a OSD or MON node

[root@ceph-node ~]# scp /etc/ceph/ceph.client.admin.keyring root@client:/etc/ceph
[root@ceph-node ~]# scp /etc/ceph/ceph.conf root@client:/etc/ceph

Mapping the RBD image on the client machine

• Now we’ll map the RBD image on the client.
• In the below example “image_01” is the name of rbd image created
• “rbd” is the name of pool used to store the rbd image

[root@client ~] rbd map image_01 -p rbd

• Create an entry in /etc/ceph/rbdmap to ensure the rbd image is mounted at boot
• You need to create an entry for each image you plan to have mapped

[root@client ~] echo "rbd/image_01 id=admin,keyring=/etc/ceph/ceph.client.admin.keyring" >> /etc/ceph/rbdmap

• Enable the rbdmap service

[root@client ~] systemctl enable --now rbdmap

Creating XFS filesystem on the RBD image

• Here we can create an XFS filesystem on mapped rbd device. Alternatively we can map multiple RBD images and stripe together with either mdadm or LVM before creating a filesystem on the images.
• For this example we will just use a single RBD image.
• If on an OS with XFS reflink support, you will want to enable reflinks when creating the XFS filesystem. This allows fast clone feature in Veeam.

[root@xfs-non-reflinks ~] mkfs.xfs -K /dev/rbd/rbd/image_01
[root@xfs-reflinks ~] mkfs.xfs -K -m reflink=1,crc=1 /dev/rbd/rbd/image_01

Mounting RBD on Reboot

• Create an fstab entry so the filesystem will mount on reboot

[root@client ~] echo "/dev/rbd/rbd/image_01 /mnt/rbd xfs defaults,_netdev 0 0" >> /etc/fstab

Verification

• We can verify that the rbd image is mapped as lsblk will show an RBD when mapped, and we can access device from either /dev/rbd or/dev/rbd/POOL_NAME/IMAGE_NAME
• We can verify the rbdmap config is correct by unmapping all RBD devices and restarting the rbdmap service, we can then see all rbd drives are mapped with lsblk
• We can verfiy the fstab config is correct by mounting the xfs filesystem after the rbd device is mapped.

Troubleshooting

RBD image feature set mismatch. You can disable features unsupported by the kern el with "rbd feature disable kb_test1 object-map fast-diff deep-flatten".
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (6) No such device or address

rbd feature disable kb_test1 object-map fast-diff deep-flatten