Scope/Description
- This article will walk through how to map a Rados Block Device (RBD) image to a client machine.
- This article will also walk through creating a XFS filesystem and mounting it to a directory.
Prerequisites
- Ceph Cluster
- RBD Pool created
- RBD image(s) created
- Ceph Common package installed on client(s)
Steps
Copying Ceph Conf and Keyring to the client machines
- Copy the ceph.conf and admin keyring from a OSD or MON node
[root@ceph-node ~]# scp /etc/ceph/ceph.client.admin.keyring root@client:/etc/ceph [root@ceph-node ~]# scp /etc/ceph/ceph.conf root@client:/etc/ceph
Mapping the RBD image on the client machine
- Now we’ll map the RBD image on the client.
- In the below example “image_01” is the name of rbd image created
- “rbd” is the name of pool used to store the rbd image
[root@client ~] rbd map image_01 -p rbd
- Create an entry in /etc/ceph/rbdmap to ensure the rbd image is mounted at boot
- You need to create an entry for each image you plan to have mapped
[root@client ~] echo "rbd/image_01 id=admin,keyring=/etc/ceph/ceph.client.admin.keyring" >> /etc/ceph/rbdmap
- Enable the rbdmap service
[root@client ~] systemctl enable --now rbdmap
Creating XFS filesystem on the RBD image
- Here we can create an XFS filesystem on mapped rbd device. Alternatively we can map multiple RBD images and stripe together with either mdadm or LVM before creating a filesystem on the images.
- For this example we will just use a single RBD image.
- If on an OS with XFS reflink support, you will want to enable reflinks when creating the XFS filesystem. This allows fast clone feature in Veeam.
[root@xfs-non-reflinks ~] mkfs.xfs -K /dev/rbd/rbd/image_01 [root@xfs-reflinks ~] mkfs.xfs -K -m reflink=1,crc=1 /dev/rbd/rbd/image_01
Mounting RBD on Reboot
- Create an fstab entry so the filesystem will mount on reboot
[root@client ~] echo "/dev/rbd/rbd/image_01 /mnt/rbd xfs defaults,_netdev 0 0" >> /etc/fstab
Verification
- We can verify that the rbd image is mapped as lsblk will show an RBD when mapped, and we can access device from either /dev/rbd or/dev/rbd/POOL_NAME/IMAGE_NAME
- We can verify the rbdmap config is correct by unmapping all RBD devices and restarting the rbdmap service, we can then see all rbd drives are mapped with lsblk
- We can verfiy the fstab config is correct by mounting the xfs filesystem after the rbd device is mapped.
Troubleshooting
- When mapping on older linux clients such as CentOS 7 you may be presented with the following error
RBD image feature set mismatch. You can disable features unsupported by the kern el with "rbd feature disable kb_test1 object-map fast-diff deep-flatten". In some cases useful info is found in syslog - try "dmesg | tail". rbd: map failed: (6) No such device or address
- Disable the unsupported features with the command provided in the error and map the RBD again
rbd feature disable kb_test1 object-map fast-diff deep-flatten
Views: 2647