KB045234 – Mapping Ceph RBD Images on a Client

Last modified: September 22, 2020
You are here:
Estimated reading time: 2 min

KB045234 – Mapping Ceph RBD Images on a Client

Scope/Description:

This article will walk through how to map a Rados Block Device (RBD) image to a client machine.

This article will also walk through creating a XFS filesystem and mounting it to a directory.

Prerequisites:

  • An existing Nautilus ceph cluster built with 45Drives Storinators.
    • When using ceph-ansible-45d to build the cluster, this process is to be done after running the “core.yml” and “dashboard.yml” playbook
    • At least 1 storage pool must be created with the RBD application tag applied
      • If not created, this can be done via CLI or the Ceph dashboard
    • At least 1 RBD image must be created
        • If not created, this can be done via CLI or the Ceph dashboard

Steps:

  • The below steps will use command for both centos7 and 8.
    • Be sure to select the correct command for the distro being used.
    • If there is only one command in the step then it is universal for either cent7 or cent 8.
  • Install EPEL repositories for extra dependencies.
[root@centos7 ~]# yum install -y epel-release
[root@centos8 ~]# dnf install -y epel-release
  • Install Ceph stable repository
[root@centos7 ~]# curl -o /etc/yum.repo.d/ceph_stable.repo http://images.45drives.com/ceph/ceph_stable_nautilus.repo
[root@centos8 ~]# curl -o /etc/yum.repo.d/ceph_stable.repo http://images.45drives.com/ceph/ceph_stable_octopus.repo
  • Install ceph-common packages
[root@centos7 ~]# yum install -y ceph-common
[root@centos8 ~]# dnf install -y ceph-common
  • Copy the ceph.conf and admin keyring from a OSD or MON node
[root@osdnode ~]# scp /etc/ceph/ceph.client.admin.keyring root@centos7-8:/etc/ceph
[root@osdnode ~]# scp /etc/ceph/ceph.conf root@client:/etc/ceph
  • Map the RBD image on the client.
    • In the below example “image_01” is the name of rbd image created
    • “rbd” is the name of pool used to store the rbd image
    • In CentOS7 you may get the message that states certain image features are not suppported. See troubleshooting section below on how to work around
[root@centos7-8 ~] rbd map image_01 -p rbd
  • Create an entry in /etc/ceph/rbdmap to ensure the rbd image is mounted at boot
    • You need to create an entry for each image you plan to have mapped
[root@centos7-8 ~] echo "rbd/image_01 id=admin,keyring=/etc/ceph/ceph.client.admin.keyring" >> /etc/ceph/rbdmap
  • Enable the rbdmap service
[root@centos7-8 ~] systemctl enable --now rbdmap
  • Create XFS filesystem on mapped rbd device
    • If on CentOS 8 you will want to enable reflinks when creating the XFS filesystem. This allows fast clone feature in application such as Veeam
[root@centos7 ~] mkfs.xfs -K /dev/rbd/rbd/image_01
[root@centos8 ~] mkfs.xfs -K -m reflink=1,crc=1 /dev/rbd/rbd/image_01
  • Create an fstab entry so the filesystem will mount on reboot
[root@centos7-8 ~] echo "/dev/rbd/rbd/image_01 /mnt/rbd xfs defaults,_netdev 0 0" >> /etc/fstab

Verification:

  • Verify that the rbd image is mapped
    • lsblk will show an rbdX device when mapped
    • You can access device from either:
      • /dev/rbdX
      • /dev/rbd/POOL_NAME/IMAGE_NAME
  • Verify the rbdmap config is correct by unmapping all RBD devices and restarting the rbdmap service. Veirfy all rbd drives are mapped with lsblk
  • Verfy the fstab config is correct by mounting the xfs filesystem after the rbd device is mapped

 

Troubleshooting:

  • When mapping on CentOS7 you may be presented with the following error
RBD image feature set mismatch. You can disable features unsupported by the kern el with "rbd feature disable kb_test1 object-map fast-diff deep-flatten".
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (6) No such device or address
  • Disable the unsupported features with the below command and map again
rbd feature disable kb_test1 object-map fast-diff deep-flatten
Was this article helpful?
Dislike 0
Views: 52
Unboxing Racking Storage Drives Cable Setup Power UPS Sizing Remote Access