Raw Device Mapping (RDM) in vSphere & Proxmox
Raw Device Mapping (RDM) is the term used by VMware to indicate that a single physical disk is fully assigned to a VM. In VMware ESXi this can be done in multiple ways with slightly different properties, but for TrueNAS the commands below are suitable because they provide a “physical” passthrough.
Setting up vSphere RDM
Section titled “Setting up vSphere RDM”To create an RDM you need to log in to your ESXi host via SSH. SSH must first be enabled in the vSphere Host Client.
In the shell you can then execute the following steps/commands.
Step 1: Identify disks
Section titled “Step 1: Identify disks”ls -l /vmfs/devices/disksThis command lists all disks in the system. You need to find the UID used in the path of the disk you want to use with RDM. It usually starts with t10. or naa.. You can also find this in the GUI under disk properties in the vSphere Host Client or the vSphere Client (vCenter).
Step 2: Create the RDM
Section titled “Step 2: Create the RDM”vmkfstools -z /vmfs/devices/disks/<diskUID> "/vmfs/volumes/<Datastore>/<Folder>/<RDM_Disk_Name>.vmdk"The command above creates a .vmdk file (virtual disk) that is a direct representation of a physical disk and forwards all I/O requests to the physical disk.
Physical vs. Virtual Compatibility
Section titled “Physical vs. Virtual Compatibility”You can also choose a virtual compatibility RDM instead of physical. This has the advantage that RDMs continue to work during a vMotion. However, the downside is that all ESXi hosts must have a connection (path) to the disk the RDM points to. Also, certain information such as S.M.A.R.T. status is not passed through to the VM and remains the responsibility of ESXi. This is only recommended in specific scenarios and is typically only used with LUNs because the Disk Shelves or SAN Controller are responsible for the disks.
Useful links
Section titled “Useful links”Setting up disk passthrough in Proxmox
Section titled “Setting up disk passthrough in Proxmox”To assign a physical disk to a VM in Proxmox, similar steps are taken.
Step 1: Identify disks
Section titled “Step 1: Identify disks”lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'The command above, like with VMware, lists all disks with their UIDs. In this case you need the first path with the UID of the disk. There are 2 listed one after another starting with /dev/disk/by-id/. You need the one where the UID after the path belongs to the disk itself, such as the manufacturer name or model/serial number printed on the disk’s sticker. In most cases the connection type is also included. SATA disks for example start with ata-.
Step 2: Assign disk to VM
Section titled “Step 2: Assign disk to VM”qm set <vmID> -sata<sata_connector_number> /dev/disk/by-id/<deviceUID>By executing the command above, Qemu is told to assign the specified physical device (disk) to the VM with the VM ID listed in the command, on the SATA connection of the specified number. Here sata stands for the disk/connector type (can also be scsi depending on your disks) and the number for a unique connection. This number must be unique. Think of it as a SATA connector on your motherboard — you cannot share it between multiple disks.
Step 3: Add serial number
Section titled “Step 3: Add serial number”Finally, the VM config must include the serial number of each disk. Otherwise TrueNAS will not behave correctly. This is done by adding the serial number of each disk in the VM’s configuration file.
nano /etc/pve/qemu-server/<vmID>.confOn the line where your disk is defined (sata<sata_connector_number>: /dev/disk/by-id/<deviceUID>, size=<disk_size_inKB>) add the serial number with , serial=<serialnumber>.
The serial number is usually embedded in the UID of the disk and is always printed on the sticker on the disk itself.