RHEV

I exported a VM from RHEV 3.0 to an export domain. Then I imported the VM into RHEV 3.1.

I was surprised that the VM would not start. Instead, I got this message:

Upon further inspection, the boot order information was completely missing from the imported KVM definition:

I solved this by just editing the imported VM's disk in the RHEV UI. I changed the disk's properties from bootable to nonbootable and saved the change. Then I changed it back to bootable again. This wrote in the boot order:

I think this happened because the VM had a CD mounted in the RHEV 3.0 data center that was unavailable in the RHEV 3.1 data center.

See also: https://bugzilla.redhat.com/show_bug.cgi?id=874952

I was setting up a hypervisor for RHEV running a full RHEL6 install instead of RHEV-H, and I wanted to tune it for better hypervisorness.

So I thought, Hey! I'll install tuned, since I am all excited about tuned after viewing the slides from Mark Wagner's Achieving Top Network Performance (PDF) at the 2012 Red Hat Summit. I was particularly excited about slide 12 entitled "Teaser 3 - latency."

Not to mention that the tuned virtual-host profile is the recommended profile for RHEV hosts.

Anyway, I went ahead and installed tuned and applied the virtual-host profile:

# yum -y install tuned
...
# tuned-adm profile virtual-host
Switching to profile 'virtual-host'
Applying ktune sysctl settings:
/etc/ktune.d/tunedadm.conf: [ OK ]
Calling '/etc/ktune.d/tunedadm.sh start': can't rename /etc/mtab.tmp to /etc/mtab: Permission denied
can't rename /etc/mtab.tmp to /etc/mtab: Permission denied


Er...what? Permission denied for root to rename?

It's probably SELinux. It's always SELinux. Sure enough:

# ls -laZ /etc/mtab
-rw-r--r--. root root system_u:object_r:system_conf_t:s0 /etc/mtab
# restorecon /etc/mtab
# ls -laZ /etc/mtab
-rw-r--r--. root root system_u:object_r:etc_runtime_t:s0 /etc/mtab


Now it works fine:

# tuned-adm profile virtual-host
Reverting to saved sysctl settings: [ OK ]
Calling '/etc/ktune.d/tunedadm.sh stop': [ OK ]
Stopping tuned: [ OK ]
Switching to profile 'virtual-host'
Applying ktune sysctl settings:
/etc/ktune.d/tunedadm.conf: [ OK ]
Calling '/etc/ktune.d/tunedadm.sh start': [ OK ]
Applying sysctl settings from /etc/sysctl.conf
Starting tuned: [ OK ]

An observation: each time you apply a tuned profile (for example, going back to the default and then to a non-default again), tuned-adm may append options to /etc/mtab. For example, we start with the following line in /etc/mtab:

/dev/mapper/[UUID] /rhev/data-center/mnt/blockSD/[UUID] ext3 rw,barrier 0 0


After doing

# tuned-adm profile virtual-host
# tuned-adm profile default
# tuned-adm profile virtual-host

the line in /etc/mtab looks like this:

/dev/mapper/[UUID] /rhev/data-center/mnt/blockSD/[UUID] ext3 rw,barrier,nobarrier,barrier,nobarrier 0 0


This was a bit confusing, but since mount and umount maintain /etc/mtab I assume they know what they're doing. Interestingly, after a reboot, /proc/mounts says:

/dev/mapper/[UUID] /rhev/data-center/mnt/blockSD/[UUID] ext3 rw,seclabel,relatime,errors=continue,barrier=1,data=ordered 0 0


That makes me sleep a little better, given the dire warnings about running ext3 without barrier=1.

My use case was pretty simple. I wanted to share out a raw device via iSCSI to a nearby host on the 172.16.2.x network.

In addition to a minimal Red Hat Enterprise Linux 6 (or equivalent) install, a few packages are needed:

I knew the device I wanted to share was /dev/sdb by looking at the output from dmesg:

The target definition in /etc/tgt/targets.conf was simple as well. Isn't everything simple after you've beaten your head against a wall for hours trying to get it working? The following configuration defines one target with one LUN shared as a raw device which may only be connected to by IP address 172.16.2.2:

Starting up the tgtd service and turning it on permanently led to wonderment and success:

Depending on your configuration, you may need to open port 3260 in your firewall.

However, after a reboot, only the controller showed up (as LUN 0). LUN 1 had disappeared!

Why is LUN 1 not showing up? Telling tgt-admin to reparse targets.conf in verbose mode leads to the reason:

What? I assure you that /dev/sdb is NOT in use by the system. Show me mounts:

Nope, /dev/sdb does not appear anywhere in the output. Show me a list of open files on /dev/sdb:

Nothing. Show me active swap:

Nothing! Finally, choirboy in the #rhel irc channel pointed me to the answer: you must create a filter in /etc/lvm/lvm.conf so that LVM leaves the device alone. Appropriate section of lvm.conf, showing new filter:

After a restart, LUN 1 persists and the world is once again a happy place to be:

Edit: mutipathd may also be the culprit. Take a look at the world-wide identifier (WWID) of /dev/sdb:

Is that identifier also listed under /dev/mapper ?

If so, add

To the defaults section of /etc/multipath.conf, remove the matching line from /etc/multipath/wwids, and restart the multipath daemon:

When Red Hat Enterprise Virtualization Manager for Servers is installed, it offers to configure iptables for you:

Here's an annotated version of what the RHEVM installer will give you: