Solution to 100% CPU Usage by Linux Guest on VMWare Fusion

As part of my testing setup, I have an Intel Mac Pro with Mac OS X 10.6 Server (which runs with the 64-bit kernel) on which I run VMWare Fusion 3.0.1 and several Red Hat virtual machines.

I noticed that even at idle, each VM was taking up a high amount (like 100%!) of a CPU core. Additionally, on one VM top was displaying in near-real-time, which was kind of neat but I doubt the intended behavior. Because of this, I suspected the time management in the kernel was off.

Sure enough, Timekeeping Best Practices for Linux Guests has some hints, and for more information than you'll ever want, try Timekeeping in VMWare Virtual Machines (I was particularly interested in the Clocksource Kernels section).

Making the following modification to /etc/grub.conf on RHEL5 brought my CPU usage down from 100% to barely noticeable:

Before:

kernel /vmlinuz-2.6.18-164.11.1.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet

After:

kernel /vmlinuz-2.6.18-164.11.1.el5 ro root=/dev/VolGroup00/LogVol00 rhgb divider=10

I took out quiet because I like to see what's happening when the system boots.

The Note on RHEL 5.4 or CentOS and divider=10 mentions that you do not need this for RHEL 5.4 for accurate timekeeping, but you do need it to prevent the excessive CPU use.

I also modified /etc/ntp.conf as described in the above article, adding

tinker panic 0

to the top of the file and commenting out the following lines:

# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available.
#server 127.127.1.0
#fudge 127.127.1.0 stratum 10


My VMs are down from 100 percent CPU use to practically zero. Think of the energy savings!

Solution to Error 1327: Invalid Drive

I was migrating a Parallels virtual machine to VMWare Fusion. After importing the VM and starting it, I tried to install VMWare Tools but repeatedly got the message Error 1327: Invalid Drive Z.

It turns out that as part of its OS X integration Parallels writes entries to the registry.

The solution was to type regedit at the Windows command line and use it to remove any entries at

HKEY_CURRENT_USER/Software/Microsoft/Windows/CurrentVersion/Explorer/User Shell Folders

that began with Z:. After that, everything was happy.

Western Digital My Book 2TB Smartware Not So Smart

I recently got a 2TB WD MyBook for doing offsite backups. Didn't realize that it comes with one of those annoying virtual CDs that mounts itself every time you plug in the drive. Dear Western Digital: I don't want to use your bundled software. I want to use the entire space on the drive.

So far there does not seem to be an easy way to get rid of it. Enough people must have complained that WD is providing a firmware update, but the update does not work for Studio versions of the drive.

Their other solution (and I use this term loosely) is to install their so-called Smartware program so that you can configure it to hide the virtual CD. No thanks.

Cacti Segfaulting on CentOS5

I was getting Cacti set up and was having trouble with PHP. Specifically, it appeared that enabling the Cacti cron job caused a PHP segfault. I looked in /etc/cron.d/cacti to see what was being run:

cat /etc/cron.d/cacti
#*/5 * * * * cacti /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1


Then I ran it from the command line myself:

# php -v
PHP 5.1.6 (cli) (built: Jan 13 2010 17:13:05)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies

# php /usr/share/cacti/poller.php
Segmentation fault


Sure enough, an ugly error was recorded in /var/log/messages:

kernel: php[5250]: segfault at bf29ce64 ip b788cbf3 sp bf29cdc4 error 6 in libc-2.5.so[b7851000+13f000]


It turns out that this is what happens when you bork the permissions on cacti's MySQL table. After issuing a command like the following to MySQL, my segfault went away.

GRANT ALL ON cacti.* TO 'cactiusername'@'localhost' IDENTIFIED BY 'secret';


APC UPS and Snow Leopard Server Not Communicating

I have used APC (American Power Conversion) power equipment for a long time. I have had both good and bad experiences with their products (a cherished memory is calling their technical support and asking about the "smell of burning plastic" emanating from one of their units).

I've recently upgraded an Intel Mac Pro to Snow Leopard Server (OS X Server 10.6.2) and noticed that the controls in Energy Saver for the UPS are different. Not in a good way.

Here's the relevant screen from OS X Server 10.5.8:

And the same screen from OS X Server 10.6.2:

After reading a thread on Apple's support site about APC UPS Charge Percentage Not Updating I changed the setting on the server to that shown in the second screenshot above; namely, since the computer can't seem to read the charge percentage, it's better to let the computer monitor how long it's been on the battery.

Another annoyance is that I keep getting the dialog Your computer is now running on UPS backup battery power. The UPS is supposed to self-test every 14 days but I'm seeing the dialog more often then this.

It makes me uneasy, and I don't like being uneasy about servers.

See also: http://www.macworld.com/article/140207/2009/05/apc_leopard.html

Performance presentation, Des Moines Web Geeks

Matt Nuzum, the webmaster of ubuntu.com, invited me to speak at the Des Moines Web Geeks meeting last night. The venue, Impromptu Studios, was a bit hard to find. There were no signs for Impromptu Studios, but after lurking in dark alleyways for a while I eventually discovered what should have been obvious from the beginning: enter the furniture store with the "Closed" sign on the front and go up the stairs past the disused lavatory with a sign on the door saying "Beware of the Leopard."

The group was great and we talked for about an hour and half about bottlenecks and optimization. I even went home with a Django book!

Since this was a diverse group I tried to talk about things that everyone has to deal with (bottlenecks for storage, network, memory and CPU) rather than anything Drupal-specific.

As promised, last night's slides: Performance, Scalability on the Server Side (PDF, 1MB).

Installing mytop on RHEL5

I recently installed mytop on a new 64-bit Red Hat Enterprise Linux 5.4 server. Here are my notes.

The mytop program requires two Perl modules. So let's make sure those are installed first. The perl-DBI package is usually installed as a dependency already. Check if it's there:


# yum list installed *DBI
Loaded plugins: rhnplugin, security
Installed Packages
perl-DBI.x86_64 1.52-2.el5 installed


Install TermReadKey module, also known as Term::ReadKey. I am installing this from the EPEL repository, which I regard as very safe (in terms of compatibility, longevity and maintenance). If you're not set up with EPEL, you can do

# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm

Anyway, installing Term::ReadKey from EPEL:


# yum install perl-TermReadKey
Loaded plugins: rhnplugin, security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package perl-TermReadKey.x86_64 0:2.30-4.el5 set to be updated
--> Finished Dependency Resolution
...
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : perl-TermReadKey 1/1

Installed:
perl-TermReadKey.x86_64 0:2.30-4.el5


Now it's time to get mytop.


# wget http://jeremy.zawodny.com/mysql/mytop/mytop-1.6.tar.gz
# tar zxvf mytop-1.6.tar.gz
# cd mytop-1.6
# perl Makefile.PL
Writing Makefile for mytop

# which mytop
/usr/bin/mytop


Eudora, Snow Leopard and Kerberos

I upgraded to OS X 10.6 "Snow Leopard" yesterday. A smooth upgrade except for one thing.

Eudora 6.2.4 for Mac OS X works on 10.6 (provided the optional Rosetta PowerPC-emulation code is installed), but not with kerberos.

To use kerberized POP, Eudora depends on the Code Fragment Manager library "Kerberos" which is part of the MIT Kerberos Extras for OS X package.

The actual library is installed to /System/Library/CFMSupport/Kerberos and is a 983k file last updated in 2003. It's labeled version 4.2.

If the MIT Kerberos Extras for OS X have been installed, and thus this library is installed, Eudora will fail to launch with the following errors:

Launch failed with error code -2821 (cfragInitFunctionErr) for application /System/Library/Frameworks/Carbon.framework/Versions/A/Support/LaunchCFM App


Eudora can be launched by removing the library. But then kerberos-enabled POP cannot be accomplished.

RHEL5 won't install on Dell OptiPlex 745 with more than 4G RAM

I wanted to install RHEL5 x86_64 on a Dell OptiPlex 745 (that's an Intel Q965 (ICH8) Express chipset). I had a spare one sitting around and it had 6G of RAM.

However, when I booted from the installer CD and chose the RHEL 5 Server x86_64 option, it got up to the initrd line and then lots of text started scrolling by. The text was hard to read but I could make out the following phrases:

• mapcount

• trying to fix it up, but a reboot is needed

After letting that scroll by for a while, I ended up with a message stating that there was a kernel panic.

I was frustrated at not even being able to run the installer, but I solved this by passing the following parameters to the kernel:

mem=2G text


I then hopefully updated the box to kernel-2.6.18-128.4.1.el5.x86_64.rpm but alas, the same problem returned at boot time.

Finally, I pulled two 1G sticks from the box, reducing the memory to 4G. Now it boots fine.

Postliteracy

Adam Engst's article Have We Entered a Post-Literate Technological Age? is worth reading.

When TidBITS contributing editor Matt Neuburg tried to teach a group of his friends' kids REALbasic one summer, he found himself frustrated at almost every turn - they lacked the conceptual underpinning that they could make the computer do something. And more important, they didn't care...