Python

Installing rpy2 2.3.x and numpy on RHEL6

R from EPEL has been updated to version 3.

We would like to run rpy2 but as of rpy2 version 2.3.x, it requires Python 2.7.

Red Hat Enterprise Linux 6 comes with Python 2.6.

So we need Python 2.7 so we can use rpy2 2.3 so we can use R 3.

The solution is to create a Python virtualenv containing Python 2.7 and install rpy2 in there. That way a user can log in and get Python 2.7 when they type python at the command line.

Take a moment to make sure standard build tools are installed. At a minimum, we need the following since we want numpy to use ATLAS, BLAS, and LAPACK, and not run into problems using SSL from Python:

yum install -y gcc atlas-devel blas-devel lapack-devel openssl-devel

Python 2.7 can now be installed from source:

curl -O http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar xvzf Python-2.7.3.tgz
cd Python-2.7.3
./configure --enable-shared
make altinstall
echo "/usr/local/lib" > /etc/ld.so.conf.d/python2.7.conf
ldconfig

Now setuptools:

wget https://pypi.python.org/packages/source/s/setuptools/setuptools-0.9.8.tar.gz
tar xvfz setuptools-0.9.8.tar.gz
cd setuptools-0.9.8
/usr/local/bin/python2.7 setup.py build
/usr/local/bin/python2.7 setup.py install --record installedfiles.txt

Finally, virtualenv. This installs the virtual environment at /usr/local/ve but you might want it elsewhere.

wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
tar xvfz virtualenv-1.9.1.tar.gz
cd virtualenv-1.9.1
python2.7 setup.py install --record installedfiles.txt
virtualenv-2.7 /usr/local/ve
source /usr/local/ve/bin/activate

That last line activates the virtual environment, so adding it to your .bash_profile might be handy.

Trying to add numpy with easy_install-2.7 didn't work for me:

(ve)[snorky]# easy_install-2.7 numpy
...
Running numpy-1.7.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-u0mtJj/numpy-1.7.1/egg-dist-tmp-kGRfag
Running from numpy source directory.
error: SandboxViolation: open('/dev/null', 'w') {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

However, pip worked like a charm:

(ve)[snorky]# pip install numpy
...
(ve)[snorky]$ python
Python 2.7.3 (default, Jul 29 2013, 17:08:17)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy

Likewise rpy2 worked fine:

pip install rpy2
...

(ve)[snorky]# python
Python 2.7.3 (default, Jul 29 2013, 17:08:17)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpy2
>>> import rpy2.rinterface

Reference: Undefined symbol (Rf_PrintWarnings) when using rpy 2.3.1 and R 3.0.1

Topic: 

Solved: yum says TypeError: coercing to Unicode: need string or buffer, list found

You to run a simple yum update and yum gives you a traceback full of python info. Huh?

up2dateErrors.py212__init__TypeError coercing to Unicode need string or buffer, list found

Traceback (most recent call last)
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 285, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 105, in main
    base.getOptionsConfig(args)
  File "/usr/share/yum-cli/cli.py", line 228, in getOptionsConfig
    self.conf
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 901, in <lambda>
    conf = property(fget=lambda self self._getConfig(),
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 358, in _getConfig
    self.plugins.run('init')
  File "/usr/lib/python2.6/site-packages/yum/plugins.py", line 184, in run
    func(conduitcls(self, self.base, conf, **kwargs))
  File "/usr/share/yum-plugins/rhnplugin.py", line 114, in init_hook
    login_info = up2dateAuth.getLoginInfo()
  File "/usr/share/rhn/up2date_client/up2dateAuth.py", line 219, in getLoginInfo
    login()
  File "/usr/share/rhn/up2date_client/up2dateAuth.py", line 186, in login
    li = server.up2date.login(systemId)
  File "/usr/share/rhn/up2date_client/rhnserver.py", line 68, in __call__
    raise up2dateErrors.SSLCertificateVerifyFailedError()
  File "/usr/share/rhn/up2date_client/up2dateErrors.py", line 212, in __init__
    f = open(certFile, "r")
TypeError coercing to Unicode need string or buffer, list found

Local variables in innermost frame
up2dateConfig <up2date_client.config.Config instance at 0x2983a70>

What is happening? Has the user been messing around with the python installation and broke it? Maybe there's something wrong with the certificates! After all, it does say SSLCertificateVerifyFailedError() in the abrt output.

No, it's far simpler. Just check your system time by running date. In my case, the local time had been erroneously set to a year earlier. (Note my subtle use of the passive voice here.)

After time was set to a sane value (and ntpd was set to start at boot with chkconfig ntpd on), all was well.

Installing rpy and rpy2 on RHEL6

I wanted to install rpy and rpy2 on Red Hat Enterprise Linux 6. Here's how I did it. There have been several fixes in the rpy SVN repository that have not shown up in the version downloadable from SourceForge. Hopefully that's been fixed by now, but here's how I installed it by retrieving rpy directly from the repository.

First I made sure that the python-devel package was installed to avoid the error src/RPy.h:63:20: error: Python.h: No such file or directory:

yum install python-devel

Then I installed R and R-devel from EPEL:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm
yum install R R-devel

Then I downloaded rpy and extracted it:

curl http://rpy.svn.sourceforge.net/viewvc/rpy/trunk/rpy/?view=tar > rpy.tar.gz
tar xvzf rpy.tar.gz
cd rpy
python setup.py install

Lo and behold, it imports without error:

python -c "import rpy"

Yay!

Besides R and R-devel, rpy2 requires that the readline-devel package be installed, otherwise installation fails with ./rpy/rinterface/_rinterface.c:79:31: error: readline/readline.h: No such file or directory.

yum install readline-devel

After that, rpy2 installs easily:

curl http://pypi.python.org/packages/source/r/rpy2/rpy2-2.2.0.tar.gz#md5=a42a7f1e6ddb10dc3a1886c2f4309fab > rpy2-2.2.0.tar.gz
tar xvzf rpy2-2.2.0.tar.gz
cd rpy2-2.2.0
python setup.py install

Custom Reporting for CrashPlan PRO using Python and the REST API

CrashPlan PRO has some nice reports but sometimes you just want to dump raw data so you can do your own thing with it.

Here's an example of how to use the CrashPlan PRO REST API to dump out tab-delimited columns of data. Output looks like this:

Computer User GB Email OS IP Server Last Connect
optiplex745 Doe, Jane 25 jane@example.com Windows 7 6.1 1.2.3.4:0 1 2011-01-18T09:29:27.332-06:00
optiplex760 Doe, John 20 john@example.com Windows XP 5.1 1.2.3.5 1 2011-01-18T06:19:27.257-06:00
macbookpro Doe, Jenny 81 jenny@example.com Mac OS X 10.6.6 1.2.3.6:0 1 2011-01-18T09:25:18.946-06:00

The script is attached below as a text file. To run it just use python scriptname at the command line on the CrashPlan server. It's just a modification of one of the examples at the CrashPlan API Architecture page. You can do much more with the API than this lowly querying and reporting.

I'm running on OS X 10.5.8 (we use old Macs for CrashPlan servers since it takes very little CPU) and I had to install Python 2.7.x using the PPC installer; the stock Python in 10.5 is ancient.

AttachmentSize
Plain text icon userreport.txt4.84 KB
Subscribe to RSS - Python