Thursday 28 July 2016

How to install tensorflow on ancient cluster machines without root?

I have eventually managed to install Tensorflow on our school's server cluster. Server machines have Python 2.6.6, no pip, gcc 4.4, GLIBC 2.12. I have no root access.

Installing Python 2.7.6 and pip as user should not be a problem.

For Tensorflow, firstly:
pip install --user --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
or (for GPU)
pip install --user --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl 
(I installed the CPU-only version as there is no GPU; the GPU version is installed in the same way on my colleague's GPU cluster)

Now, when you import tensorflow in Python, you will encounter the notorious GLIBC-version-not-found error:
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.17' not found (required by /usr/local/lib/python2.7/dist-packages/tensorflow/python/_pywrap_tensorflow.so)

To proceed, firstly download and extract glibc 2.17:
cd ~ 

wget http://launchpadlibrarian.net/137699828/libc6_2.17-0ubuntu5_amd64.deb 

wget http://launchpadlibrarian.net/137699829/libc6-dev_2.17-0ubuntu5_amd64.deb 

wget ftp://rpmfind.net/linux/sourceforge/m/ma/magicspecs/apt/3.0/x86_64/RPMS.lib/libstdc++-4.8.2-7mgc30.x86_64.rpm

ar p libc6_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx

ar p libc6-dev_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx

rpm2cpio libstdc++-4.8.2-7mgc30.x86_64.rpm | cpio -idmv

Then, create an executable shell script named python, e.g., at $HOME/bin/python and put in the following line:
$HOME/lib/x86_64-linux-gnu/ld-2.17.so --library-path "$HOME/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH:$HOME/usr/lib64/" $HOME/.local/bin/python $*
where $HOME/.local/bin/python is the correct version of Python, which should usually be in /usr/bin/python, (in my case, since I have installed Python 2.7.6 as user to over-ride the system's Python 2.6.6, my own Python 2.7.6 is put in $HOME/.local/bin/python)

Add related paths into your $HOME/.profile, make sure in your $PATH, $HOME/bin is before that of the correct version of Python:
export LD_LIBRARY_PATH="$HOME/lib:$HOME/.local/lib:$LD_LIBRARY_PATH"
export LIBRARY_PATH="$HOME/lib:$HOME/.local/lib:$LIBRARY_PATH"
export CPLUS_INCLUDE_PATH="$HOME/include:$CPLUS_INCLUDE_PATH"
export C_INCLUDE_PATH="$HOME/include:$C_INCLUDE_PATH"

export PATH="$HOME/bin:$HOME/.local/bin:$PATH"

Now, you should be able to run your Tensorflow script using the python:
python your-tensorflow-code.py

Take note, if you used to put #!/usr/bin/python as the first line in your script and type ./your-tensorflow-code.py to run, you need to change that as well, or just explicitly type "python ./your-tensorflow-code.py".

QED