Intel Xeon Phi and Geospatial Computation:

2013 was a very exciting year for the people, interested in high performance computing. With the release of Xeon Phi coprocessor, the computing become much more cheaper. Now, almost everyone can dream about stepping into the arena of massive parallel computing. Xeon Phi coprocessors work synergistically with intel xeon processors to enable dramatic performance gains for highly parallel code. These coprocessors provide up to 61 cores, 244 threads. They come in a variety of configurations to address diverse hardware, software, workload, performance, and efficiency requirements.

For Geospatial technology lovers its a blessing, specially signal processing, spatio-temporal modeling and range of remote sensing applications. For an instance rule based classification which involves sophisticated rule based classification algorithms, demands heavy computation. Xeon Phi can be the way out.

Compilation and Installation Challenges:

Being fascinated, I have dived into this sector and in no time i got lost in range of challenges and extreme level of excitement. My wish is to prepare a system with Ubuntu servers in cluster (12.04 LTS) where python 2.7 will be installed with geospatial libraries (i.e., gdal, numpy, scipy etc.), exploiting the idea of going parallel with math kernel library (MKL), BLASH, LAPACK and intel Xeon Phi coprocessors etc. I have chosen the specific version of OS and Python to address the system, software and different geospatial library compatibility issues. For an instance, we know, to convert gdal raster to numpy array we need numpy version 1.9 or higher and ubntu 12.04 is quite stable for geospatial libraries and tested in integrated environment to ensure efficient use and interoperability.

Walking through the installation journey, I have found the physical machine fitting, coprocessor driver and intel C Compiler (icc) installations and  different environment integration processes are  not that convenient. So its quite some task. So, i though of sharing my learning in an organized manner to save some time for others who are interested in massive parallel geospatial computing.

 Xeon Phi in Ubuntu:

After having the physical machine ready with Ubuntu 12.04, first key thing to have a look is the Intel® Manycore Platform Software Stack (MPSS) which is necessary to run the Intel® Xeon Phi Coprocessor. It is dependent on Linux kernels 2.6.34 or later and it has been tested to work with specific versions of 64-bit Operating Systems. The supported OS are red Hat Enterprise, SuSe Linux Enterprise Server and Microsoft Windows Server. This is ok but I cant say thats grate because most of the massive computing system are heavily dependent on the cluster’s of ubuntu servers and that is because of high level engagement of open source developers and ubuntu is not a supported operating system by Xeon Phi Coprocessor provider. So what is the solutions?

As soon as the coprocessor came in the market, ubuntu community has been started working on that and still they are. If such system fascinates you, its good to start with the very first post by Michael Meier  who has used a specific version of MPSS (2.1) and rebuilt the Linux kernel module for the ubuntu from the source. He has provided necessary patch files which supports 3.2 kernel. After grasping the basic idea of installations its recommended to have a look into another detailed post by Peng Zhou, to get more familiar with the system architecture and installation processes. He has used Ubuntu 12.04 server with updated version of MPSS. Based on these two posts, Sasanka Nagavalli  came up with another complete Linux kernel package for ubuntu system. He has used MPSS-gold-update, which is not available as a distribution anymore.

There is a nice discussion in Intel forum, which can be a real good help and I would recommend to follow the discussion to have a successful installation in ubuntu 12.04.

Xeon Phi and Python:

After successful installation of MPSS and Intel C Compiler (which comes with Intel C Composer XE) the system would be ready to cross compile python using intel C compiler (icc) with math kernel library (MKL) and so. Here is a nice guideline by Emilio who have got to work Python as a native application which facilitates the default multicore processing capability. There are also another nice repository which is called PythonForMIC which aims to bring a working version of Python to Intel Xeon Phi but has been tested for python 2.6. Besides, Here  you can find a rather old post but i think its really helpful to understand and prepare the knowledge ground. After installing python 2.7 then the question arises about library compilation like numpy, scipy with Math kernel Library and heres another post which provides a promising guideline.