Running PETSC on windows

 

PETSc, stands for portable, extensible toolkit for scientific computation, it is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations.  It employs the MPI standard for all message-passing communication.  There are hundreds of applications that uses petsc in the area of nano-simulation, medical, fusion, Geo sciences, surface flow, CFD, optimization, and much more.  

As one of the most important scientific libraries that run on top of MPI, I decided to try it out for our windows HPC cluster.  It turned out that it was quite easy to get running using visual c++ 2008, Microsoft windows MPI stack, and cygwin as a build environment. 

I took some simple notes to help anyone that needs to get it running on windows:

1. Install the following:   petsc, windows HPCS 2008 SDK, Microsoft Visual C++ 2008, and Cygwin with python.

2. run visual studio command prompt for 32 bit build, and visual studio command prompt win64 for 64bit builds...

This ensures that you have CL (the ms c++ compiler in your path). Then, run c:\cygwin\cygwin.bat to get into the cygwin environment.

3. unpack petsc:  gzip -cd petsc*.tar.gz |tar xf -

 

4. cd into the petsc directory, then Configure petsc, but first run:  export PETSC_DIR=`pwd` # THIS SETS UP bash environment for you.

5. Run the command below (one line), this is a 32 bit build.   You will have to change the path if you are building 64bit to x64 instead of i386.

config/configure.py --with-cc="win32fe cl" --with-fc=0 --download-c-blas-lapack=1
--with-mpi-include="/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Include"
--with-mpi-lib="/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Lib/i386/msmpi.lib"
--useThreads=0 --with-shared=0

6.(only 32bit version).  

edit python/BuildSystem/config/packages/MPI.py and comment out the line
> >     self.functions        = ['MPI_Init', 'MPI_Comm_create'

You'll also need to patch up the code, because 32 bit MSMPI stack uses STDCALL convention, thus it needs a little patch from petsc folks. to add MSMPI.  64 bit always uses fastcall convention, thus this step is not needed.

The patch file basically puts a MPIAPI macro in front of all their calls.  This patch is either available upon request or by emailing petsc support and refer to [PETSC #17869].

7.   Once this is all configured, type make all.

This will take a few hours......  it's a BIG library to compile.

8.   Let's build example 23....in dir:  ~/petsc-2.3.3-p13/src/ksp/ksp/examples/tutorials

type make ex23

$ make ex23

output:
/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13/bin/win32fe/win32fe cl -o ex23.o -c -
wd4996 -MT -Z7 -I/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13/src/dm/mesh/sieve -I
/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13 -I/cygdrive/c/Users/wenmingy/petsc-2.
3.3-p13/bmake/cygwin-c-debug -I/cygdrive/c/Users/wenmingy/petsc-2.3.3-p13/includ
e -I/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Include -D__SDIR__="sr
c/ksp/ksp/examples/tutorials/" ex23.c

Petsc shows you all the link options...

9.   Now, let's run it: 

$ mpiexec -n 4  ex23     // running 4 processors locally
KSP Object:
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-030
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-007, absolute=1e-050, divergence=10000
left preconditioning
PC Object:
type: jacobi
linear system matrix = precond matrix:
Matrix Object:
type=mpiaij, rows=100, cols=100
total: nonzeros=298, allocated nonzeros=700
not using I-node (on process 0) routines
Norm of error 0.000101234, Iterations 502  

10.   That's it, you just solved it.  and you can submit this job to the compute cluster job scheduler, but I am on my laptop, that would be another blog.

As you can see, with some minor tweak, PETSc plays nicely with MSMPI stack.

I would also like to thank Serguei O. for his help!

Comments

• Anonymous
  September 07, 2008
  PingBack from http://www.easycoded.com/running-petsc-on-windows/

• Anonymous
  November 13, 2008
  Have you guys really tried this on Windows Server 2008 x64? The Cygwin python is broken as described in http://cygwin.com/ml/cygwin/2007-12/msg00441.html . We can't cross-compile, since the configure will try to run the x64 bins on an x86 machine and we are kind of stuck. What am I missing?

• Anonymous
  February 13, 2009
  The comment has been removed

• Anonymous
  March 05, 2009
  Just tried to build  with intel compilers.  Only issue I had was that you must run it with –with-batch, then restart the configure process…   ./config/configure.py --with-mpi-include="/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Include"  --with-mpi-lib="/cygdrive/c/Program Files/Microsoft HPC Pack 2008 SDK/Lib/amd64/msmpi.lib"  --with-fortran=0 --with-x=0 --with-debugging=no --useThreads=0 --with-shared=0 --download-c-blas-lapack=1 --with-cc="win32fe icl --nodetect" -AR="win32fe lib" --with-batch then follow instructions….

• Anonymous
  March 21, 2009
  The comment has been removed

• Anonymous
  June 17, 2010
  The comment has been removed

• Anonymous
  June 15, 2013
  We have  built a nice version of PETSc with a Windows installer. You can find it here: www.mic-tc.ch/.../PETScforWindows.aspx