edit · print · PDF

Please note that all the SIEpedia's articles address specific issues or questions raised by IAC users, so they do not attempt to be rigorous or exhaustive, and may or may not be useful or applicable in different or more general contexts.

Step by step guide for the installation and configuration of a cluster to run ParaView

Introduction

As part of a student project that I was mentoring, I built a small ParaView server (4 nodes, 16 CPUs). Since the ParaView documentation didn't seem to have a step-by-step guide on how to do it, I put together the notes of what I did. I'm sure many things could be done better, and I would appreciate any comments on how to improve things. Some of the decisions taken for the installation of our cluster reflect the type of cluster that we wanted to have, and therefore the guide is certainly not comprehensive, but I put it here hoping that it can be useful to someone else. Certainly, some issues (like X connections) seem to be a common source of errors, and the way we solved it can probably be useful to others.

Our test cluster is made up of four x86_64 nodes (each with four CPUs) connected via a Gigabit switch, and the frontend has two network cards. eth1 is connected to the outside world, while eth0 (as per the other nodes) is connected to the switch. In order to make the installation of the cluster easier, I decided to use the Rocks cluster kit, in order to have the basic cluster up and running as quickly as possible (although whether you use other cluster kit or even build the basic cluster manually should not matter much for the installation of ParaView).

Installation of the basic cluster, with Rocks 5.1

The latest version of Rocks is 5.2, but we needed to have the PVFS2 parallel file system installed, and this version does not have (at the time I did the installation) the PVFS2 roll (a pre-packaged and ready-to-install bundle), so we went for Rocks 5.1 instead. The installation of Rocks to create the basic cluster is very easy, and only a few steps are needed.

  • I downloaded the Jumbo DVD x86_64 and the PVFS2 roll from http://www.rocksclusters.org/wordpress/?page_id=97. Basically you put the Jumbo DVD in the CD tray, start the computer and follow the steps (if using a USB DVD drive, you should check out this tip.
  • You can find the Rocks 5.1 documentation here, and first you should install the frontend, following this. For the ParaView cluster, when asked, we select the following rolls to install: base, ganglia, hpc, kernel, os, web-server, and pvfs2 (for this you will need to swap DVDs). This is very similar to a standard distribution install, so you should decide on which partitioning you want (I selected autopartition), network details, keyboard layout, etc.
  • Once the installation of the frontend is finished, when you open for the first time a terminal, you will see that the ssh keys are created automatically. Verify that the network works OK, and before proceeding further, let's make a small change to make sure that the compute nodes will have an X server running (not usual in a computational cluster, but necessary to run ParaView). For this, we follow these instructions.

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • And modify extend-compute.xml so that it becomes:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • And then do:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • With this, the compute nodes will have all the necessary stuff to run ParaView, so we can proceed with their installation. For this, we follow these instructions. We run insert-ethers as root in the frontend, and proceed as per the mentioned instructions, although when asked about appliance type, we select Compute-PVFS2 I/O, which means that our nodes will be both computing nodes, and part of the PVFS2 file system (in a production cluster, these roles would probably belong to different nodes, but for our test cluster, this is OK). Installing everything does take a while, and the command rocks-console is useful, but the VNC connections are not always available, so you should also verify the status of the installations with ping, ssh, etc.
  • When the compute nodes have been installed, it is time to configure PVFS2, and for this we just need to do the step mentioned here, which basically is just running rocks sync pvfs. With this in place, we can now verify that PVFS2 works fine:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • The Rocks 5.1 installation comes with a web server, with Ganglia and some documentation, but the iptables rules as configured by Rocks are too restrictive for our setting. The file /etc/sysconfig/iptables had lines for https and for www with a --source option, which we comment out, and copy without it (our server is in an internal network behind a firewall, so we can allow unrestricted access). With this, we can monitor our server.

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • Next step is create regular users, which is very easy. We just create a normal user in the frontend with the command useradd and then synchronize to all nodes with the command rocks sync users (the first time I run it, it complained about clock skews, which was solved by waiting some time, in the order of one or two hours. The second time I installed the cluster I didn't have this problem).
  • We are basically done with the basic cluster installation. We just need to verify that we can run (as a regular user) MPI programs. In the regular user account created above, we create a file named machines and try a basic MPI program.

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • So all is good, and we can proceed with the ParaView installation.

Installation of ParaView

  • The stable version of ParaView at the time of writing was 3.6.1, which we download from here. We follow these instructions, which imply that we need to get Cmake and Qt. For Cmake we get version 2.6.4 and for Qt we get version 4.3.5.
  • Installation of cmake is just ./bootstrap ; make ; make install
  • Installation of Qt is simply ./configure ; make ; make install
  • For the installation of ParaView (assuming you unpacked the paraview-3.6.1.tar.gz in the /root directory, which creates the ParaView3 directory) we just do:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • When running ccmake, I modify the following options, so that they have these values:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • Then after selecting the option generate and exit, we can just compile ParaView:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)

Running ParaView

Depending on how you want to run ParaView, different settings could be possible. Our goal was to have the cluster only accessible to users via ssh. Some of the steps below could be easier if we left an X session open by one user, but we didn't like this idea very much, so the following will work assuming that no user is logged-in in any of the nodes in the cluster (which is convenient, since this is just the state you would get after rebooting. If you have a root session open in the frontend, you could log out and work remotely from now on).

  • The ParaView wiki tells you that configuring properly X connections is one of the most common problems with setting up a ParaView server. The way we solved it was with four basic scripts in /share/apps:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • grant.X.access will copy the appropriate magic cookies to the given user, so that he/she will be able to open an X application in the local monitor of one of the nodes. remove.X.access will just remove the magic cookie for that monitor. rocks-grant will get all the magic cookies for all the nodes and rocks-remove will delete them. Ideally you would like to write a script that will run in all the nodes of your cluster (without hardcoding their names as in the examples above, but this was just convenient for our small cluster). In any case, you should run the scripts grant.X.access or remove.X.access sequentially in each node and not in parallel, to avoid problems when locking the .Xauthority file.
  • Now it is very easy to add or remove permissions (to open X applications in the local monitors) for a given user (and we could easily integrate the granting and removing of permissions with a queuing system):

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • Now we can verify that after running rocks-grant, the given user can open an X application in the local monitor. For example:

Before running rocks-grant


GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)

After running rocks-grant (connected via ssh from another machine, and opens a window in the local monitor of the cluster)


GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • So now is time to run glxgears (as suggested in the instructions) in parallel. In our cluster we found the problem that when running glxgears, it complained that the "GLX" extensions were missing:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • This was easily solved adding the line Load "glx" in the file /etc/X11/xorg.conf, and restarting the X server in all the nodes with the command rocks run host gdm-restart

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • Now (after running rocks-grant again, since each time that the X server is started, the magic cookies change), glxgears is no problem (though VERY slow, this is a very old test machine):

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • You could verify that glxgears is really running in all nodes:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • We are almost there, and now we just need to do the final test, in which we will have a ParaView client connected to the ParaView server in the cluster. In the server we run ParaView in parallel (make sure that the name of the frontend is the first one in the machines file):

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • In another workstation I download the "Linux x86, 64 bit" ParaView software, and running it gives no trouble:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • We now have to define the server to connect (File -> Connect), and we specify a new connection, with Host being the name of your server, and Startup Type set (for the moment) to Manual. We click to connect, and then in the ParaView server we see the last line appearing, showing a partial success.

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • We try rendering a sphere in the ParaView client GUI (Sources -> Sphere, and we select Theta Resolution and Phi Resolution to 1024), but after clicking Apply we only see a quarter of the sphere and some error messages appear in the server terminal:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • This (which probably doesn't happen in most settings) was due to different screen resolutions:

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • To get the frontend in line with the other nodes, I add the Modes "800x600" line to /etc/X11/xorg.conf, and then restart the X server.

GeSHi Error: GeSHi could not find the language shell (using path /datos/ext/sieinvens/siepedia/cookbook/geshi/geshi/geshi/) (code 2)
  • With this in place, we can rocks-grant permission to our regular user again, restart the server and the client, and now when rendering the sphere as for the steps above, there are no errors, and everything works as expected!!

Comments or suggestions to improve this guide can be sent to

Section: HOWTOs

edit · print · PDF
Page last modified on November 17, 2009, at 10:14 AM