It is possible to also use the GPUs on parallel.westgrid.ca for batch rendering. Many applications, such as ParaView, are capable of being scripted using scripting languages like Python. Once a visualization pipeline has been created using an application like ParaView, it is often desirable to render lengthy animations of such a visualization. This can be quite time consuming. Rather than using the GUI interface for ParaView, it is possible to render such animations without the user interface. Performing such a batch rendering job is very similar to using the interactive nodes for remote visualization, with the exception that you do not need to create the client/server connection to the X desktop.
Running a batch rendering job can be accomplished as follows:
- Acquire a GPU node for rendering
- Establish the GPU you were assigned
- Run a batch visualization application (e.g. ParaView) using the assigned GPU
See below for details.
Acquire a GPU node for rendering
To request a visualization node on which to work, log in to Parallel (parallel.westgrid.ca) with your SSH terminal client, then, use qsub to submit an interactive job with the following parameters:
qsub -q interactive -I -l nodes=1:ppn=1:gpus=1,walltime=1:00:00
You may add additional options or modify the ones above, such as the walltime limit, ppn, or number of gpus, using the qsub -l argument (as explained on the Running Jobs page).
Establish the GPU you were assigned
The queing system keeps track of the GPU nodes that you were assigned in a file accessible through the PBS_GPUFILE environment variable. This file will contain lines of the form "cn0553-gpu0" where cn0553 is the node you have been assigned and gpu0 indicates that you have the zero-th GPU. Because we want the X display associated with this GPU, we need to extract the index of the display. This can be done as follows (assuming the bash shell).
Get the GPU info assigned to us:
firstgpu=$( head -n 1 "$PBS_GPUFILE" )
Get the index (the X display number) for the GPU.
We then need to tell any graphics/visualizaiton application that we are using that we want to use the appropriate X display.
Run an application with no visible display
It is then quite straightforward to run any graphics application on the GPU assigned. Note, because we have no remote display capability, we will NOT see anything.
Run a batch visualization application
Our ultimate goal in this case is to run a batch ParaView application. In order to run ParaView to create a set of frames in an animation, we simply need to run the pvbatch command on a python program that loads in a ParaView visualization pipeline (the visualization pipeline must load a data set that has time steps) and writes out each time step into an image file.
The WriteAnimation.py script is quite simple as well. It simply loads in a ParaView visualization pipeline (it assumes the dataset read has at least 50 time steps), renders each time step, and outputs that time step to a file. The code for this is given below.
except: from paraview.simple import *
mypath = '/home/USERUD/'
for i in range(50):
AnimationScene2 = GetAnimationScene()
AnimationScene2.AnimationTime = i