About this QuickStart Guide

This QuickStart guide gives a brief overview of the WestGrid Cortex facility, highlighting some of the features that distinguish it from other WestGrid resources. It is intended to be read by new WestGrid account holders and by current users considering whether to move to the Cortex system. For more detailed information about the Cortex hardware and performance characteristics, available software, usage policies and how to log in and run jobs, follow the links given below.

Introduction

Cortex is the head node for a collection IBM pSeries SMP machines ranging from 4 up to 64 processor cores, with different speeds and memory sizes, as shown in the table in the hardware section below. All the systems run the IBM AIX operating system.

The machines available through Cortex are suitable for parallel applications that require a large shared memory and/or fast communications between processes, or for serial applications that require a large shared memory. The batch scheduler is configured so that the largest systems are restricted to "large" jobs (either in terms of numbers of processors or memory.

Users may login only to the head node, cortex.westgrid.ca. For all the other machines, jobs must be submitted to the batch scheduling system on Cortex.

Hardware

Processors

Some basic characteristics of the machines comprising the Cortex complex are shown in this table:

Interconnect

The processors in the individual machines listed above are connected by IBM proprietary interconnects that provide for high bandwidth and low latency.

Storage

All the hosts share a total of 2.7 TB of disk space, which can be accessed as /scratch_ibm. In addition there is 690 GB of local scratch space on Synapse, 680 GB on Dendrite, and 136 GB on Bigfoot.   Here is an overview of the various file systems and their role in the Cortex computing environment:

/scratch_ibm

This is a high performance general parallel file system (IBM's GPFS). It contains most of space for users' home directories. We recommend that jobs be run from this file system. Due to the space constraints, data that is not actively being used should be moved to the WestGrid storage facility.

/ibm_global

/ibm_global is a GPFS file system dedicated to storage of software that is needed across the Cortex complex.

/scratch_local

This is disk space physically located on Dendrite, Synapse, and Bigfoot, mounted there as /scratch_local. These local file systems are NFS-mounted on Cortex as /synapse_scratch_local, /dendrite_scratch_local. /bigfoot_scratch_local, respectively. Due to the space constraints on these 'local' disks, all data older then 30 days will be removed from these file systems.

/scratch_ds4700i

This is another GPFS file system designated for home directory space for some users. Its I/O performance is better then /scratch_ibm.

Software

See the main WestGrid software page for tables showing the installed software on Cortex and other WestGrid systems, including information about the operating system and compilers.

Using Cortex

To log in to Cortex, connect to cortex.westgrid.ca using an ssh (secure shell) client. For more information about connecting and setting up your environment, see the QuickStart Guide for New Users.

Users developing their own programs should refer to the Cortex programming page. 

The Cortex login node may be used for short interactive runs during development, including parallel runs using at most two processors for up to one hour of CPU time.  Longer runs should be submitted as batch jobs.

As on other WestGrid systems, batch jobs are handled by a combination of TORQUE and Moab software. For more information about submitting jobs, see the general Running Jobs page and some Cortex-specific notes. The maximum walltime limit for any batch job submitted on Cortex is 24 hours.