You are here

Grex Updates and Current Status: TEST MODE

Grex WestGrid University of Saskatchewan

On May 17, a new lustre file system with 418 TB of usable disk space was installed on Grex. During this time an outage was scheduled with user access disabled to allow migration of all data to the new system. Some updates on the status of the new system are provided below:

Current Status of Grex: TEST MODE

Right now Grex is available in test mode. Users can access their data and run jobs, but the system may need to be restarted if problems are found. 

Any jobs submitted prior to the outage were kept in hold but should be finished now. If you notice that your queued jobs have failed, or have new jobs failing, please contact

 Automount Introduction and No Result from “ls”

Grex now uses the auto-mount mechanism to enable /global/scratch/$USER on demand. Do not panic if “ls /global/scratch” does not list your $USER directory: it will be available after you access it in some way, like “cd /global/scratch/$USER”. 

New Directories & Quotas

To allow users to keep their old scripts (or programs) running the same way, the same path to /global/scratch has been implemented on the new system. Each user can find their data under the directory: 


The new default quota for /global/scratch is 1TB of data and 1 million files per user. Please note that having a large number of files may affect the file system performance of Grex, therefore, it is recommended that users archive directories (especially those with many small files) when they are done analyzing their data. 

The old /home directories remain as they were and were not migrated. For that reason, the space on /home has not increased, and the quota is limited to 30 GB. Please keep your /home directory use below this value to keep /home usable for all Grex users. 

It is recommended that all users run their jobs and analysis on /global/scratch and keep less data on /home.  The paths to the software and modules are still the same as before the outage. Please refer to the Grex Quickstart Guide for more tips on using this system. 

Missing / Corrupt Files

If you are missing files on the new /global/scratch/$USER the previous storage is still available, in read-only mode, on /ddn/$USER so you can inspect and copy any missing data yourself. 

Note that some files were already corrupted from outages during the last year, and were not transferred to the new /global/scratch. These files report question marks “????” instead of size in the output of “ls –l” command. 

If you have further questions or problems, please ask for help by sending an email to:

Thank you for your help and patience during the Grex outage and the test transition period.