Compute Canada has begun one of the biggest advanced research computing renewals in Canada’s history. Funding from the Canada Foundation for Innovation (CFI) and provincial partners is being used to replace several ageing systems with four new national systems that will consolidate resources and centralize services. For more information on the national migration process and the latest update to the technology refresh program, please visit the Compute Canada website.
This page provides details specifically about migrating users and data off WestGrid’s legacy systems including any action WestGrid users need to take.
Thank you in advance for your patience during this migration period. We are working hard to limit the extent of disruptions or impacts to your research during this transition. If you have questions or concerns about any aspect of this migration process, please contact email@example.com.
Table of Contents:
- WestGrid Systems Defunding Schedule
- RAC 2017 Allocations
- Do I Need to Migrate?
- How to Migrate your Data
- Where to Migrate
- Future Use
- Data Retention Policy for Defunded Systems
- Decreased Performance and Limited Support
- Getting Started on the New Systems
- Support & Other Links
*** Note: “Defunded” means that the system is no longer funded, operated or maintained as part of Compute Canada’s national cyberinfrastructure platform, which is funded through the Canada Foundation for Innovation’s (CFI) Major Science Initiatives (MSI) program. After this time the system is no longer available for use by Compute Canada or WestGrid users.
Once a system has been defunded, local institutions may choose to keep a system running, at their own cost and discretion. This may result in usage limited to a certain group of users (e.g. only those from the local institution).
Users are encouraged to contact the Site Lead of the defunded system for further information.***
|System(s) & Location||Migration Period||Defunding Date||Data Deletion
(University of Saskatchewan)
|Complete||March 31, 2017||March 24, 2017||Complete. No action required.
User Migration off the Silo system was handled by WestGrid / Compute Canada staff.
|Nestor & Hermes
(University of Victoria)
|May 1-31, 2017||June 1, 2017||July 31, 2017||Local UVIC jobs ONLY after June 1. Storage access for any user until July 30. Refer to the Compute Canada Documentation Wiki WestGrid Migration Guide for migration instructions.
Contact firstname.lastname@example.org for local use policies.
|Breezy & Lattice
(University of Calgary)
|August 1-31, 2017||Aug 31, 2017||Aug 31, 2017||Refer to the Compute Canada Documentation Wiki WestGrid Migration Guide for migration instructions.
UofC researchers should contact email@example.com or information about accessing these systems after they have been defunded.
|Hungabee & Jasper
(University of Alberta)
|Sept 1 - 30, 2017||Sept 30, 2017||Oct 1, 2017||Refer to the Compute Canada Documentation Wiki WestGrid Migration Guide for migration instructions.
UofA researchers should contact firstname.lastname@example.org for information about accessing these systems after they have been defunded.
Users are asked to move their data to a new system during the Migration Period listed above and BEFORE the Defunding Date.
Any remaining data WILL BE DELETED from the system by the Data Deletion Date listed in the table above. WestGrid will not retain any long term back-up copies of user data. Users should ensure they take the appropriate steps to comply with any data management requirements their institution or project may require.
Legacy Systems to be Defunded in 2018
|System(s)||Institution / Location||Defunding Date||What To Do...|
|Bugaboo||Simon Fraser University||Mar 31, 2018||Wait for further instruction from the WestGrid Support Team. If you have questions, email email@example.com.|
|Grex||University of Manitoba||Mar 31, 2018||Wait for further instruction from the WestGrid Support Team. If you have questions, email firstname.lastname@example.org.|
|Orcinus||University of British Columbia||Mar 31, 2018||Wait for further instruction from the WestGrid Support Team. If you have questions, email email@example.com.
Note: Limited warranty support after January 1, 2017. See below for details.
|Parallel||University of Calgary||Mar 31, 2018||Wait for further instruction from the WestGrid Support Team. If you have questions, email firstname.lastname@example.org.|
The RAC 2017 allocations were implemented on WestGrid systems and the new Cedar system on June 30, 2017. Users should see their jobs running with their RAC 2017 priorities. WestGrid systems scheduled for defunding were not allocated, and will remain with their RAC 2016 priorities. Current jobs should continue to run during the RAC 2017 updates to the schedulers, however, due to complicated configurations required for the mix of new and old schedulers and the change of RACs some jobs may fail. Please contact us if there are any issues with current jobs terminating or new jobs failing to start.
Any user of a system in the first table above (Legacy Systems to be Defunded in 2017) will need to move any stored data and/or compute use to another system BEFORE the listed defunding date. Please move your data well in advance of the defunding date to avoid network bottlenecks with file transfers.
Please refer to https://docs.computecanada.ca/wiki/WestGrid_Legacy_Systems_Migration for detailed instructions and recommendations for each of the to-be-defunded systems.
We strongly recommend users make backup copies of important data as soon as the new National Data Cyberinfrastructure becomes available.
To move your data off a system, follow these instructions:
- Delete any unneeded files.
- Prepare your data for transfer by archiving and compressing files.
- Read detailed recommendations on where to go for each of the to-be-defunded systems.
- Refer to the General Directives for Migration page on the Compute Canada Documentation Wiki for step-by-step instructions of how to transfer your data.
- Email email@example.com to notify us when you have moved your data.
Detailed recommendations on where to go for each of WestGrid's to-be-defunded systems can be found on the Compute Canada Documentation Wiki. Please Note: The 2017/2018 resource allocation year (April 1, 2017 to March 31, 2018) will be a transition year with a mix of legacy and new systems available for use.
The table below lists the WestGrid legacy systems which will be available to WestGrid users until March 31, 2018.
Type of jobs
/home: 300 GB per user, 1M files
/global/scratch: 1 TB per user, 1M files/user
/scratch: depends on compute node
No backup. Files removed after job completion
Bugaboo is intended for jobs that require access to large storage systems (terabytes of data, for example).
More generally, the Bugaboo system can be used as a general purpose system appropriate for serial and parallel jobs and jobs that require long walltimes. There are a few nodes available that provide 48000 MB of memory - most nodes have either 16000 MB or 24000 MB.
/home: 250 GB per user, 250,000 files
/global/scratch: 500 GB per user, 200,000 files
Orcinus is intended for both serial and parallel applications.
/home: 30 GB per user, 500,000 files
/global/scratch: 953 GB per GROUP, 1M files/user
No backup. Long-term data retention not guaranteed.
Grex is primarily intended for applications that can take advantage of non-blocking low latency InfiniBand networks and large memory per compute node. It also hosts the WestGrid-wide license for Gaussian computational chemistry software. Grex is a general-use system so no special access requests required for registered WestGrid users.
/home: 50GB, 200,00 file limit per directory
/global/scratch: 450 GB, 200,000 files; email support to increase quota
Users are responsible for their own backups.
Files older than 30 days are subject to deletion.
Parallel is intended for multi-node parallel applications that can run in a relatively short time (less than 3 days) and can take advantage of its InfiniBand interconnect or special GPU-based nodes. It can also be used for applications that have license restrictions that prevent them from being run elsewhere.
Local institutions may choose to keep a defunded system running at their own cost and discretion. However, this may result in usage limited to a certain group of users (e.g. only those from the local institution). Researchers are encouraged to contact their local sites directly for further information:
- University of Alberta: firstname.lastname@example.org
- University of Calgary: email@example.com
- University of Victoria: firstname.lastname@example.org
User data on defunded systems will be deleted. Users are responsible for migrating their data to alternate storage. WestGrid will give as much advance notice of data deletion dates as possible, and will notify users of data deletion dates as soon as they become available. WestGrid will keep users of defunded systems informed about timelines for migration, and will provide support for the migration process.
For defunded systems, the host institution assumes full control of the system, including managing the storage file systems and backups. Users should contact the Site Lead of defunded systems for further information about the host institution’s data retention and deletion policies.
IMPORTANT: Non-local user data on defunded systems will be deleted after the published deletion date. WestGrid has arranged with the host institutions to keep data until the published deletion date, but can make no guarantees about data retention after that date. WestGrid will not retain any long term or back-up copies of user data and as noted above users must arrange for migration of their data. Users should also ensure they take the appropriate steps to comply with any data management requirements their institution or project may require.
Please Note: Legacy systems scheduled for defunding are not being maintained under comprehensive vendor support programs due to the prohibitive cost. Only critical components (interconnects and shared storage) are maintained. No new (or replacement) compute nodes will be added which will likely result in a reduced number of available compute nodes and cores over time as components die. We also expect the reliability of these resources to decline and downtime to increase due to additional needs for maintenance and repairs. There may be significant outages. If jobs are lost due to an outage, they will need to be resubmitted.
The University of Alberta would like to emphasize that as of January 01, 2017, hardware support for these systems is limited to critical components only. No new (or replacement) compute nodes will be added which will likely result in fewer available compute nodes and cores over time as components die.
As of January 01, 2017 the Orcinus storage system is not under vendor support. Due to the redundant nature of the storage controllers we feel that this is an acceptable risk. Orcinus will be backed up to the new storage system, the National Data Cyberinfrastructure (NDC) located at SFU and Waterloo. There may be some lack of continuity in backups as we transition between legacy and new tape systems, so we recommend users keep their own copies of critical data.
Compute Canada's new national systems are now available for use. Instructions for logging on and getting started with the systems can be found here. Descriptions of the new systems can be found on the New Systems page of the Compute Canada Documentation Wiki.
As a rule-of-thumb those users located in the West (Saskatchewan to BC) should choose Cedar-GP2, and the rest should choose Graham-GP3. This has no particular technical justification but does give at least some guidance for the choice. For Cedar-GP2 and Graham-GP3, the parallel filesystems, interconnects and external persistent storage is the same with a slightly different mix of compute nodes.
Note: WestGrid users will need to use their Compute Canada username and password to login to the new systems (the username and password used to log in to ccdb.computecanada.ca). See User Accounts and Groups on the Compute Canada Documentation Wiki for more information.
For Frequently Asked Questions (FAQs) about migration, the national systems, and other Compute Canada services, visit the FAQ page on the Compute Canada Documentation Wiki.
Please email email@example.com for general help and questions at any point during the migration process, or refer to the following links for more tools, tips and support related to migration:
- Compute Canada Documentation Wiki - WestGrid Legacy Systems Migration
- Globus File Transfer - Background & Instructions
- Recommendations for use of new national systems
- Getting Started with the National Systems (mini-webinar series)
- Cedar System Details
- Graham System Details
- Code and job migration
- Available software on national systems
- Compute Canada's Rapid Access Service (opportunistic use of systems)