Funding from the Canada Foundation for Innovation (CFI) and provincial partners is being used to replace several ageing systems with new national systems that will consolidate resources and centralize services. For more information on the national migration process and the 2017 Technology Refresh Update, please visit the Compute Canada website.
This page provides details specifically about migrating users and data off WestGrid’s legacy systems including any action WestGrid users need to take.
Thank you in advance for your patience during this migration period. We are working hard to limit the extent of disruptions or impacts to your research during this transition. If you have questions or concerns about any aspect of this migration process, please contact firstname.lastname@example.org.
Table of Contents:
- WestGrid Systems Defunding Schedule
- Online Office Hours
- How to Migrate your Data
- Getting Started on the New Systems
- Data Retention Policy for Defunded Systems
- Decreased Performance and Limited Support
- Support & Other Links
*** Note: “Defunded” means that the system is no longer funded, operated or maintained as part of Compute Canada’s national cyberinfrastructure platform, which is funded through the Canada Foundation for Innovation’s (CFI) Major Science Initiatives (MSI) program. After this time the system is no longer available for use by Compute Canada or WestGrid users.
Once a system has been defunded, local institutions may choose to keep a system running, at their own cost and discretion. This may result in usage limited to a certain group of users (e.g. only those from the local institution).
Users are encouraged to contact the Site Lead of the defunded system for further information.***
"Decommissioned" means the system is no longer in operation and has been retired from all further service.
Legacy Systems to be Defunded in 2018
|System(s)||Institution / Location||Defunding Date||What To Do...|
|Bugaboo||Simon Fraser University||
Jan. 31, 2018
March 1, 2018 Decomissioned
Update (Feb. 1, 2018): The decommissioning of the Bugaboo system has been delayed to the end of February 2018. The delay is necessary because the hardware expansion of the Cedar system has not arrived in time. For that reason we were not able to migrate groups with RAC allocations on Bugaboo to Cedar.
All groups without a RAC allocation on Bugaboo should have migrated to Cedar by now.
We will no longer install additional software packages on the Bugaboo system
|Grex||University of Manitoba||Mar. 31, 2018||
2017 RAC holders will be contacted directly to arrange data transfer.
|Orcinus||University of British Columbia||Mar. 31, 2018||
UBC will continue operating this system for any existing WestGrid account holder as of March 31, 2018 (no new accounts will be created after this date). Priority allocation will be given to UBC researchers. If you have questions, please email email@example.com.
Note: Limited warranty support after January 1, 2017. See below for details.
|Parallel||University of Calgary||Mar. 31, 2018||
System Use & Access After March 31:
Local Users (any researcher located at or collaborating with the University of Calgary): Please email firstname.lastname@example.org to clarify local use policies or to confirm if you qualify as a local user.
Non-local users (WestGrid users who are not based at or affiliated with the University of Calgary):
STORAGE ACCESS: Non-local users will be able to ACCESS any data stored on these systems until April 30, 2018.
COMPUTE ACCESS: Non-local users will NOT be able to submit or run any jobs on these systems after March 31, 2018.
If you have questions, email email@example.com.
Users are asked to move their data to a new system BEFORE the Defunding Date.
Any remaining data WILL BE DELETED from the system by the Data Deletion Date listed in the table above. Please refer to our Data Storage Policies for more information. Users should ensure they take the appropriate steps to comply with any data management requirements their institution or project may require.
Tuesday, March 6, 2018
10:30 - 11:00 am Pacific
Members of WestGrid's Support Team will be available to answer inquiries and help you troubleshoot issues on anything related to migration, including:
- how to prepare your data to be moved
- how to transfer files
- tips for code and job migration
- getting started on the national systems
* Note: No formal presentation is being given. This is an open Q&A session with WestGrid Support Staff.
To move your data off a system, follow these instructions:
- Delete any unneeded files.
- Prepare your data for transfer by archiving and compressing files.
- Read detailed recommendations on where to go for each of the to-be-defunded systems.
- Refer to the General Directives for Migration page on the Compute Canada Documentation Wiki for step-by-step instructions of how to transfer your data.
- Email firstname.lastname@example.org to notify us when you have moved your data.
Compute Canada's new national systems are now available for use. Instructions for logging on and getting started with the systems can be found here. Descriptions of the new systems can be found on the New Systems page of the Compute Canada Documentation Wiki.
As a rule-of-thumb those users located in the West (Manitoba to BC) should choose Cedar, and the rest should choose Graham. This has no particular technical justification but does give at least some guidance for the choice. For Cedar and Graham, the parallel filesystems, interconnects and external persistent storage is the same with a slightly different mix of compute nodes.
Note: WestGrid users will need to use their Compute Canada username and password to login to the new systems (the username and password used to log in to ccdb.computecanada.ca). See User Accounts and Groups on the Compute Canada Documentation Wiki for more information.
User data on defunded systems will be deleted. Users are responsible for migrating their data to alternate storage.
For defunded systems, the host institution assumes full control of the system, including managing the storage file systems and backups. Users should contact the Site Lead of defunded systems for further information about the host institution’s data retention and deletion policies.
IMPORTANT: Non-local user data on defunded systems will be deleted after the published deletion date. WestGrid has arranged with the host institutions to keep data until the published deletion date, but can make no guarantees about data retention after that date. WestGrid will not retain any long term or back-up copies of user data and as noted above users must arrange for migration of their data. Users should also ensure they take the appropriate steps to comply with any data management requirements their institution or project may require.
Please Note: Legacy systems scheduled for defunding are not being maintained under comprehensive vendor support programs due to the prohibitive cost. Only critical components (interconnects and shared storage) are maintained. No new (or replacement) compute nodes will be added which will likely result in a reduced number of available compute nodes and cores over time as components die. We also expect the reliability of these resources to decline and downtime to increase due to additional needs for maintenance and repairs. There may be significant outages. If jobs are lost due to an outage, they will need to be resubmitted.
As of January 1, 2017 the Orcinus storage system is not under vendor support. Due to the redundant nature of the storage controllers we feel that this is an acceptable risk. Orcinus will be backed up to the new storage system, the National Data Cyberinfrastructure (NDC) located at SFU and Waterloo. There may be some lack of continuity in backups as we transition between legacy and new tape systems, so we recommend users keep their own copies of critical data.
As of January 1, 2018 the Bugaboo storage system is not under vendor support. Due to the redundant nature of the storage controllers we feel that this is an acceptable risk. Bugaboo will be backed up to the new storage system, the National Data Cyberinfrastructure (NDC) located at SFU and Waterloo. There may be some lack of continuity in backups as we transition between legacy and new tape systems, so we recommend users keep their own copies of critical data.
Any user of a system in the table above will need to move any stored data and/or compute use to another system BEFORE the listed defunding date. **Please move your data well in advance of the defunding date to avoid network bottlenecks with file transfers.**
Please refer to https://docs.computecanada.ca/wiki/WestGrid_Legacy_Systems_Migration for detailed instructions and recommendations for each of the to-be-defunded systems.
Local institutions may choose to keep a defunded system running at their own cost and discretion. However, this may result in usage limited to a certain group of users (e.g. only those from the local institution). Researchers are encouraged to contact their local sites directly for further information:
- University of Manitoba: email@example.com
- University of Alberta: firstname.lastname@example.org
- University of Calgary: email@example.com
- University of British Columbia: firstname.lastname@example.org
- University of Victoria: email@example.com
All RAC 2018 allocations will be implemented on the new systems. Efforts will be made to get you on a system you are already familiar with. Please note, if you elected to Fast Track and your allocation was on a to-be-defunded legacy system, your 2018 allocation will be moved to a new system.
For more Frequently Asked Questions (FAQs) about migration, the national systems, and other Compute Canada services, visit the FAQ page on the Compute Canada Documentation Wiki.
Please email firstname.lastname@example.org for general help and questions at any point during the migration process, or refer to the following links for more tools, tips and support related to migration:
- Compute Canada Documentation Wiki - WestGrid Legacy Systems Migration
- Globus File Transfer - Background & Instructions
- Recommendations for use of new national systems
- Getting Started with the National Systems (mini-webinar series)
- Cedar System Details
- Graham System Details
- Code and job migration
- Available software on national systems
- Compute Canada's Rapid Access Service (opportunistic use of systems)