You are here
Troubleshooting Tips: Data backups and storage options on Compute Canada systems
Compute Canada provides a range of storage options, from high-speed temporary local storage to different kinds of long-term storage, so that users can choose what best corresponds to their needs and usage patterns. The main Compute Canada documentation for managing your file storage and data backup needs can be found at:
To complement that information, below we've highlighted some of the most important things to consider in relation to backups.
- From the table at:
you'll note that files in the /scratch file system on Cedar are not backed up and files there will be deleted if unused for 60 days. Click on the tabs in the table to see the policies that apply for the various Compute Canada clusters. It is your responsibility to delete unneeded files or to move files you want to save from /scratch to one of the other file systems, or your own computer.
When moving files between file systems, such as from /scratch to /project, to avoid quota problems it is important to pay attention to the details on the page about transferring data https://docs.computecanada.ca/wiki/Transferring_data. This is also discussed on the Compute Canada FAQ pages:
- Files in /home and /project are kept for the life of a project (but, details such as what happens to files belonging to sponsored researchers when an account is no longer needed or a sponsor withdraws support are not well defined at this time). An additional short-term backup of these files is made nightly, so, there is the chance to recover files for up to 60 days if you accidentally delete them.
- For files that are no longer actively being used, you could consider moving them to Nearline storage, as described at:
- Each file system has quotas (limits on file space and number of files) a shown at:
You can use the diskusage_report command on Cedar and Graham to see your current usage and the quotas. As indicated in the somewhat cryptic Footnote 3 under the table at:
"Project space can be increased to 10 TB per group by a RAS request. The group's sponsoring PI should write to technical support to make the request." Here "technical support" is firstname.lastname@example.org and RAS stands for Rapid Access Service, which is just the default level of access to Compute Canada resources. In this case, a "RAS request", means that no special justification is needed (compared to what would be required for a larger storage quota in an application to the annual Resource Allocation Competition). Finally, when making such a request, you should specify the system involved, such as the SFU-based storage facility accessible through Cedar or the University of Waterloo-based storage facility accessible through Graham.
- Another important page to read is the scratch purging policy at:
Email notices about files to be purged are sent out at the beginning of the month. Particularly if you plan to request an increased quota in /project to accommodate your storage or if you have terabytes of data to move, you will need to act promptly upon receipt of such notices.
If you have any questions about these tips, or about data backups in general, email email@example.com at any time.