You are here
Case Study: Applications for the CANFAR network-enabled-platforms implementing Software-as-a-Service for Big Data Analytics
WestGrid user and University of Victoria Associate Professor Falk Herwig joins us to discuss some of the latest projects within CANFAR, the Canadian Advanced Network for Astronomical Research. As observational astronomy continues to become ever more data intensive, WestGrid and Compute Canada storage and computing resources are essential for driving much of CANFAR's work. Modern instrumentation on optical and millimeter-wave observatories produce 100s of gigabytes of data per night and radio telescope observing projects provide users with tebibytes of data over the course of a week’s observation. These RAW datasets are then processed through calibration systems that double their size and then further analyzed and sub-sampled into catalogues and mosaics of images, providing an additional doubling in size of processed outputs. Although the ‘RAW’ observations are stored by telescope archive facilities, the processed datasets are the responsibility of the principle investigator. Typically, large (survey) observing program consist of 50 or more nights per, resulting in a storage requirement of 5 to 10 tebibytes (TiB)of data per year.
CANFAR is an operational research portal for the delivery, processing, storage, analysis, and distribution of very large astronomical datasets. An innovative but challenging new feature of the research portal is the operation of services that channel the onslaught of telescope data through Canadian networks to the computational grid and data grid infrastructure (components of Compute Canada). CANFAR is currently used by many astronomy projects with users located at many institutions across Canada. These projects are using data generated by peer-reviewed allocations of a significant amount of observing time on three of Canada’s telescopes:
- the Canada-France-Hawaii Telescope,
- the James Clerk Maxwell Telescope,
- the Herschel Space Observatory,
- as well as data from other facilities such as the Hubble Space Telescope.
Initial and ongoing development of the CANFAR research portal is supported by grants under CANARIE’s NEP Program to the University of Victoria with additional support from the NRC Herzberg. The portal is operated and maintained by the Canadian Astronomy Data Centre (CADC).
Photo Caption: Next Generation Virgo Cluster Survey images superimposed on the night sky from Mauna Kea, Hawaii. NGVS is an international project using the Canada France Hawaii Telescope (seen in the foreground).