WestGrid Resource Allocation Policy

Philosophy

WestGrid HPC resources should be optimized for high-performance computing, as they were designed. This will put the emphasis on larger jobs that could not be run elsewhere.

Resource Allocation Committee

The WestGrid Resource Allocation Committee (RAC) is charged with recommending rules for the distribution of computing and storage resources on the WestGrid high-performance computing network. Resource allocation will be done on the basis of research groups or projects. These will typically be based around a faculty member or large-scale projects.

A limited fraction of the resources at each site are set aside for "general use". There is a default allocation for everyone to access this portion of the system. A significant fraction of the resources are allocated by the RAC on a group/project basis. Each group or project is allocated a "share" of the system. Over time they will get the allocated share, but the instantaneous priority will be controlled by the queuing systems.
A small fraction of each system is set aside with fast access for code development.

Job Priority

Computing resource scheduling is done with a "fairshare" algorithm. Each research group is given a fairshare target. The priority of each user's jobs is calculated on the recent system usage in that group relative to that group's target. As the actual usage comes closer to the target, the priority will go down, to allow other users a chance. On the other hand, older system usage is faded out so that priority will rise again. However, users do not gain any extra priority from long periods of non-usage. Users who find that this does not give them enough compute time (which is to say their jobs wait too long in the queue) can apply for a special allocation with a higher fairshare target.

The total RAC allocations will amount to a major share of a system, with the rest left for the other users. The default allocation is a fairshare target of 2 percent of the machine. This is oversubscribed so users without RAC allocations may not be able to reach this target. Researchers with RAC allocations will be able to use the resources they have been allocated, and those allocations may total about 80% of the machine, so the share of the machine remaining for others is not enough to ensure that the others can actually obtain their default two percent.

Criteria

1. The most important criterion for resource allocation will be the quality of the research that is being proposed. (In order to assess this, the RAC will rely heavily on peer review by the funding agencies.)
2. The need for the computing resources.
3. Impact on other WestGrid projects
4. Suitability of the proposal to WestGrid resources.

Applying to the RAC

Users who believe that they need more than the default allocation may make a special application to the RAC by completing a RAC Application.

RAC allocations are given to researchers who can justify a need for more resources. RAC applications are evaluated for their scientific merit by the committee.

Allocations are given to "accounting groups", not to individual users. If your research group requires access to different systems or has multiple research projects please include all of this information in one application.

Storage is also a resource which needs to be carefully utilized. In general there are no quotas (yet) but each user is expected to keep his or her usage under control. Usage will be monitored and users can expect to be asked to reduce their usage if there is a problem. Users with extra large storage requirements will need to apply for a storage resource allocation.

Current Members of the Resource Allocation Committee

Seamus O'Shea (University of Lethbridge, Chair)
Dennis Salahub (University of Calgary)
John Chen (University of Calgary)
Noham Weinberg (University of the Fraser Valley/Simon Fraser University)
Arif Babul (University of Victoria)
Stanimir Bonev (Dalhousie University)
Ryan Brinkman (University of British Columbia)
Bart Hazes (University of Alberta)
Guohui Lin (University of Alberta)
Andriy Kovalenko (University of Alberta)
Glen Marshall (University of Alberta)
Rob Simmonds (WestGrid CTO)

 

 

---

Updated 2009-10-26.