Applying for VLSCI resources in 2016
The Facility currently operates at 855 teraflops. For information about the hardware itself, click here.
A team of skilled system administrators, programmers and application specialists are on hand to assist researchers to maximise their use of compute time and to get the most out of this significant resource. They are always evaluating the use of the systems, monitoring performance and suggesting improvements.
In 2016, VLSCI is moving to a more flexible, responsive resource application and allocation process. All Australian life science researchers may submit a ‘light-touch’ application at any time of the year, detailing their project, resource requirements, and evidence of project merit. Once approved for an account, most users may submit their jobs whenever they are ready, rather than in a specific quarter. This will smooth out some of the peaks and troughs experienced in the current quota system.
In Nov 2015, the 2016 VLSCI resource applications opened for existing 2015 users, for access from 1 January 2016. Submissions closed 11 Dec 2015. From then on, all applications are processed as they are received, for system access as soon as possible.
User categories for applications
Users are categorised as either high-end (over 200,000 CPU hours per year) or general users (expect to use less than 200,000 CPU hours per year).
High-end users, by definition, know what their requirements are and will be asked to give those details and will be allocated a specific share of the system to enable efficient scheduling of these big jobs.
General users, by definition, are less sure of their system requirements as the nature of the work makes it difficult to estimate both job times and resource needs. They will be allocated open access for reasonable use.
(Note: If users start in one category and wish to move across to another during the year, submitting a help request will initiate that change.)
What is new in 2016?
Two common concerns raised by past ‘general’ users was the difficulty of accurately estimating compute requirements and time-frames up-front, and the relatively infrequent resource allocation rounds. An extensive review of compute usage patterns over our five year history showed that most of these types of projects could be scheduled outside of a quarterly quota system, particularly with the extra capacity gained by the new x86 cluster ‘Snowy’. Some projects and researchers, of course, needed very significant amounts of parallel compute and these users were usually able to forecast their usage. For them, this forecast is important for ensuring these significant, finite resources are used effectively.
This new approach recognises these differing users’ needs and allows for users to shift to the alternative scheduling model if their needs change.
Usage monitoring by our system administrators is key, ensuring single users do not monopolise the resources for long periods without justification or in dealing with unforeseen bottlenecks. As always, urgent/unanticipated jobs will be accommodated where possible.
We are confident that these processes make applying for resources far less onerous and provide more flexible access to our systems. If problems occur, as always, users should make a quick call to the Facility Manager or submit a help ticket at any time.
Project management and reporting
- Applicants allocate personnel to various roles on an application for resources.
- Project Managers are responsible for keeping records of account holders on the systems up to date.
- Project Supervisors are required to submit one annual report on research outcomes (in February of the year following the year when systems were used).
VLSCI has limited data storage capacity. VLSCI data storage is intended only for current work, not long term archive. Data storage and backup capacity is becoming a serious problem and we need your assistance. By regularly removing your completed data from the system you make the system more responsive, minimise backups, and help us from running out of storage space. Could you please ensure all project members remove any data not needed for their immediate compute needs.
Costs of Access
There will be no charge for use of computational resources. Usage of data storage beyond that granted for a project may incur a charge.
There is some budget to install application software and datasets that are commonly used by the life sciences research community. Commercial software required by only one project may be purchased at the Facility Manager's discretion.
Users are expected to access the Facility via AARNet and charges are covered but users must pay for any communications charges billed directly to them from AARNet and any other data communications organisations that they may use.
PRINCIPLES GOVERNING RESOURCE ACCESS
The governing principles of the new application and allocation processes are informed by the requirements and priorities of VLSCI member institutes and our observations of how researchers use our systems. In summary, we are:
- simplifying the application and reporting processes to reduce the burden on researchers
- implementing flexible application submission and evaluation timelines that recognise the often unpredictable and intermittent demands for research-driven computing
- introducing a more flexible allocation process for the majority of our users
- maximising productivity of our systems by finding cloud and local cluster solutions for smaller jobs, if appropriate, and continuing to offer support and training to ensure jobs run efficiently
- prioritising resource requests from VLSCI member institutions (currently Melbourne, Deakin, RMIT and La Trobe Universities), and providing comprehensive, timely reporting for them
- opening our doors to life science-related industry customers.