The key points from the ARC / ANDS webinar on Data changes in ARC Funding Rules are listed below. ANDS has also provided a guide on filling in the data management section of your application.
A recorded copy of the webinar is available via the ANDS YouTube.
How the data management plan impacts on applications
ARC applications will now have a requirement for applicants to specific how they will manage their data, this statement of data management will not be assessed "but a good plan may help your application across the line". Putting this into context your data plan should be incorporated in the criteria in section C of "Research Environment" in this case dissemination of information, and in the "Feasibility and benefits" criteria. The ARC is not expecting a 10 page data management plan but instead a synopsis. Applicants should ask themselves what would the assessor want to see, not what would the ARC like to see. Outline how you will manage the data. So a statement of my institution will store it is not a good/well thought out plan. In writing up the plan researchers should consider how other researchers and the public (where appropriate) might use their data. Guidance and advice is available from ANDS.
Key points arising from the webinar
- The rules for data management are listed with the funding rules and supporting documentation (see below).
- The ARC is not yet mandating open access to data. However, it is increasingly clear that research policy internationally is driving for data from publicly funded research to be available wherever possible.
- The ARC is interested in the best ways to manage, store, disseminate and reuse data and recognises major differences between disciplines and between projects
- No weighting will be given for the data management plan in the application at this stage but it will contribute to the selection criteria.
- Assessors will receive information in the assessors handbook about things to look for with regard to data management.
- Assessors will be looking at data management plans from a discipline perspective and will be making an assessment based on discipline norms. If the norm for a discipline is for open data, and the plan says that data will not be shared, there needs to be a clear explanation of why this is not possible, or the data plan may be viewed unfavourably.
- The focus is on Storage, Access and Reuse. Are not interested in high levels of technical detail but are interested in where it will be stored, the principles of access and the rationale.
- The world is moving to open data. There is an increasing trend to publications linking to data. For example, PLOS ONE requires the data that supports publications to be available in an open access format as well as the publication.
- The costs of arranging open data are eligible costs during the project period but as with all costs will need to be justified. Costs could include the costs of servers, software and systems to provide an open access platform. On projects with substantial data requirements costs could also include data curation staff.
- A section on data management will be required in the final report. If researchers fail to implement the data management plan an explanation will be required similar to other parts of the project which go 'off track'
- The Assessors Handbook is being amended to include guidance to Assessors on data management.
Additional information (provided by the ARC in the presentation)
Discovery project funding rules DP A11.5.2 page 19
A11.5.2 Researchers and institutions have an obligation to care for and maintain research data in accordance with the Australian Code for the Responsible Conduct of Research (2007). The ARC considers data management planning an important part of the responsible conduct of research and strongly encourages the depositing of data arising from a Project in an appropriate publically accessible subject and/or institutional repository.
Selection Criteria Section C DP15 pages 14-15
Discovery projects instructions to applicants and frequently asked questions
3.12 What information am I required to provide in relation to Management of Data in the Project Description? In line with responsibilities outlined in the Australian Code for Responsible Conduct of Research (2007) and international best practice, the ARC has updated wording in relation to the management of data. The ARC does not mandate open data. However, researchers are encouraged to consider the ways in which they can best manage, store, disseminate and re-use data generated through ARC-funded research. The Project Description requires researchers to articulate briefly their plans for the management of data generated through the proposed Project. In answering this question researchers need not include extensive detail of the physical or technological infrastructure. Answers should focus on plans to make data as openly accessible as possible for the purposes of verification and for the conduct of future research by others. Where it may not be appropriate for data to be disseminated or re-used, justification may be provided. Further information and resources on managing data are available on the Australian National Data Service (ANDS) website.
3.13 Is it sufficient to answer the Management of Data section in the Project Description by noting that I will comply with my institution's requirements? No. Whilst the ARC recognises that some institutions may have infrastructure and/or processes in place for storing, managing and sharing data and that these are valuable resources, to take into account the differences that may exist between institutions, disciplines and research projects, researchers are encouraged to highlight specific plans for the management of their research data in this section. The Management of Data section in the Project Description aims to encourage consideration of ARC-funded research data at both an individual and institutional level, in accordance with the responsibilities outlined in the Australian Code for Responsible Conduct of Research (2007). Researchers, in consultation with their institutions, are best placed to consider the management and future potential of their research data. This approach allows individuals to take into account the differences that may exist between disciplines and research projects as well utilise institutional resources and support available. Details of compliance with institutional requirements should be included in this section, provided that they are supported by a description specific to the data arising from the individual research Project.
Research opportunity & performance evidence (ROPE)
The quality of Investigators involved in Proposals for funding with the ARC will be assessed on the basis of their Research Opportunity and Performance Evidence (ROPE), and other scheme dependent criteria.
ROPE was first introduced under the Discovery Projects scheme for funding commencing in 2011 and subsequently under all other schemes. It replaced the selection criterion of 'track record relative to opportunities'. ROPE was introduced to help provide a more realistic consideration of a researcher's capabilities and assist those who have had career interruptions for family and other reasons.
The ARC have now released information regading how they assess ROPE information which may be of assistance in the development of this section of your application. For further information go to the ARC ROPE Statement webpage
ROPE aims to ensure the assessment processes accurately evaluate an Investigator's career history relative to their current career stage, and considers whether their productivity and contribution is commensurate with the opportunities that have been available to them.
ROPE provides a framework within which the quality and benefit of achievements is given more weight than the quantity or rate of particular achievements. It considers working arrangements, career histories and personal circumstances and provides an acknowledgement of research performance given the opportunities available.
Assessing applications for the ARC
The quality of assessments is a key factor in ensuring the good reputation of the Peer Review process of the ARC’s National Competitive Grants Program. The ARC has received notification of inappropriate assessments through applicant rejoinders, research office correspondence and from members of the College of Experts. The following types of assessments were identified as inappropriate in recent scheme rounds:
- minimal text provided in the assessment (e.g. a one sentence statement)
- assessment text does not match the scores provided in the assessment (e.g. low scoring, but positive, encouraging text and vice versa)
- comments copied across multiple assessments or from previous assessments/scheme rounds
- assessment text indicates the assessor has not actually read the PDF
- scores are detailed within the text
- assessment text includes information that indicates an undeclared conflict of interest
- inappropriate comments, e.g. malicious or denigrating remarks. (Note: hard constructive criticism is permitted)
The following two concepts illustrate two key points of a good assessment.
1. Assessments show consistency between the text and score.
While assessment text is released to applicants for rejoinder, assessment scores are not – they are only visible to the ARC College of Experts. Consequently, it is important that text and score agree, as this allows the applicant to address weaknesses in the proposal in their rejoinder and provides clearer information for the ARC College of Experts to better understand the scores the assessors have assigned against the selection criteria. Positive text with a weak score can give the applicant a false sense that the reviewer supports their proposal and also does not provide an opportunity for applicants to learn how to improve proposals in the future.
2. Assessments give clear details on the strengths and weaknesses of the proposal.
This assists the applicant and gives the ARC College of Experts a clear understanding of the assessor’s arguments and how these underpin the scores assigned to proposals. This is immensely valuable as College members rely on the expertise of assessors to bring to light the merits and weaknesses of proposals.