Close
Sign in

Solent Research Challenge

Research Excellence Framework (REF)

Introducing the REF

The REF is a peer review assessment of the quality of research in UK Higher Education Institutions (HEIs). The review takes place every 5-7 years. It is run by the government's Higher Education Funding Councils and is used to determine the annual quality-related (QR) grant to UK HEIs.

The last REF was REF 2014. All the submissions and results for REF 2014 remain available to view. The next REF exercise will be REF 2021.

For enquiries about REF at Solent contact ref@solent.ac.uk.

How the REF works

The REF assesses research excellence through a process of expert review. The reviews are undertaken by panels of experts appointed to a Unit of Assessment (UoA). There are 34 UoAs, which are split between four main panels (A-D).

Each expert panel will review all of the outputs, impact and environment for the HEI in that UoA and will produce a quality profile. The overall quality profile of the HEI will reflect all of these UoA level assessments.

Example of a profile:
Example of a profile
UoA AFTE category A staff submittedPercentage of research activity in the submission judged to meet the standard for:
     4*  3*  2*  1*  u/c
University A  27.45  12  42  30  15  1
University B 55.6 14 25 53 8 0
Star ratings:
Star ratings
 LevelDefinition
 4* World-leading in terms of originality, significance and rigour
 3* Internationally excellent in terms of originality, significance and rigour, but which falls short of the highest standards of excellence
 2* Recognised internationally in terms of originality, significance and rigour
 1* Recognised nationally in terms of originality, significance and rigour
 Unclassified (u/c) Falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of the assessment

 

Only 3* and 4* results contribute to the allocation of QR funding.

The results are published at sub-panel (UoA) and main panel (A-D) level. The UoA profiles are reported as a percentage of the total submitted research activity deemed to have met each of the defined levels of quality. The overall profile comprises an aggregate of the weighted profiles produced for research outputs, research impact and research environment.

In a change to REF 2014 the weighting of the 3 different elements of the assessment exercise will be:

  • Outputs (60%)
  • Impact (25%)
  • Environment (15%)

Timetable for REF 2021

The REF timetable can be found here.

Key dates are:

Key REF 2021 dates
1 January 2000 - 31 December 2020  Underpinning research for impact (minimum 2*)
1 August 2013 - 31 July 2020  Assessment period for impact
1 August 2013 - 31 July 2020  Assessment period for environment data
1 January 2014 - 31 December 2020  Assessment period for outputs
Summer  - Autumn 2018  Publication of draft guidance and consultation on UoA panel criteria
Winter 2018-19  Publication of final UoA panel guidance and criteria
Summer 2019  HEI codes of practice approved by HEFCE
 31 July 2020  Staff census date
 November 2020  Closing date for submissions
 2021  Assessment year
 December 2021  Publication of outcomes
 Spring 2022  Publication of submissions and reports

Assessment

The REF assesses research excellence through a process of expert review, informed by metrics where appropriate. HEIs submit evidence of their research outputs, research impact and research environment to UoAs, which are discipline-based expert panels who conduct the review.

The role of metrics in research assessment:

HEFCE conducted an independent review to ascertain the extent to which metrics could be used in the assessment and management of research. This was reported in The Metric Tide report that concluded that metrics are not yet robust enough to replace peer review. The consultation proposes that metrics are used to inform peer review in some UoA (similar to REF 2014). The Forum for Responsible Metrics, established in 2016, produced advice in 2017 for HEFCE (and the other HE funding bodies) on the use of quantitative indicators in the assessment of outputs in REF 2021 (with further discussion planned later for assessment of impact and environments).

The Forum defines responsible metrics as using the following principles:

  • Robustness - basing metrics on the best possible data in terms of accuracy and scope
  • Humility - recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment
  • Transparency - that those being evaluated can test and verify the results
  • Diversity - accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system
  • Reflexivity - recognising and anticipating the systemic and potential effects of indicators, and updating them in response
Use of citation information:

Previously the following UoAs used citation information as an indicator in addition to expert peer review

Main panel A - All

Main panel B - UoAs 7, 8, 9 and 11

Main panel C - UoAs 14 and 16

Main panel D - None

Research Outputs

Peer review of research outputs will contribute to 60% of the HEIs' overall REF outcome.

For a research output to be eligible for submission ot the REF it should be considered against criteria such as:

  • it is a product of research: a process of investigation leading to new insights, effectively shared;
  • it was first made publicly available during the REF publication period (01 January 2014 - 31 December 2020);
  • it is authored / co-authored by a member of staff at hte HEI on hte submission census date;
  • it is a journal article, conference contribution, book, book chapter, physical artefact, exhibition, performance, digital artefact, web-content

In all UoAs individual staff may submit between 1 and 5 outputs, however, the overall submission must be an average of 2.5 outputs

e.g. UoA submitted staff = 20, so UoA submitted outputs = 50

  • A minimum of one output will be required for each eligible staff member employed on the census date
  • A maximum of five outputs may be attributed to individual staff members (including those who have left)
  • Data on the distribution of outputs across staff in the UoA, including staff who have left, will be provided to the sub-panels for consideration in relation to the assessment of the environment

Output portability: A transitional approach is being adopted whereby outputs may be submitted by both the institution employing the staff member of the census date and the originating institution where the staff member was previously employed when the output was demonstrably generated. 'Demonstrably generated' will be determined by the date when the output was first made publicly available. This applies to the whole REF 2021 period.

Open access: The REF Open Access policy requires outputs within the scope of the policy (journal articles and some conference proceedings) to be deposited in an open access repository as soon after the point of acceptance for publication as possible, and no later than 3 months after this date from 1 April 2018. Due to concerns around deposit on acceptance, a deposit exemption will be introduced from 1 April 2018 and remain in place for the rest of the REF2021 publication period. The will allow outputs unable to meet this deposit timescale to remain compliant if they are deposited up to 3 months after the date of publication.

Research Impact

Peer review, including review by research end-users, will contribute 25% ot the HEIs' overall REF outcome.

For a UoA to be eligible for submission to REF, it must have a minimum of 2 impact case studies. For every 15 staff added to the first 15 staff submitted a further impact case study must be submitted.

For a research impact case study to be eligible for submission to the REF it shoudl be considered against criteria such :as:

  • the underpinning research was produced in the submitting unit between 01 January 2000 and 31 December 2020
  • the impact took place between 01 August 2013 and 31 July 2020
  • the underpinning research is assessed as at least 2* in the excellence framework
  • the impact case study demonstrates 'reach and significance' supported by evidence

A template for case studies will be issued and will include mandatory fields to be completed including the following:

  • a title
  • summary of the impact
  • description of the underpinning research including key research insights which underpinned the impact, including reference to where the research was first made publicly available
  • details of the impact including how the underpinning research made a distinct and material contribution to the impact, and the nature and extent of the impact
  • documented evidence of the above
Criteria and definitions of the assessment of impact:
Criteria and definitions of the assessment of impact
LevelDefinition
4*  Outstanding in terms of reach and significance
3*  Very considerable in terms of reach and significance
2*  Considerable in terms of reach and significance
1*  Recognised in terms of reach and significance
Unclassified (u/c) Little or no reach and significance; or not eligible; or not underpinned by excellent research produced by the submitting unit

 

Impact in REF 2014 was defined as 'an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. Impact included, but was not limited to, an effect on, change or benefit to:

  • the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice or understanding
  • of an audience, beneficiary, community, constituency, organisation or individuals
  • in any geographic location whether locally, regionally, nationally or internationally'

Impact included the reduction or prevention of harm, risk, cost or other negative effects.

Further guidance will be issued by HEFCE on inclusion in REF 2021 of impacts on research or the advancement of academic knowledge within the higher education sector nationally or internationally, and impactsw on teaching or other activities with the submitting HEI.

Research Environment

Peer review of the research environment will contribute 15% to the HEIs' overall REF outcome.

For research environment to be eligible for submission to the REF it should be considered against criteria such as:

  • research degrees awarded - number of doctoral degrees awarded in each academic year of the assessment period to students supervised within the submitting unit (taken from HESA data)
  • research income - total external research income (grants and contracts) in each academic year of the assessment period in the submitting unit (taken from the Financial Statistics Return (FSR) returned to HESA)
  • research income-in-kind - estimated value of Research Council facility time allocated through peer review and used by staff in submitting units (data provided by HEFCE)

A template will be issued and will include mandatory fields to be completed including the following:

  • research strategy
  • people
    • staffing strategy and staff development
    • research students
  • income, infrastructure and facilities
  • collaboration and contribution to the discipline
  • impact
Criteria and definitions of the assessment of environment:
Criteria and definitions of the assessment of environment
LevelDefinition
4* Conducive to producing research of world-leading quality, in terms of its vitality and sustainability
3* Conducive to producing research of internationally excellent quality, in terms of its vitality and sustainability
2* Conducive to producing research of internationally recognised quality, in terms of its vitality and sustainability
1* Conducive to producing research of nationally recognised quality, in terms of its vitality and sustainability
Unclassified (u/c) An environment that is not conducive to producing research of nationally recognised quality

 

Staff selection

Working with the REF 2021 main panels, HEFCE will provide further guidance on identifying staff with significant responsibility for research. This will be published in the guidance on submissions and panel criteria. This guidance will not prescribe a fixed set of criteria that all staff would be required to meet, but will set out a 'menu' of what HEFCE would consider may be appropriate indicators of 'significant responsibility' for research.

Recognising that there are staff who have more significant responsibility for other activities, HEFCE will implement an approach whereby institutions, working with their staff and with guidelines, identify who is in scope for submission among staff meeting core eligibility criteria. HEFCE has defined the core eligibility criteria as:

Category A eligible:

  • academic staff with a contract of employment of no less than 0.2 FTE
  • employed by the submitting institution on the census date (31 July 2020)
  • whose primary employment function is to undertake either 'research only' (independent researchers only) or 'teaching and research'
  • have a substantive connection to the submitting institution
  • for staff on 'research only' contracts, the eligible pool should only include those who are independent researchers, not research assistants

Category A submitted:

  • describes the staff from the 'Category A eligible' pool who have been identified as having 'significant responsibility for research' on the census date

Where the 'Category A eligible' staff definition accurately identifies all staff in the submitting unit with a signficant responsibility for research, the unit should submit 100% of staff. Where it does not accurately identify all staff in the submitting unit who have significant responsibility for research, institutions will need to implement processes to determine this and document this in a code of practice, approved by HEFCE with advice from the Equality and Diversity Advisory Panel (EDAP).

  • staff who have left the HEI by 31 July 2020 will not be eligible to be returned in the UoA, except where outputs were 'demonstrably generated' at the submitting HEI
  • QR funding is allocated on the basis of the 'assessed quality' of the submissions (research outputs, impact and environment) and the submitted volume (FTE) of submitted academic staff who were in post on the census date and were eligible for submission based on the HEIs' Code of Practice
  • each HEI will decide which staff to submit in order to maximise both the quality (which includes the assessment of their research outputs) and the resultant funding (linked to submitted staff volume). The peer review panels do not assess the quality of individual staff.

The Code of Practice is drawn up at each HEI in accordance with guidance issued by HEFCE. The Code of Practice will set out the criteria for eligibility of staff for submission to the REF (staff volume) and the selection process for outputs. Each HEI's Code of Practice will be approved by HEFCE before implementation, to ensure decisions are justifiable and non-discrimintatory.

FAQs

Q: What is submitted in a UoA?

A: Five sections are submitted:

  • staff data
  • research outputs
  • impact case studies
  • environment data
  • environment narrative

Q: Who do I contact about the REF?

A: ref@solent.ac.uk

Q: Where can I find information about the previous REF?

A: http://www.ref.ac.uk/2014/

Q: Which UoA(s) will the University submit to?

A: Final decisions on which UoAs will be submitted to will be based on guidance following an internal and external peer assessment exercise

Q: How is the REF assessed?

A: Academic experts are appointed to REF UoA sub-panels and review all the research submissions, research users also sit on panels and provide input into the assessment of impact case studies

Q: What is the role of a main panel?

A: There are four main panels (A-D) who comprise a group of expert researchers representing a broad discipline area whose role is to lead and oversee the conduct of the sub-panels under their remit. They are responsible for setting the assessment criteria, working methods and consistency of approach and adherence to criteria across the sub-panels.

Q: What if there is a conflict of interest for a sub-panel reviewer?

A: Where a sub-panel member declares an interest they do not participate in the assessment of departments or institutions where it may be perceived that they could be open to external influences, for example; where they or a close family member are employed.

Q: How are research outputs submitted to the REF?

A: Guidance on submission formats will be issued in due course.

 

Page last updated on Friday 9 February 2018 at 5.39pm.

Accessibility

Change font size

Change colours

Text Only

Accessibility on the Portal

Cookies and Southampton Solent University

We need to place cookies on your computer to ensure your browsing experience with us is enhanced.

Please continue browsing as normal if you are happy for cookies to be placed on your computer.

Find out how these cookies are used and how to change your settings

Hide this message