Submitting outputs as PDF or via URL
The REF submission system has the facility for institutions to upload outputs as PDFs, or to provide access to outputs via a URL. This page provides further guidance.
Journal articles and conference proceedings
For all journal articles and conference proceedings (type D and E outputs), the institution must provide Digital Object Identifiers (DOIs) wherever available. This will enable the REF team to access the published outputs directly via the publishers' websites. Research England will store these outputs and make them available to panel members for REF purposes.
There will be some type D and E outputs which it will not be possible for the REF team to access directly, and we will not attempt to source type D and E outputs flagged as 'pending publication'. In these instances the HEI will be required to upload a PDF of the output to the submission system. Reports will be available on the submission system identifying type D and E outputs that the REF team has not been able to source directly, and will therefore need to be uploaded by the HEI.
It is each HEI's responsibility to run the reports regularly and provide PDFs of all submitted type D and E outputs which cannot be sourced by the REF team. The REF team will not be contacting individual HEIs to request these outputs.
Other output types
Output types other than D and E will not be accessed by Research England and must be provided in electronic format by HEIs or sent to the warehouse. In addition to journal articles and conference proceedings that the REF team cannot source, HEIs are encouraged to upload other types of outputs as PDFs to the REF submission system, or provide access to the output via a URL as applicable to the output type. The table in annex K of the Guidance on submissions indicates which output types can be provided in these formats. Where there is a choice between providing an output in physical or electronic format, an electronic copy should be provided wherever the output is already available in this format. Where the output can be converted or scanned into a PDF of sufficient quality, within the file size limit, it should be provided in this way as described below.
Creating PDF outputs
Where the output is not already available as a PDF, a PDF of sufficient quality should be created while minimising the file size as far as practicable. A PDF of 'sufficient quality' means that it will be easily readable whether viewed on screen or printed, and provides an accurate representation of the output.
To minimise file sizes, the output should be converted directly to PDF if possible, rather than scanned. If this is not possible, the output should be scanned avoiding high resolution and colour as far as possible while ensuring the PDF is of sufficient quality.
A PDF output cannot be uploaded to the submission system if the file size exceeds 100MB (and we anticipate that most PDF files will be significantly smaller than this). If a PDF of sufficient quality cannot be produced within this file size limit, the HEI should submit the output in physical form instead, either in hard copy or in digital form (on CD, DVD or a USB).
Note that all outputs submitted as PDFs will be available for panel members to view electronically; however, individual panel members may choose to print the PDF to view as a hard copy, and this may be in black-and-white. This should be considered when producing PDFs.
Only one PDF may be uploaded for each output being submitted.
When to upload PDF outputs
HEIs should regularly run reports on type D and E outputs, and upload any such outputs that the REF team is unable to source, as well as any other outputs to be submitted as PDFs, as soon as practical throughout the submission phase. HEIs should not wait until the end of the submission period to upload large numbers of PDFs as this may cause strain on the submission system. Should the submissions system be adversely affected we may implement a queuing system for HEIs to upload outputs.
We expect to be able to retrieve outputs from the majority of publishers by July 2020. From this date, where the REF team has been unable to source the type D and E outputs within three weeks of the HEI listing the output on the submission system, the HEI should upload the output. This applies up to October 2020. For any type D and E outputs added to the submission during November 2020 that the REF team are unable to source within one working day, the HEI should assume the REF team will not be able to source these outputs and must upload the PDFs themselves.
All PDF outputs must be uploaded by the submission deadline on 27 November 2020, except for outputs 'pending publication' (outputs expected to be made publicly available between the submission deadline and the end of the publication period, 31 December 2020). Type D and E outputs flagged as 'pending publication' will not be sourced by the REF team. The institution must upload all PDF outputs flagged as 'pending publication' by 29 January 2021.
Depositing physical outputs
All outputs included in a submission that are not sourced by the REF team or provided electronically to the submission system must be provided as physical outputs for assessment. Physical outputs may be submitted in hardcopy form, in digital or other media on CD, DVD or USB, or as a combination of these.
Outputs deposited in digital or other media on CD, DVD or USB must be in commonly accessible formats. HEIs should ensure that digital files and media are free from viruses, worms, trojans and other digital threats.
Artefacts, devices, products and items of great rarity or value should not be deposited. A representation of the output and sufficient information about the research it embodies should be provided to enable the panel to assess it, without providing the item itself. However, institutions should be prepared to make the item available should panel members request to view it. In these instances the REF team will make the necessary arrangements with the institution.
Each physical output must be deposited as a separate item or package (whether in hardcopy, on CD, DVD, or USB or a combination of these). Multiple outputs must not be provided on a single CD, DVD or USB.
Where the same physical output, of any type, is attributed to two staff members in the same unit of assessment, the institution need only supply it once. If the institution submits the same physical output more than once in different units of assessments, a separate copy must be supplied for each unit of assessment, individually labelled.
Where reserve outputs have been identified for double-weighted outputs or outputs pending publication, these should also be deposited.
Further information will be provided in due course on labelling, packaging and delivery of outputs.
Uploading Corroborating Evidence
Corroborating evidence for Impact case studies needs to be uploaded by 29 January 2021. HEIs will be able to upload corroborating evidence before the REF submission deadline of noon on 27 November 2020, and the facility to do so will remain open until noon on 29 January 2021.
The corroborating evidence must be uploaded as a ZIP file. The ZIP file should contain one or more documents in PDF format. No other document formats should be included in the ZIP file. The ZIP file must be no larger than 100MB.
Each item of corroborating evidence should be appropriately titled and numbered according to section 5 of the impact case study template, to indicate the relationship to the specific claims made in the impact case study. Where the evidence can be freely accessed online, a URL can be provided either directly in section 5 of the REF3 template, or embedded in a PDF document and submitted as part of the corroborating evidence.
PDFs that will be uploaded as corroborating evidence should be of sufficient quality, and should where possible be created according to the same methods as for PDF copies of outputs - see Creating PDF outputs.