Other kinds of "impact" REF 2014

David Willetts speech-at UUK conference today was interesting for many reasons. The biggest 'impact' for me was where he was talking about what will be REF'able and how research will be assessed for the Research Excellence Framework 2014.
The most surprising thing was that you could include unpublished work and journal impact factors would have no relevance!


 "The REF is an assessment of research excellence. It is not about "journal rankings" or "journal impact factors". Whether the research output has been published in an open access journal, a traditional publication, or even if it hasn't been published in any journal at all, this does not affect the assessment of its quality in the REF. REF panels will not make use of “journal impact factors”, rankings or lists, nor the perceived standing of the publisher, in assessing the quality of research outputs. An underpinning principle of the REF is that all types of research and all forms of research output shall be assessed on a fair and equal basis by experts in the relevant disciplines." 

I wonder if this will actually be the case when the panel is assessing research. 
 From the REF Panel working methods document:

"REF2: Details of publications and other forms of assessable output which they have produced during the publication period (1 January 2008 to 31 December 2013). "

So they will be assessing publications, but I wonder what they mean by "other forms of assessable output"



"Panels have been instructed to define criteria and adopt assessment processes that enable them to recognise, and treat on an equal footing, excellence in research across the spectrum of applied, practice-based, basic and strategic research, wherever that research is conducted; and for identifying excellence in different forms of research endeavour including interdisciplinary and collaborative research, while attaching no greater weight to one form over another." p6

This sounds like a tricky job for the assessment panels if they are not allowed to use journal impact metrics. which to be fair are flawed anyway.  I'm wondering if they will be looking at any other forms of impact metrics out there at the moment, or do they just use peer review? 
Whatever happens they will probably change it again for the next REF equivalent, so we will have to start all over again. Watch out for the next crazy train.