By Phil Goldman (Grievance Officer, QUFA)
Faculty going forward for renewal, tenure and/or promotion all confront the need to provide a list of potential referees who are “qualified” to assess their scholarship or creative work. The CA does not provide much direction on what to think about when making a list other than specifying how many referees are required for each personnel process and how many must come from the candidate’s list. Nevertheless, some broader understanding of the centrality of referee selection can be gleaned from the history and protocols governing RTP processes.
The first collective agreement QUFA and the University signed departed in significant ways from how RTP matters were dealt pre-certification. The aim was to eliminate as far as possible practices and biases that were believed to hinder fairness, equity, and scholarly diversity. They included the operation of personal hostilities, rigid methodological and paradigm disagreements, gender and racial bias, and a general “clubiness” test. Some of these dispositions remain, as evidenced by our experience that a large majority of RTP difficulties that come to QUFA emanate from women and racially identified candidates. But the provisions in the CA, including the centrality of external scholarship validation, have minimized the impact that these attitudes might otherwise have.
What follows is a short list of best practices that have informed the referee selection and assessment process.
- Your selections should be informed by how you define your area of expertise. It is therefore important for you to describe your scholarship and where it is situated in your research areas. Increasingly, we see candidates whose work overlaps a number of fields. However, to justify proposing one or more referees in areas that are outside the normal boundaries of work in your department requires some exploration of what it is you do. This can be done well or badly, so don’t slough this off.
- A potential referee’s title, e.g. Dean of …, doesn’t necessary qualify him or her as an expert. Think about it this way: is this a person to whom a journal in your field would send your manuscript for evaluation?
- Because of the importance attached to expert assessments of the content of your scholarship (hence the “journal” model), it is never sufficient to have a committee rely upon “proxies” for quality. Referees are expected to read what you submit and to comment on that. The number of publications, the putative standing of the publishers, the frequency of citations, etc., are not, by themselves, sufficient to constitute an evaluation of quality. If they were, there would be little reason to require “qualified” referees.
- While it may seem obvious, your referees should be amongst scholars who would make up your normal academic audience. For example, should your research interest be geographically defined as South Asia, then your referees should also be expected readers of your work by virtue of their work on South Asia or at least on cognate areas. One implication is that the relevant vehicles for publication may deviate from the standard list of top journals because of who your audience is.
- An RTP committee’s primary responsibility when it comes to research is to provide a full and fair reading of the referees’ evaluations. It is never appropriate for a committee to substitute its view of the quality of scholarship for those of the referees unless it provides clear and compelling reasons as to why those (one or more) assessments are unreliable.
- Some of the language in the CA dealing with faculty assessments is best characterized as “superlative”. “Excellent” or “distinguished” is a bit like A+ in grading. There is often considerable variation amongst faculty in awarding such high grades. So, too, is it the case when referees evaluate scholarship. Our experience is that referees from Britain or the Continent seem less inclined towards superlatives. Faculty might keep this in mind when selecting referees. Additionally, academic cultural differences highlight the importance of not relying on “magic words”, but rather on paying attention to the full content of submitted evaluations.
- We regularly confront issues of quantity versus quality. The CA speaks about quality. Nevertheless, there should be a sufficient quantity of scholarship for the referees to be able to make a reliable judgment as to quality. This need not take the form of many publications since there is considerable variation in numbers and length, usually related to fields of inquiry. Books, monographs, articles, research notes, etc., are differently valued in different areas so one cannot prejudge questions of quantity and length.
At the end of the day, the selection of your referees reflects what you are about. How you define your scholarship is an exercise in academic freedom, but it also carries with it a high level of academic responsibility. Your list of referees should be made up of scholars who are comfortable with your academic interests and whose academic credentials are substantial. Careful attention to their selection is second only to you presenting a solid academic file.
Until 2010, QUFA and Faculty Relations held joint sessions to explain the RTP process, including a discussion about referee letters. The 2010 sessions, Part 1 in June and Part 2 in September, were recorded and you can view them via links on this page where you will also find a couple Know Your Collective Agreement articles about the Renewal Tenure and Promotion Process, https://www.qufa.ca/know-your-collective-agreement/. Some of the earlier dates in the process have changed since then, but the principles guiding the process remain the same. You will find some useful documents on the Faculty Relations website, http://queensu.ca/facultyrelations/rtp.
You will also find links to handouts for the sessions that give an overview of the process and the presentations. Again, remember that the dates earlier in the process have changed and you should refer to the current collective agreement for the correct dates.
This Grievance Corner was taken from the July-August, 2013 edition of QUFA Voices.