By Lynn Laidlaw, Public Contributor and
Elspeth Mathie, ARC EoE Researcher
Lynn: Opening my e mails one day I spotted one from a research journal (Research Involvement and Engagement) I had completed two reviews for, I opened it with a feeling of trepidation, what would it say? It turned out to be a link to the author's response to my review comments and the editorial boards decision, re-reading my comments, and the other reviews was informative. My comments had been taken on board and the editorial team had suggested changes before the article could be published, I was relieved I hadn’t broken the review process!
As I do so often these days I tweeted about my experience as a lay reviewer and had some lively debate with other public contributors. We broadly agreed that open review (where the reviewers are named and their review published alongside the research) was positive leading to honesty, politeness, care and carefulness, encouraging us to treat authors as we ourselves would like to be treated. I mentioned that I feel lay reviewers are dropped in at the deep end and I wanted someone to review my reviews to ensure that I was “doing it right “. Following this, Elspeth and I had a conversation and we tried to find out if there are any resources specifically for lay journal reviewers.
I started to reflect on this question, do we ask what resources and training non academics need when reviewing for medical and healthcare journals, are we worth investing in? I have thought for some time that I am continually torn between the side of me who has become more research literate and sees the need for structure, process and methods. The other side is the patient who has an emotional response to research and understands the messiness of lived experience, which side of me should comment?
How many public contributors have managed to access training in research critical appraisal, is our strength perceived to be our lack of knowledge and research naivety? I have completed reviews for two journals, both times I was sent the link to access the article and the online platform but not offered any training or assistance on how to navigate said platform. This resulted in me losing a review and having to start the process from scratch, which was hugely frustrating. Lay, and academic reviewers, aren’t given any financial reward for their efforts, or offered expenses.
Reviewers need access to IT equipment and data to complete reviews, we bear this cost, we need to also consider the digitally excluded and those who choose not to be digitally enabled, do we provide alternative ways for people to contribute? Does this mean that only the people who can afford to review are given the opportunity to do so, how does this impact on the diversity of lay reviewers at a time when questions are being asked about inclusivity and equality in research?
Do we have a transparent process for appointing lay reviewers and should all journals that publish research, which impacts on people living with conditions, have lay review of their content? The irony that much research is published behind a paywall, which public contributors don’t have the institutional affiliations to access, is galling. Does this mean we can potentially review research for a publication, for free, and then not be able to access it when published?
Like many debates in patient and public involvement much of this concerns resources, training and recognition that public contributors have value and are worth investing in. We want to see excellent research published and be an integral part of that process. It’s time to co-produce some resources to assist lay reviewers in the task, provide expenses to cover digital costs and provide alternative ways to submit reviews. All journals should provide mechanisms whereby lay reviewers can access feedback on their reviews, especially when they are starting out. We are worth it!
Lynn Laidlaw is an experienced public contributor and is involved in Patient and Public Involvement and co-production across the UK.
Elspeth: I saw Lynn’s views on reviewing articles on Twitter and it is something I had also been wondering about recently. Including the voices of those with lived experience in the research process is now part of health and social care research, and this has now been extended to journal review. However, is this another role where Patient and Public involvement contributors are being asked for input but not necessarily offered support or reimbursement? Is it another area where those involved do not always get feedback on their comments? Previously, a Patient and Public Involvement contributor had reached out to me after reviewing a paper of mine to see if, and how, her comments had been taken into account in the final paper.
There are a couple of points in Lynn’s words that stuck out for me and continue to be well debated over the years; firstly, remaining ‘lay’ but also seeking research training and secondly, the diversity of PPI contributors (i.e. reviewers). Lay is a well debated term (disliked by many), what does ‘lay’ mean and who uses the term and for what purpose? What are reviewers being asked to do, review only from their own experience and not comment on the research design? In whose interest is it for Patient and Public Involvement contributors to remain ‘lay’ and be denied training? Support and learning and inclusive opportunities are now part of the UK Standards for Public Involvement. However, who provides support and training for those who want to review?
Some of you may know we carried out research into the lack of feedback for PPI contributors. PPI contributors were tired of sending comments and not hearing from researchers whether their comments were received, useful or used (Mathie et al, 2018). The reasons they wanted to receive feedback included motivation for further involvement and to assist their learning and development (see Guidance). There is clearly a need for some co-produced resources to support the peer reviewing process and an opportunity for shared learning? Secondly, those without digital skills or access to internet (for whatever reason, confidence, equipment or financially) will find it very difficult to take part in the process (see Blog). I have had to submit papers on behalf of Patient and Public Involvement co-authors as the system was too hard to navigate and use my own personal email account for a co-author who does not have an email address. At the moment the review process excludes a wide range of people with lived experience and knowledge to lay review.
As a researcher I am also asked to review papers, however, my motivations and rationale maybe different and I also have access to support and the practical means to participate (a work lap-top, technical support, email address). Reviewing is part of my job and career development, I might do it within, or out of, work hours but I am paid (as part of my job). It helps me keep up to date with new publications and provides insights for when I write papers and also helps me to learn how to deal with reviewer’s comments.
Regarding support, I have colleagues who are available if I want to run something by them (it was easier when we were in the same physical building!) and who can provide reassurance. One experienced colleague suggested I did not have to go off and read extra literature and it was okay not to comment on all sections of a paper. My colleagues provided informal support, going through the process themselves. When I started reviewing, I felt very similar to Lynn and wanted to know if I was ‘doing it right’ but see there is no one ‘right’, but, a variety of opinions and views. Whilst it is acknowledged that some Patient and Public Involvement contributors maybe part of a group and their Patient and Public Involvement lead maybe able to provide some support (more ‘invisible work’) many will not be linked into these networks and will often have to navigate the system alone.
I searched for on-line reviewer training and note that some journals do offer guidelines for reviewers (however, not specifically for PPI contributors). However, whilst training is important, support at the time when you using these new skills is vital. Whilst patient and public involvement is now an established (and reimbursed) part of reviewing funding applications, reviewing articles is a relatively newer activity and one we need to think carefully about. Lay reviewing is not reimbursed, to our knowledge.
Patient and public involvement contributors should be reimbursed for reviewing and to expect people to do this critical thinking, and give up time, for free is inequitable and unlikely to attract a wide range of reviewers. Who would fund this? How many lay reviewers currently carry out reviews and are their comments the same/different to researchers? Whilst, Patient and public involvement contributors acknowledge the importance of financial reimbursement the wider benefits of being involved must not be overlooked. Is reviewing for journals another example of PPI being fitted into a system which is not conducive to shared learning and co-production?
Elspeth Mathie is a Senior Research Fellow at the University of Hertfordshire and is Co-lead of the Inclusive Involvement in Research theme, National Institute of Health Research (NIHR) Applied Research Collaboration (ARC), East of England.
We would be interested to hear your thoughts about Patient and Public Involvement in journal review.
Follow @lynn_laidlaw and @elspeth_mathie on Twitter or email:
lynn.laidlaw@manchester.ac.uk and E.J.Mathie@herts.ac.uk
If you would like to share your views and experiences,
please leave a comment on this Padlet.
Acknowledgements:
We would like to thank Julia Jones, Richard Stephens and Sophie Staniszewska for their helpful comments whilst we were writing this blog.