On 1st June, the Association of Learned and Professional Society Publishers (ALPSP), along with the Publishers Association (PA), distributed a press release advertising the results of a survey that had recently been commissioned.
 |
| Audrey McCulloch |
The objective of the survey had been to estimate the likely impact of making journals freely available after a six-month embargo — as many advocates of Green Open Access (OA) have been proposing.
The results of the survey, the press release said, suggest that a six-month embargo would cause research libraries to cancel 65% of their Arts, Humanities and Social Sciences journals and 44% of their Scientific, Technical and Medical journal subscriptions. “ALPSP is very concerned about the effect this may have on non-profit publishers, many of whom may not survive,” commented ALPSP’s Chief Executive Audrey McCulloch.
The press release indicated that the report had been prepared by Linda Bennett of Gold Leaf.
Below is an email Q&A I had with McCulloch about the survey, and the report.
Survey
RP: I have a number of questions about the ALPSP survey undertaken in May 2012. I understand this was done by someone called Linda Bennett of Gold Leaf.
AM: Please let me have your questions. I’d be happy to help you where I am able.
RP: Can you say who or what Gold Leaf is, who Linda Bennett is, and why she was commissioned to do the 2012 survey? Was the work put out to competitive tender and she submitted the best proposal, or what?
AM: Goldleaf is the name of Linda Bennett’s consultancy. The survey started life as a question I wanted to know the answer to and wished to speak to a small number of librarians about. Linda works regularly with librarian groups so I contacted her to ask a UK sample of librarians the simple question you see in the full survey.
The responses that Linda received were concerning and I then wondered if this was peculiar to a small UK subset or if they were reflected in the wider library community.
RP: I believe the survey was jointly commissioned by the ALPSP and the Publishers Association? Was the Publishing Research Consortium (PRC) involved in any way?
AM: ALPSP and the PA together decided to extend the scope of the limited survey. PRC was aware but was not involved.
In order to ensure that we could compare the results of the wider survey with the responses from the original survey, the PA and ALPSP asked Linda to send out the same question as before but to a global sample of librarians.
RP: Bennett is herself Chair of the ALPSP research committee is she not?
AM: In both the original and wider survey, Linda clearly declared her position as Chair of the ALPSP research committee.
RP: To those she contacted yes. I believe Bennett is also the ALPSP representative on the steering group of the PRC, a publishers’ organisation. And she describes herself on the Book Fair web site as a field reporter for the PA. I am wondering whether — given the sensitivity of the issues surrounding self-archiving and Green OA, and the way the results of the survey have been used to promote the cause of the publishing industry — it might not have been better to hand the initial work that Linda had done over to an independent consultant, and to have commissioned them to do the follow-up study?
AM: The responses to the survey speak for themselves and are presented as an opinion piece. Linda was not commissioned to interpret the responses, only to report the answers to the question. The PA pays Linda as a field reporter for selected events, but this does not make her an employee of the PA and as an independent consultant she works for many other organisations.
We commissioned Linda because she has practical experience in conducting surveys. But the nature of the survey makes the contractor irrelevant. Using someone else would not have influenced the result.
I have also mentioned in our exchanges several times that I represent ALPSP on the PRC Steering Committee. Linda is an invited attendee.
RP: To confirm, the earlier survey was sent out to 34 librarians and the results were never published?
AM: As exactly the same question was presented to a much wider group of librarians, the responses of the earlier survey (conducted a few weeks previous), were combined with the latter and a single report produced.
Different conclusion
RP: It has been pointed out to me that the ALPSP commissioned a similar survey in 2006. That one was undertaken by the independent consultant Mark Ware, and published as the “ALPSP survey of librarians on factors in journal cancellation”. It seems that Ware’s survey came to a very different conclusion to Bennett’s. Why do you think that was so?
AM: The 2006 survey was six years ago. It is interesting to note the changes in the last six years, as open access publishing has evolved. It will be interesting to discover how different things are six years from now.
RP: In discussing why journals are cancelled, Ware explained in the 2006 report: “The three most important factors used to determine journals for cancellation, in declining order of importance, are that the faculty no longer require it (i.e. relevance to research or teaching programme), usage and price. Next, availability of the content via open access (OA) archives and availability via aggregators were ranked equal fourth, but some way behind the first three factors. The journal's impact factor and availability via delayed OA were ranked relatively unimportant...With regard to OA archives, there was a great deal of support for the idea that they would not directly impact journal subscriptions.”
In commenting on the survey OA advocate Peter Suber wrote at the time, “Bottom line: journals have much more to fear from their own price increases than from OA archiving.” You say things have changed in the last six years. What exactly has changed?
AM: 2006 was an important year in the development of publishing via Open Access, as evidenced by Peter Suber himself. Information about such changes and their understanding takes time to filter to all stakeholders. Attitudes towards, and opinions, of Open Access are evolving.
RP: You may know that Heather Morrison is a librarian at Simon Fraser University in Canada. She drew my attention to the earlier Ware survey and asked, “Since the 2006 study found that the first 3 most important factors in considering journal cancellation are needs, usage, and price (OA being fourth), why conduct a survey in 2012 omitting these factors?” How would you answer Morrison’s question?
AM: I have already stated that the question that was asked was one to which I wanted to know the answer; it’s a snapshot in time of librarian opinion and is presented as such.
RP: Morrison also points out that the typical journal cancellation process, as described by Ware, “follows a path of analysis, consultation, review and finalisation. The consultation may involve the librarian proposing candidates for cancellation, or providing data but asking patrons to suggest cancellations. It may also involve reader surveys of varying sophistication.”
Morrison then asks, “Since ALPSP conducted a study in 2006 which found that librarians do not make cancellation decisions without consulting faculty, why would they design in 2012 a survey asking librarians to respond Y or N to a question about cancellations?” She adds that librarians do not run universities, but act merely as a service point, and further asks, “Why ask people what decision they would make, when you know (or ought to know if you read your own research reports) that they do not have the decision-making power?”
How would you respond to Morrison on these points?
AM: We asked librarians their opinion and presented the results. That is clear from the report.
RP: The Bennett report was published three days after the PEER end of conference event. The PEER study looked at the question of self-archiving, and concluded that there is no evidence of any harm to publishers as a result of embargoed Green OA. Indeed, it found evidence that Green OA through the PEER project actually drives usage at the publisher site. Why do you think the conclusions of the Bennett survey are so different to those of the PEER group study?
AM: It is very difficult to compare the results of the large-scale PEER group study, which looked at the impact of green deposits on author behaviour, user behaviour and journal downloads, to responses to a single question posed to just librarians hypothesising a largely OA world, and I don’t believe they should be compared.
Opinion piece
RP: You say the 2012 report was presented as an “opinion piece”. I do not understand the implications of the distinction you are making. Both the press release and the document itself describe the document as a “report”. What is an opinion piece in this context and in what way was the document presented as an opinion piece? I.e. how would the reader have known that it was only opinion, not empirical evidence? After all, in the press release you said, “The responses in the report show that the ‘green’ model of open access will reduce the number of journals and thus choice available to academics.”
AM: The responses to our survey speak for themselves. We asked a single question of librarians, hypothesising a world of widespread Green OA on six-month embargoes. We did not control for any variables, so the survey stands as an opinion piece only. PEER, as is clear, was a four year longitudinal observatory. And the variables studied were different: user and author behaviour.
RP: Would you agree with those who argue that the PEER study is a more accurate predictor of the likely impact of Green OA and embargoed access than Bennett’s opinion piece?
AM: Probably not. We continue to believe that a high volume of articles available on short embargoes will undermine the subscription model.
RP: The PA’s Graham Taylor said of the Bennett survey that 1,000 research librarians were contacted and 200 replied. He added, “We don’t claim this to be definitive or statistically significant.” The report itself says, “The target respondents were not chosen at random. The aim was to obtain a set of representative responses from librarians at the different types of library served by academic publishers, while at the same time focusing particularly on obtaining replies from librarians at the world’s most prestigious academic libraries.” In fact, it was not just prestigious academic libraries, or even just research librarians, that were contacted, but schools, colleges, and corporate libraries too. Given the acknowledged limitations of the Bennett survey, what would you say to those who might argue that the results were oversold, and presented to the press in a somewhat hysterical manner?
AM: I disagree that the results were presented in a “hysterical manner”. We asked a question, and reported on the answers that we received.
RP: As a result of the press release you distributed about the report, the Times Higher Education (THE) published a news story suggesting that a six-month embargo would “bankrupt publishers”. THE cited the report saying, “Libraries would be impacted by the collapse or scaling down of academic publishing houses … Most publishers would be obliged to review their portfolios; and a substantial body of journals, especially in AHSS subjects, would cease or be financially imperilled.”
How would you respond to those who might argue that the publication of the report (three days after the PEER end of conference event) was done solely in order to undermine the PEER findings — which found that Green OA and embargoed access causes no harm whatsoever to publishers?
AM: The THE chose their own words, as journalists do. The reporter did not consult us. Journalists look for stories and put their own spin on reports. We did not release the results of our report to undermine PEER, such a conspiracy has never occurred to us. The timing was a coincidence.
RP: You say THE chose their own words and put their own spin on the story. In fact, as I noted, they quote the words used in the press release and the report itself. True, the report did not use the word “bankrupt”, but in the press release you yourself say, “ALPSP is very concerned about the effect this may have on non-profit publishers, many of whom may not survive.” Does that not imply that publishers will go bankrupt?
AM: Any business that provides a service and is not remunerated for that service (or is unable to obtain appropriate income from some other source) will not survive.
RP: Why did the 2012 survey not cite the PEER Group study, and why did it not refer back to the 2006 study?
AM: As I have said already, this is an opinion piece and has flagged an area of concern. It would be incorrect to compare it to the other studies.
RP: I understand that the 2006 survey is a membership benefit. Non-members are asked to purchase it for £90. As such, it sits behind ALPSP’s paywall. However, it did not use to be behind the paywall, or at least a summary of its findings was freely available here. The summary seems no longer to be available, even on the Wayback Machine. Can you say when and why the freely available summary of the 2006 report was removed from the ALPSP site?
AM: ALPSP upgraded their main database at the end of 2011. Our old and new systems were incompatible and much transfer work had to be carried out manually. The new URL is here.
Goldleaf
RP: It does seem regrettable that before it released its new survey ALPSP did not provide a URL redirect to the old one. However, can you point me to the web site of Linda Bennett’s consultancy and say how many consultants are attached to it?
AM: Goldleaf currently has no online presence.
RP: You do not say how many consultants work at Gold Leaf. Can you also say where it is based? I can find no details of any such an organisation anywhere, including at Companies House.
AM: Just one, Linda, but why does that matter?
RP: Maybe it doesn’t matter, but some might argue that presenting the 2012 survey as one undertaken by an organisation called Gold Leaf is not the same thing as flagging that it was produced by the research committee Chair of the ALPSP, an organisation that has a vested interest in the outcome, and which itself commissioned the survey. As you say, the sample email on Page 6 of the report indicates that Linda is the ALPSP research Chair. But anyone who read the press release alone would read only, “The report has been prepared by Linda Bennett of Gold Leaf.” Moreover, those reading the report might easily fail to make the connection between the author of the report and the author of the email question cited on page 6.
In addition, Bennett made a recommendation in the conclusion of the report that no mandate be issued requiring all or most journal articles to be made available free of charge after a six month embargo. Do you not feel that it might have been more transparent if Bennett’s affiliations had been made clearer, both in the report and in the press release? As it is, many are unaware that the author works for both the ALPSP and the PA, and that the recommendation came in effect from ALPSP and the PA, not from an independent consultant?
AM: The PA and ALPSP asked Linda to produce a report from the work she had done. Neither ALPSP nor the PA feel it is acceptable to make any changes to such reports unless they are being professionally typeset. I should point out that Linda does not work for either ALPSP or the PA.
RP: But as we noted, Bennett is Chair of the ALPSP Research Committee. If journalists and others reporting the results had known that, would they not be likely to have concluded that she had commissioned herself to write the report? And would they not have drawn attention to the connection?
AM: The survey was not carried out in Linda’s capacity as Chair of the Research Committee. I have already explained how the questioning came about; I wanted an answer to a question and Linda was able to help me discover the answer. Using someone else would not have changed the result.
RP: Let me stress that I am not for a minute suggesting that anything inappropriate occurred, or that the survey was not undertaken properly and responsibly. But I wonder whether the way in which it was carried out, the decision to have ALPSP’s own research committee Chair undertake it, and the dramatic way in which the results were released to the press — even though the findings are acknowledged not to be statistically significant, and the variables were not controlled for — would be likely to conspire to give the wrong impression, and encourage people to reach the wrong conclusions. Would you agree? If not, why not?
No comments:
Post a Comment