Context

Books and outputs related to books (book chapters, reports etc.) are an important scientific output in the social sciences and the humanities. As reported in the previous session, therefore, books need to play a role in research evaluation procedures if the evaluation is to provide meaningful results. This opens the next question: how, then, are books evaluated? This is relevant for two reasons: First, evaluation is a time-consuming activity (if it is done properly) and, therefore, to avoid evaluating outputs twice, one could benefit from taking into account the evaluation used during the publication process. Second, evaluation is a complex exercise and studying how books are evaluated by those who publish them means benefiting from a few hundred years of experience in evaluating books.

So, by reflecting on how books are evaluated today and in the past, in different contexts, we are gaining insights in what are the roles of the book in scientific communication, what are ways how a book is constructed, how are publishers evaluating books and how are peers reading and assessing those books. We might also identify different types of books by the way they are treated by publishers, authors and reviewers (e.g., books for career advancements, books to a wider audience, books to bring different strands of works into a coherent bigger picture, books to present the state of the art of a topic etc.).

Obviously, like academic publishing and research in general, book publishing is constantly transforming. Currently, digitalisation is a hot topic as well as Open Access. Somehow, it seems that there is no link between the two topics and the evaluation of books, but still these themes are often discussed together. Generally, it seems that the evaluation of books (just as well as books in evaluations) is not scrutinised enough. While there is bibliometric research on book publishing (e.g., Chi, 2016; Engels et al., 2018; Gorraiz et al., 2013), other aspects of the links between books and evaluations are under-researched. Three aspects I want to put forward that seem to me of need for conceptual scrutiny: On the one hand, books are often discussed from the point of view of publishers’ prestige, thus committing the same errors as focusing on Impact Factor for journal articles. Second, there is a confounding of Open Access, digitalisation, and prestige when discussing the transformation of book publishing. However, very little attention is paid to actual commercial aspects of publishing, including the services a publisher is to provide, but also the demand of books by the general population or by professionals (Giménez-Toledo et al., 2019). Third, there is very little conceptual scrutiny on what Open Access means and how it can be achieved, i.e. there is a dichotomy between big publishers making a lot of money with scholarly publishing vs. all scientific output must be immediately freely available to everyone. However, it is worthwhile discussing whether it would be more advantageous to perceive this as a continuum rather a dichotomy. Such a discussion will also have to include reflections on the turn from “pay to read” to “pay to publish” and what that entails.

Publishers’ Prestige

Scholars publish books to present the outcomes of their research. Most often, research published in books reflects long-term projects and complex issues investigated from various perspectives. In many SSH disciplines, books represent prestige as it shows that the author(s) have spent considerable time and effort on the subject. Similarly, books are an important part of career advancement, which also works a little against the books as prestigious research outputs as every PhD is published as a (quasi)book. Still, books serve to prove that the author is an expert on the topic (and PhDs should be an expert on the topic of their thesis).

However, profiles of publishers and their reputation differ sometimes strongly across disciplines (Giménez-Toledo et al., 2013; Mannana-Rodriguez & Giménez-Toledo, 2018). Books can fulfil different roles; even a differentiation between academic books and non-academic books is sometimes difficult to make. Some publishers specialise on specific aspects of knowledge dissemination (e.g., communication of research results to the profession in local language vs. having strong editorial procedures and focusing on specific topics in an academic discipline). Evaluations of the books are necessarily different across those diverse roles publishers fulfil in knowledge dissemination. The publishers’ prestige also changes according to what specific function a book takes in the dissemination process. Therefore, assuming that the scrutiny of evaluation of manuscripts increases by reputation of the publisher seems a contested issue as reputation and evaluation varies across the functions books can take in the knowledge generation and dissemination process. What strikes in the discussion of prestige and Open Access is that it seems that prestige is perceived as a given (that actually also applies to journals but I think the situation regarding journals has changed in recent years due to the experience): a prestigious publisher is a prestigious publisher and a new one is not. However, while prestige takes time to build and is more stable than is functional, prestige is a result of merit. If a reputed publisher consistently publishes bad books, the prestige will drop; consequently, if a new publisher appears, consistently publishes good books, it will become a prestigious publisher. Therefore, the discussion on OA and prestige seems often to be weird because it is a discussion on something that will change according to practice, as a function of whether the publisher will provide useful services. Finally, regarding the role of books in evaluations, it is obvious that using the publishers’ prestige as a proxy of quality is very limited for the exact same reasons as the Impact Factor: it is an ecological fallacy. Not all books published by a prestigious publisher are good; and certainly not all books published by less prestigious publishers are bad.

Open Access and Digitalisation

Another interesting point lacking scrutiny is the non-discrimination of Open Access and Digitalisation. Digitalisation can very well happen without Open Access (and that’s a point that is indeed undisputed), but there is Open Access without Digitalisation. Books can be made available to the public without digitalisation. Very obviously, digitalisation helps a lot as it eases access dramatically (open as well as closed access), furthermore, the change that comes with digitalisation can be used to introduce simultaneously other changes, like Open Access. But confounding the two transformations might lead to a reductionist perception of forms that Digitalisation and Open Access can take. If this discussion also involves prestige, things become very blurry.

With regard to the SSH, it is interesting that Open Access seems to be conceptualised as “freely available anytime, anywhere”. The argument is that research is publicly funded, so it should be available to the public. Interestingly, this comes with the assumption that the product “book” is made without any price and that publishers don’t do anything. Instead of having the idea of available for free to anyone, anytime, it might also be considered that research needs to be available to the interested public, but for a small fee. This would render visible that libraries are a form of Open Access for printed sources. Of course, any way of making things available for free comes with paying for services the publishers provide. Such services are rarely discussed and prices governments are ready to pay for them are even less discussed. Digitalisation comes with changes in publishing practices, Open Access is the idea to make research results accessible. Both processes are separate and merit a separate discussion in order to have clarity of what we are negotiating. Instead of thinking Open Access rather than Digitalisation, it would be worthwhile to think first about Digitalisation and what services are provided by whom, and then what costs come with them. If that is settled, ways of making things accessible to the relevant audience can be discussed. Basically, after having discussed the changes to book publishing through digitalisation, we can reconceptualise “Library” with regard to those new ways of publishing. This would open new ways of negotiation with publishers.

With regard to the evaluation of books, it seems that it is often perceived that Open Access books are not scrutinised. It is important to acknowledge that Digitalisation and Open Access are not linked to peer review and quality assurance of books. Rather, books fulfil many different roles in knowledge production and dissemination. They therefore also have different requirements regarding accessibility (both from perspective of digitalisation as well as from open access: some books are clearly meant for being printed, some books are clearly meant to be sold, some books might be meant to be available digitally but sold physically). This also means that the manuscripts should be evaluated regarding their purpose. At the same time, not only since Open Access has become relevant and Digitalisation has become dominant, books are published without academic scrutiny. There have been publishers taking advantage of the need of scholars to publish their PhDs or books and published for a fee without further evaluation. Regardless of Digitalisation and Open Access, it is relevant to distinguish predatory publishing from academic publishing.

Open Access and publishing

This reflection leads straight to the next point: Oftentimes, there is a dichotomy between the good (Open Access, freely accessible for anyone anytime) and the bad (commercial publishers gaining ridiculous amounts of money from public sources). The academic is put in front of those two options and needs to decide. However, it seems to me more fruitful to understand the two as extremes of a continuum. There are several ways of making scientific research publicly available. The European approach to Open Access (Plan S) is not the only possible one, and not the first (Debat & Babini, 2019). The aspect of power is not to be neglected. In several meetings on Open Access with EC representatives in which I participated it was made clear by them that the Open Access agenda’s main goal was to regain the central role of Europe in the international research market as Europe has come under pressure recently. More interestingly, the EC envisages to allow funding for APC also for researchers outside the EU when it is needed in the context of Horizon Europe (e.g., for a special issue or an edited volume), knowing that “pay to publish” might be discriminatory. However, the funding is possible under the condition that the research presented (or the authors) support the “European Values”. While we might agree on the relevance of some basic values like non-discrimination etc., this policy raises several interesting issues: who controls? which values exactly? What if we want to learn how, for example, social security systems are organised across the world but cannot include contributions from autocratic states because they do not share the European values? If non-European researchers need the agreement of European funders or even the Commission to publish in relevant journals, this puts academic freedom quite into question and, ironically, includes a funny definition of “open” science (and the European value “non-discrimination” as well).

Because an important focus lies on keeping Europe on top regarding visible research output, it is not seen as a problem that the current agenda simply moves from a “pay to read” to a “pay to publish” model and that this not necessarily means that the public saves money because the focus is not on the prices for specific services or on monopoly but on whether the producer or the consumer of the content pays for the distribution (Armstrong, 2021). It is to be noted that the “pay to read” was always very easy to circumvent for researchers: a mail to the author was usually all it needed to receive a copy of the document. However, “pay to publish” is much more difficult to circumvent. There is the idea of waivers applied widely in the brochures of the Open Access publishers. But in many cases, they publish in “special issues” where, for example, one article in five is waivered. This is relevant for the SSH because SSH scholars usually have smaller funds and special issues can be on the topic of regions, such as Eastern Europe, and many colleagues from Eastern Europe might not be able to get the funds for the Book Processing Charge and are then excluded from the special issue, as I had to experience as an editor of such a special issue (see also Armstrong, 2021). The aspect of political power and control of Plan S is discussed and seen much more critically outside Europe than within (Albornoz et al., 2018; Debat & Babini, 2019).

Another aspect that is not often discussed is that not all research necessarily needs to be available to the full public. Mostly, research results are presented in a specialised manner only intelligible to specialists; they do not necessarily need to be available to the non-academic audience. Publishing is not just making things available. Publishing consists of several sub-services including editing, layout, making it available in print or on screen; but it also includes distribution, marketisation, target-oriented presentation etc. The importance of those aspects of publishing is not to be underestimated but is normally totally ignored by scholars. The fact that a text is written and available does not mean that the text will be found, read and understood. It needs to be put into context, presented, advertised etc. Not to forget audience-targeted presentation (and that includes writing: a text to the wider public must not be written in the same way as to specialist peers). Small publishers are often specialised in such issues. This is especially relevant for SSH disciplines: some SSH research addresses professional practitioners who need to be made aware of the existence of new research, some address the wider public where even more efforts are needed to reach out. These services are similarly relevant as whether the research is published in Open Access or for a fee. The pandemic has shown that the public is not interested in reading each research result, quite the opposite: they talked of the cacophony of science because some scientific articles said A while others said B, so how can both be scientific if A contradicts B? Open Access should not be confounded with dissemination to the wider public. This brings us back to the topic of evaluation: The Open Access agenda puts a lot of focus on availability of research instead of quality of research, quality of presentation and efficient dissemination of research. However, Open Access comes with changes as to how research is evaluated for different purposes because the publication process changes dramatically. It also changes incentives for authors and readers (Armstrong, 2021). These changes must be part of the policy making in Open Access and the focus on availability and prices needs to be expanded to a more holistic idea of the changes the idea of Open Access brings. Finally, Open Access and Digitalisation also come with potential changes in how to evaluate. This interlink between Digitalisation, Open Access, and evaluation needs further scrutiny going beyond printed books equal no review versus Open Access books open the possibility of “new” open peer review; or the opposite: printed book are prestigious, Open Access books are just pay for publish without any quality assurance. The processes are complex, but the issues remain the same: dissemination changes with technological change; dissemination entails commercial aspects and services change when technology changes; evaluation needs to take changes in publishing, dissemination and the societal needs into account. These are all distinct discussions and a clearer separation of the processes would help finding new options.

Conclusions

The evaluation of books undergoes radical changes, not only because the evaluation procedures evolve and concern more and more the book-oriented disciplines of the humanities (Guillory, 2005, p. 34). Rather, publishing as such is at a crossroad. The advent of Digitalisation, Open Science and, specifically, Open Access impacts how books are published and disseminated, but also how they are evaluated. In this short introduction, I have argued that the discourse on Open Access is too narrowly focused on who has to pay how much for research to be published. The scientific community should engage in a more systematic discussion on dissemination of research results to different audiences, the roles publishers can and should take and what the prices for specific services should be. The evaluation process of books must be part of this discussion.

The contributions in this session of the conference and its proceedings cannot tackle all those aspects in detail, but they offer specialist insights into some selected aspects, i.e. Open Access, reputation and evaluation; peer review of books; and the opportunities, risks and limitations of Open Access in the humanities.