This website requires JavaScript.
Avatar
Evaluation of books in the SSH
Abstract
Evaluation of Social Sciences and Humanities in Europe. Hcéres Colloquium Proceedings - Paris IAS, 16-17 May 2022 Session 1 "Books and Monographs" - Books in Evaluation

Books have several important functions in the social sciences and humanities. Monographs and edited books allow for presenting original research based on methodologies or forms of collaboration that the format of the journal article does not serve as appropriately. Books are also used to engage directly with society, both for professional communication with specific audiences and for enlightenment and discussions in interaction with more general audiences.

I will start by describing the different genres of book publishing in the SSH in more detail and then return to the possible problems with classifications and criteria that may arise whenever the authors of books in the SSH are exposed to research assessment.

Scholarly books

Publications are just as important for the presentation of results from original research in the SSHas they are in other areas of research. The difference is that both journals and books remain important for this purpose in the SSH (@sivertsen*patterns_2016). The edited book represents another way of organizing research collaboration in the SSH than the multi-authored article represents in the sciences. The edited book builds on collaboration on a shared topic, but authors are more independently responsible for their chapters. With its extensive format, the *scholarly monograph* – with one or more authors responsible for the work as a whole – allows for more comprehensive research methods and a synthesis of deeper or broader investigations. Books represent necessary forms of collaboration and investigation in the SSH and are needed as a supplement to journals. Hence, books are not disappearing from scholarly communication in the SSH (Engels et al., 2018), and a specialized segment among publishers is devoted to publishing them (Giménez-Toledo, 2020).

Books for societal interaction

SSH researchers respond to the interest of wider audiences when they publish books for other purposes than presenting original research. A survey of all academic staff of Norwegian universities showed that contributions to the enlightenment literature are more frequent in the SSH than in the other areas of research (Sivertsen, 2019). SSH research is about culture and society and the researchers interact directly with culture and society. I prefer the term ‘societal interaction’ instead of the often-used terms ‘social impact’ or ‘societal impact’ because the relation between responsible SSH research and society is not unidirectional (Sivertsen & Meijer, 2020). Books for societal interaction often represent a professional multilingualism among SSH researchers (SIVERTSEN, 2018). The authors use, or are translated into, domestic languages to reach wider audiences while their scholarly publishing often needs to be international.

The wider audiences of the books may be both professional and more general. The term ‘popularization’ is insufficient because the SSH also communicate professionally beyond academia in specific genres for specific audiences such as handbooks for target professions or specific areas of cultural life, reference books for public management and legal procedures, books contributing with evidence to topics of political and cultural concern, and textbooks for students. Such books follow disciplinary standards without necessarily presenting original research performed by the authors.

In addition, social scientists and humanists are highly visible in the literatures of enlightenment presented in the non-fiction departments of general bookstores and libraries, and in similar online information. An interesting genre that may encompass all audiences is the essay, which may appear both as an article and as a book. Knowledge-based essays written in a reflexive and critical intellectual style often emanate from the SSH. Such books can become widely translated classics. Widely used and translated are also illustrated books (coffee table books) that are based on knowledge, investigations and documentation originating from the SSH. Another example is that handbooks for tourists, diplomats, or business representatives are unavoidably based on facts, insights and language competences created by the SSH.

Classification problems

I have distinguished above between scholarly books, professional books, and enlightenment books. These distinctions are important for the assessment of books in the SSH because the purpose and genre of a book needs to be acknowledged. However, the categories may overlap in practice. The two purposes of books in the SSH, to present original research and to engage directly with society, may be fulfilled with one and the same book. Some scholarly books become widely read in society. Prins et al. (Prins et al., 2019) provides examples and suggest categorizing them as “hybrid” because they reach wide audiences. In my view, books in the SSH should be defined by genres (hence by the merits of how they are written in relation to their purpose) while audiences may be a secondary characteristic. Scholarly books may reach wider audiences while still following scholarly and disciplinary standards for presenting new ideas and results. These standards do not change to ‘hybrid’ when reaching a wider audience.

‘Non-scholarly’ and ‘grey literature’ are two other problematic terms because they do not indicate the purpose and genre of a book. Hicks (Hicks, 2005) classifies ‘the four literatures of the social sciences’ as: international journals, national journals, books, and the non-scholarly press. Instead of ‘books’, I would say scholarly books, and instead of ‘the non-scholarly press’, I would say professional books and enlightenment books to describe the wider spectre used for societal interaction. ‘Grey literature’ is a term used by librarians to classify documents published by noncommercial public organizations that perform or commission research. An example is UNESCO’s Flagship Global Reports, which are often based on the united efforts of research in the social sciences and humanities. Public reports are based on scholarly standards but written for societal use, and they may have the format of an edited book or a monograph. Reports are an important genre of societal interaction, particularly in the social sciences. To classify them as ‘grey literature’ is void of meaning in the context of research evaluation and funding. Publications deserve to be classified by their genre and purpose.

Evaluation of books in the SSH

Research evaluation and funding systems exist at all levels of the research system, from the treatment of individual researchers’ applications for positions, promotions or local resources up to national systems for performance based institutional funding and international sources of project and programme funding. In all such contexts, publications serve to document competences and achievements in research and the potential to perform future research successfully.

A problem for the SSH is that only scholarly publications, not the genres of societal interaction, traditionally play the central role in research evaluation and funding systems. The act of communicating with society is usually not evaluated as such. The reason seems to be that in most areas of research, scientists can make their results useful without communicating with society as authors. Scholarly publications for peers will suffice. The knowledge transfer is still there when, e.g., contractors build a new airport, a general practitioner advises her patient and prescribes medication, pupils are taught chemistry at school, small firms innovate new software, or a new satellite is launched. In the SSH, however, to fulfil the purposes of research, publications are not only used to report new results among scientists. As shown above, they are also used to interact directly with society, both with the general public and with authorities and professionals in specific societal and cultural sectors.

The contexts and procedures of research assessment often use filters and hierarchies as criteria to define which publications should be evaluated and how they should be valued. If only scholarly publications are considered, the genres of societal interaction will be filtered out. If they are included, but treated as secondary, a hierarchy is introduced. A hierarchy is present in research evaluation and funding systems whenever some features of the publishing patterns are systematically valued over other features. Filters exclude some publications while hierarchies may rank the included publications by genre, language, publication channel, coverage in a specific database, etc., without taking into consideration the individual qualities of each publication.

There are two types of filters. One type regulates the contents and length of the submitted documentation. As an example, CV’s and track records submitted with funding proposals to the European Research Council may be of maximum two pages each. If previously performed research, documented with scholarly publications, then take most of the space, a hierarchy is established versus other genres. The other type of filter is how the underlying data for bibliometrics is defined. An example is the limited coverage of the SSH in the journal indexing services of Scopus and Web of Science (Aksnes & Sivertsen, 2019). Both types of filters tend to prioritize articles in international scientific journals. A much-used hierarchy is then to rank the included journals according to prestige or citation impact. Such filtering and ranking may be reasonable, depending on the purpose of the evaluation and funding system, but it may also lead to a distorted picture of what researchers and their organizations try to achieve. Researchers in the SSH are vulnerable to such distortions because they publish new results in books, not only in journals, and because they often use other genres and domestic languages for interaction with society (Kulczycki et al., 2020).

Filters and hierarchies may marginalize or disregard book publishing in the SSH. Particularly threatening are regimes of assessment that do not make their criteria explicit but simply take certain data sources for granted or refer to ‘rules’ of length and inclusion. An overview of European countries showed wide differences in how books are treated in national research assessment systems (Giménez-Toledo et al., 2019). The general problem is that the SSH differ from the sciences regarding the purpose of books, how they are valued, and how often they are published compared to journal articles. This difference will not, and should not, disappear. A constructive strategy for the SSH is instead to make the evaluation and funding systems responsible for their filters and hierarchies and discuss the implications of their criteria: What happens to research in the social sciences and humanities and how they serve society if scholarly books, professional books and enlightenment books are not included and valued properly?

Bibliography
SIVERTSEN, G. (2018). Balanced multilingualism in science. BiD: Textos Universitaris de Biblioteconomia i Documentació, 40.
Sivertsen, G. (2016). Patterns of internationalization and criteria for research assessment in the social sciences and humanities. Scientometrics, 107(2), 357–368. https://doi.org/10.1007/s11192-016-1845-1
5/16/2022