sotonDH small grants: ‘A Connected Island?’: measuring academic influence

This second blog post about the Connect Island project, funded by a sotonDH small award, discusses the relative influence of Central European Palaeolithic researchers using the H-index measure.

Figure 1: H-index scores of Central European Palaeolithic researchers (left) versus Iron Age (right) researchers.

It has been claimed that Central European archaeologists specializing in Stone Age studies are quite well-known in the West compared to their colleagues leading research in later epochs. To test this anecdotal supposition we analysed the H-index of Central European Palaeolithic researchers.

The H-index (Hirsch 2005) is a measure of an author’s academic impact that takes into account both the number of papers published by the author and the number of citations to these papers (Bornmann and Daniel 2005; 2007). Its main advantage is that it balances the effects of a small number of high hitting papers and a large number of rarely cited publications. Neither a researcher with a one-hit-wonder paper, nor one producing hundreds of mediocre publications will score high. The H-index therefore favours enduring performance both in terms of quality and quantity. We used publications and citations recorded in Google Scholar as it covers a higher number of publications than ISI Web of Knowledge, especially for the fields of Social Sciences and Arts and Humanities (Kousha and Thelwall 2008). In contrast to ISI Web of Knowledge, however, Google’s bibliographic indexing is automated and not routinely manually edited by Google staff making it prone to inconsistencies and duplication. We noticed that the H-index results for archaeologists were unrealistically low when only taking publications in Web of Knowledge into account, and Google Scholar was therefore considered the lesser of two evils.

To provide a benchmark, we compared the results with a large sample of Central European Iron Age researchers. The Central European Iron Age is quite extensive, well-studied and some of its main proponents are well-known internationally. Arguably, the fact that we are using Iron Age researchers for this benchmark is irrelevant, any sub-discipline within archaeology would have done the job. In order for the anecdotal statement we are trying to test to be true, however, the H-index scores of the Palaeolithic researchers should be close to or higher than the Iron Age researchers’.

The results strongly confirm the intuitive observation (see Table 1 and Figure 1). Compared to a test sample of Iron Age specialists, Central European Palaeolithic researchers have been quoted more extensively and their papers were more influential abroad (as reflected in Google Scholar), indicating that they had a higher direct impact (as measured by the H-index) on the discipline globally.

Palaeolithic researchers		Iron Age researchers
Karel Absolon	9	Kazimierz Bielenin	4
Viola Dobosi	5	Anna Bitner-Wróblewska	2
Boleslaw Ginter	7	Éva Bónis	3
Jan Fridrich	4	Jaroslav Böhm	6
Bohuslav Klíma	10	Miloš Čižmář	3
Michal Kobusiewicz	8	Jana Čižmářová	1
Janusz Krzysztof Kozlowski	10	Sylwester Czopek	2
Stefan Kozlowski	9	Petr Drda	4
Gábori Miklós	5	Jan Filip	11
Martin Oliva	9	Kazimierz Godlowski	7
Romuald Schild	22	Eszter Istvánovits	2
Josef Skutil	5	Libuše Jansová	3
Jiří Svoboda	14	Fitz Jenő	9
Karel Valoch	14	Piotr Kaczanowski	5
László Vértes	11	Andrzej Kokowski	3
		Jerzy Kmieciński	4
		Valéria Kulcsár	2
		Karel Ludikovský	1
		Henryk Machajewski	2
		Renata Madyda-Legutko	3
		Magdalena Mączyńska	3
		Jiří Meduna	4
		Szabó Miklós	5
		Karla Motyková-Šneidrová	2
		Jerzy Okulicz-Kozaryn	2
		Emanuel Šimek	4
		Jaroslav Tejral	8
		Andrea Vaday	3
		Natalie Venclová	5
		Jiří Waldhauser	3
		Ryszard Wołągiewicz	2

Table 1: all Palaeolithic and Iron Age researchers included in the analysis with their H-index scores.

The Matthew effect?

We suspect that we are dealing here with a good example of the “Matthew effect” in science. Coined by Robert K. Merton (1968), the term refers to a passage from the Gospel of Matthew: “For to all those who have, more will be given, and they will have an abundance; but from those who have nothing, even what they have will be taken away.” – Matthew 25:29.

In simple terms it can be referred to as the “rich get richer” effect. Applied to academia it describes the phenomenon of more established, better-known scholars receiving disproportionately more credit than their lesser-known colleagues for equal or even smaller contributions to the research. Thus, they are more likely to spread their results wider and to have a higher impact on the discipline. Lower Palaeolithic archaeology had an additional boost when it came to creating a strong Matthew effect. The few irregularly distributed Lower Palaeolithic sites could be studied and published by only a handful of specialists. As a result, only a limited number of archaeologists were drawn into Palaeolithic studies and those who did were exempt from the fierce competition that their colleagues working on later epochs faced.

This also meant that invitations to conferences, scientific collaboration and co-authoring would be shared within a smaller cluster of scholars creating a self-propelling positive feedback loop and strengthening the natural Matthew effect. Combined with the nature of Palaeolithic data which is of global relevance and the high demand for Palaeolithic researchers in the second half of the 20th century, this could have contributed to a better recognition of Central European Palaeolithic researchers in the West, giving them more opportunities to collaborate, publish and spread their results in the international research community. Such a process could account for the higher H-index compared to their colleagues specializing in later epochs.

Bibliography

Bornmann, L., H.-D. Daniel. 2005. “Does the h-index for ranking of scientists really work?” Scientometrics 65 (3): 391-392. doi:10.1007/s11192-005-0281-4.
Bornmann, L., H.-D. Daniel. 2007. “What do we know about the h-index?” Journal of the American Society for Information Science and Technology 58 (9): 1381-1385. doi:10.1002/asi.20609.
Hirsch, J. E. 2005. “An index to quantify an individual’s scientific research output.” Proceedings of the National Academy of Sciences of the United States of America 102 (46) (November 15): 16569-16572. doi:10.1073/pnas.0507655102.
Kousha, K., M. Thelwall. 2007. “Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines.” Scientometrics 74 (2): 273-294. doi:10.1007/s11192-008-0217-x.
Merton, Robert K. 1968. “The Matthew Effect in Science.” Advancement of Science 159 (3810): 56-63.
Merton, Robert K. 1988. “The Matthew Effect in Science II. Cumulative Advantage and the Symbolism of Intellectual Property.” Sociology. The Journal of the British Sociological Association 159: 606-623.