Prof. Dr. Katharina Anna Zweig
Network Analysis Literacy in an Algorithm Driven World
Complex network analysis has made its way into mainstream data mining and it is currently used by governments and companies to guide their decisions, e.g., regarding hiring and arresting. Algorithmic decision making can help to build a more objective world - but it can also be intransparent and create new injustice. In this talk, various results in complex network analysis are discussed which had an impact on policy - or are likely to increase their impact soon. It will be shown that especially the analysis of complex networks entails many modeling decisions which are not trivial and which might strongly influence its results. It is thus necessary to make sure that we watch our algorithms for any side effects they might have in applications we did not even dream of while designing them.
Katharina Anna Zweig is a biochemist (2001, Universität Tübingen) and computer scientist (2006, Universität Tübingen) who has analyzed and designed multiple algorithms and network models in complex networks analysis since 2003. She has just published a book called "Network Analysis Literacy" which focuses on the question of how and when to apply network analytic measures to answer a given research question. She is junior fellow of the German Society of Computer Science, has been selected as a "Digitaler Kopf 2014", is a member of the counseling board of the German Ministry of Education and Research (BMBF), and has co-founded algorithmwatch.org last year, a private think-tank to watch algorithmic decision making.
Prof. Dr. Loet Leydesdorff
On the Validity of Scientometric Constructs
The use and further development of scientometric/bibliometric indicators for research evaluation has become a quasi-industry. Because of the skewness of scientometric distributions, however, one can expect that most of the indicators are very sensitive for parameter choices. The model has an effect on the results in addition to other (e.g., measurement) sources of error. However, one always needs a model when comparing apples with oranges such as numbers of publications with numbers of citations. Averages or impact factors can be considered as simple models.
I will discuss the following case studies of model effects:
- Using the Leiden Rankings 2016, we studied Dutch economics departments. The correlation between our results and another ranking of the same institutes turned out to be negative;
- Carnegie Mellon University had rank 24 in the Leiden Rankings 2013 and rank 67 in 2016. This change in position is for more than 72% an effect of the model.
- Using a topic model or a co-word model we found, for example, Cramér’s V = .31 (p = .36) as the correlation between classifications. However, both topic models and co-word models are based on co-occurrences of words.
- The common practice of using the impact factor for the assessment of institutional or individual sets of papers implies an ecological fallacy. An impact fallacy is possible when using short-term citations for the measurement of quality.
Loet Leydesdorff is Professor at the Amsterdam School of Communications Research (ASCoR) of the University of Amsterdam. He is Associate Faculty at the Science and Technology Policy Research Unit (SPRU) of the University of Sussex, Visiting Professor of the Institute of Scientific and Technical Information of China (ISTIC) in Beijing, Guest Professor at Zhejiang University in Hangzhou, and Visiting Fellow at the School of Management, Birkbeck, University of London. He has published extensively in systems theory, social network analysis, scientometrics, and the sociology of innovation. With Henry Etzkowitz, he initiated a series of workshops, conferences, and special issues about the Triple Helix of University-Industry-Government Relations. He received the Derek de Solla Price Award for Scientometrics and Informetrics in 2003 and held “The City of Lausanne” Honor Chair at the School of Economics, Université de Lausanne, in 2005.