Panel Session on methodological difficulties encountered in global surveys as evidenced in the Worlds of Journalism Study (WJS) IAMCR Conference, July 27-31, 2016 Leicester, UK ‘The methodological difficulties of a global survey’


Thomas Hanitzsch, University of Munich, Germany

The Worlds of Journalism Study (WJS) now comprises 60 countries. A study on such a scale inevitably throws up methodological as well as managerial challenges. The study has not only has to safeguard the validity, reliability and integrity of its individual samples, but also ensure that its conceptual design is globally meaningful and applicable. The Chair of the Worlds of Journalism Study outlines some of the complexities that have to be considered when working with dozens of different researchers on one project. Some nations, such as India, present many challenges in just one country when attempting sampling equivalence. The size of the country, the differences in accessibility conditions and the multitude of languages present the researcher with countless hurdles.  Another challenge presents itself in countries with controlled media environments where some of the assumptions, on which the survey is based, have difficulty taking into account the multiple, competing role perceptions held by journalists, as is the case in Ethiopia. However, there may be an entirely different way of approaching the comparative study of journalistic culture as is suggested in the last presentation, which puts forward an organic, multilevel model.

  1. Comparative Research as Managerial Challenge

The coordination and execution of large-scale multinational comparative projects can constitute a substantive managerial challenge. Based on experience from various cross-national studies in the field, this paper identifies different models of management and coordination in these contexts. In addition, it will point to some major challenges in the process of multinational coordination, including participation and transparency, funding and accountability, methodological preferences and cultural sensitivity, fairness and intellectual commitment, as well as data sharing and protection.

Thomas Hanitzsch

Department of Communication Studies and Media Research

University of Munich, Germany

2. Methodological and Practical Challenges of Large Scale Methodological Research

Large scale, cross-country comparative research has several benefits from creating a network of scholars who share different interpretive techniques to providing multiple benchmarks (country, region, continent, political systems/conditions, media systems) against which a country’s data can be evaluated for macro level practical and policy implications, both descriptive and analytic. At the same time, large scale, cross-country comparative research presents several hurdles that need to be overcome and, if that is not possible, recognized and used as contingencies within which data is interpreted. Among the major methodological challenges are establishing construct and linguistic equivalence, sampling equivalence, and equivalence of measurement unit and of testing conditions. Practical challenges, among others, include different accessibility conditions to populations of interest, logistics of travel and time for data collection, funding, translation costs and quality of translation, as well as cultural understanding in terms of approach, rapport building, time and space values, and so on. This presentation will present several examples of both methodological and practical challenges using India and Botswana as cases.

Jyotika Ramaprasad

School of Communication

University of Miami, USA

3. Cross-cultural challenges in comparative journalism research

The value of comparative journalism research depends on the study’s validity across borders. Although this may appear as common sense, there tends to be a lack of discussion of cross-cultural difficulties when addressing methodological concerns in various comparative global journalism studies. This paper seeks to address the issue in light of experiences with the latest Worlds of Journalism Study (WJS), referring particularly to experiences from the Africa chapter of the study, and in particular Ethiopia. The challenges encountered in the data collection were not limited to issues of semantics and translation of the questionnaire, but concerned underlying assumptions, which are characteristic of studies designed in a liberal, Western setting. For example, studies typically presume that professional roles and preferences are an expression of individual judgement, whereas journalists in controlled media environments may operate with multiple, competing role perceptions simultaneously. Besides discussing theoretical issues, the paper makes practical suggestions regarding comparative survey studies such as WJS. The author does not argue for major changes in the questionnaire format, but encourages researchers to pay more attention to cross-cultural issues in the discussion of research methodology when presenting research findings.

Terje Skjerdal

NLA University College

Kristiansand, Norway

4. The Organic Multilevel Model

This paper suggests an Organic Multilevel Model, a method for the analysis of the context influences of journalists through the application of multilevel models in comparative studies around the world.

The last decade has seen the expansion of comparative studies of journalistic culture beyond a single country or western journalism studies. These studies around the globe show that the conceptions journalists have within journalistic cultures are based on the context influences from the most important systems – economic, political, social, academic, media, etc. – and the technology levels within these countries. Therefore, this paper proposes the application of the Organic Multilevel Model, based on the concept of organic analysis of journalism, supported on the ideas of dynamism, change and heterogeneity that define journalistic cultures denominated as Intermediate Journalistic Countries.

The structure of this model changes the concept of superposed levels of previous models where journalists are at the center. This organic structure of analysis describes forms, methods and patterns of journalistic cultures as living systems, stressing the inter-dependence of the component parts, as well as its differentiation. In this structure, the “roots” (Systems level) are the structuring systems and the base of a country – those are political, economic, academic, social, mediatic and technological; the “trunk” (Institutional level) symbolizes the institutions and organizations, such as media organization, structures, processes, routines, editorial lines, rules and profiles of journalists as a group; and the “branches” and “leaves” (Actor level) represent the individual journalists, including the socio/demographic, economic and political factors, and professional/personal roles.

Martin Oller Alonso

Universidad de las Américas,

Quito, Ecuador