ESRC initiative on Comparative Cross-National Research Methods
The ESRC initiative on Comparative Cross-National Research Methods aims to generate expertise and knowledge relevant to the methodological development of all types of comparative cross-national research. It calls for advances in methodologies as well as increased understanding of the implementation of best practice in this field.
Our project focuses on one of the most extensive sources of existing quantitative cross-national data: international social surveys. In particular, we consider questions on individuals’ values and attitudes, broadly interpreted, which are typically measured using several related questions (survey items).
Latent variable models
When multiple items are used to represent abstract and complex constructs such as attitudes and values, they are often analysed using the statistical method of latent variable modeling. This technique represents responses to the items as measurements of unobserved (latent) constructs.
The most widely used latent variable models are linear factor analysis models and their extensions. These are not fully satisfactory for survey data, because they treat the items as continuous variables. Typical survey items allow only a limited number of response options, and thus produce categorical variables which have only a fixed set of two or more possible values (which may be ordered or unordered; linear factor analysis is particularly inappropriate for the latter). Our project deals chiefly with latent class and latent trait models, which do treat items as categorical. There is a pressing need to develop and disseminate such tools if the investment in the design and data collection of cross-national surveys is to be matched by the sophistication and validity of the statistical analysis.
One of the key methodological challenges in international surveys is the question of cross-national equivalence of measurement. Essentially the issue is, does a survey question measure the same concept and in the same way in all countries? If it does not, respondents from different countries can give different expected responses even if they have the same level of the concept of interest. Lack of equivalence can potentially compromise any substantive cross-national comparisons. Yet it is quite plausible in surveys which cover many countries, perhaps because of cultural differences in how a question is understood, or variations in questionnaire translation. It has even been argued that measurements should be assumed non-equivalent by default (see e.g. Kohn 1987), in which case equivalence should always be demonstrated first of all.
It is difficult to address the question of measurement equivalence completely at the design stage of even the best-designed cross-national surveys. Thus, a major part of the evaluation of measurement equivalence needs to take place during the analysis of the completed survey data.
We are considering, in particular, methods which use latent variable models for this purpose. Equivalence is then examined by comparing models where measurement models for individual items do or do not vary between countries.
Overall project approach
In our project we are studying the use of latent variable models for categorical data in the analysis of cross-national surveys, to answer substantive comparative research questions and to assess measurement equivalence. The work draws on the statistical and methodological literatures in several areas, including design and analysis of cross-national surveys (Harkness et al. 2003; Saris and Gallhofer 2007); general latent variable modeling (Bartholomew and Knott 1999, Skrondal and Rabe-Hesketh 2004); factor analysis and structural equation modeling (Bollen 1989); latent class and trait models (Hagenaars 1993, Heinen 1996, McCutcheon 1987); closely related work in psychometrics and educational testing (where latent trait models are known as Item Response Theory models (van der Linden and Hambleton 1997) and non-equivalence of items as differential item functioning (Holland and Wainer 1993, van de Vijver and Leung 1997); and measurement equivalence and model selection. The project team combines substantial statistical and methodological expertise with extensive and ongoing substantive research and experience in the development of cross-national surveys. We are testing and illustrating the methodological research of the project by applying the methods to data in three substantive areas of interest.
- Bartholomew, D J & Knott, M (1999). Latent variable models and factor analysis. Arnold.
- Bollen, K A (1989). Structural equations with latent variables. Wiley.
- Hagenaars, J A (1993). Loglinear models with latent variables. Sage.
- Harkness, J A., van de Vijver, F J R & Mohler, P Ph (Eds.) (2003). Cross-cultural survey methods. Wiley.
- Heinen, T (1996). Latent class and discrete latent trait models: Similarities and differences. Sage.
- Holland, P W & Wainer, H (Eds.) (1993). Differential item functioning. Lawrence Erlbaum.
- van der Linden, WJ & Hambleton, RK (eds) (1997). Handbook of modern item response theory. Springer.
- McCutcheon, A L (1987). Latent class analysis. Sage.
- Saris, W E & Gallhofer, I N (2007), Design, evaluation, and analysis of questionnaires for survey research. Wiley.
- Skrondal, A & Rabe-Hesketh, S (2004). Generalized latent variable modeling: multilevel, longitudinal and structural equation models. Chapman & Hall/CRC.
- van de Vijver, F & Leung, K (1997). Methods and data analysis for cross-cultural Research. Sage.