Comparative Research Designs (2023)

Benoît Rihoux

benoit.rihoux@uclouvain.be

Université catholique de Louvain

Benoît Rihoux is a full professor of political sciencewhose research interests include political parties, new social movements, organisational studies, political change, and policy processes.

He is manager of the COMPASSS international research group on comparative methods, in the development and refinement of which he plays a leading role, bringing together scholars from Europe, North America and Japan in particular.

Benoît is a convenor of international methods initiatives more generally, and has published Innovative Comparative Methods for Policy Analysis: Beyond the Quantitative-Qualitative Divide(Springer/Kluwer, ed. with Heike Grimm 2006) and Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques (Sage, ed. with Charles Ragin 2009).

He has published extensively on systematic comparative methods (QCA in particular) and their applications in diverse fields – especially policy- and management-related – with interdisciplinary teams.

Course Dates and Times

Monday 17 – Friday 21 February 2019, 14:00 – 17:30 (finishing slightly earlier on Friday)
15 hours over five days

Prerequisite Knowledge

Little specific knowledge is expected. Prior training in qualitative and/or quantitative methods is of course an asset, but by no means a requirement.

You should simply be willing to reflect openly about your research design –there is no bestor one-size-fits-allapproach.

Short Outline

This course teaches you how to conceive and conduct the most appropriate comparative research design, broadly defined as any research enterprise that comprises at least two ‘cases’ or units of analysis. It answers fundamental questions, including:

  • What is comparison?
  • Why compare; what is the value of comparison?
  • What should be the mindset of a good comparative researcher?
  • What is the link between a research puzzle and the choice for a comparative research design – and what would the alternative(s) be?
  • At which level(s) should ‘cases’ be envisaged?

We will examine the practicalities of different types of comparative research, following these hands-on steps:

  1. prior arbitrations and ‘casing’, i.e. the definition of cases;
  2. case selection, through basic or advanced strategies;
  3. collecting and managing comparative data;
  4. comparative data analysis (qualitative, QCA and quantitative options).

The course alternates between lectures and interactive sessions, giving ample time for questions, open discussions, and solution-finding for your individual projects.

By the end of this course, you will know how to conceive and conduct the most appropriate comparative research design – the latter broadly defined as any research enterprise that comprises at least two ‘cases’ or units of analysis.

Tasks for ECTS Credits

2 credits (pass/fail grade):

  • Complete the short pre-course survey
  • Read the daily texts (see reading list)in advance of each class
  • Attend at least 90% of course hours
  • Deliver the fourdaily assignmentsfrom Monday to Thursday (to be delivered by noon the next day, Tuesday to Friday)

4creditsAs above, plus write up atake-home paper to be evaluated by the teaching team. The format, focus, evaluation criteria, submission deadline etc. will be explained on Days 1 and 5. There is some flexibility in terms of focus (more details on Day 5), with – among other possibilities – a paper laying out out all the main elements of one’s CRD, or a paper focusing on one specific step of one‘s CRD.

Long Course Outline

This course will teach you to how conceive and conduct the most appropriate comparative research design – broadly defined as any research enterprise that comprises at least two ‘cases’ or observations. It will cover fundamental questions upstreamof practical and hands-on choices:

  • What is comparison?
  • Why compare; what is the added value of comparison?
  • What are the logical underpinnings and mental operations behind comparison?
  • What should be the mindsetof a good comparative researcher?
  • What should be his/her goals?
  • What is the link between a research puzzle and the choice for a comparative research design?
  • What would be the alternative(s)?
  • Does one conceive and does one perform comparison in the same way when the ‘cases’ are situated at the micro (i.e. individuals), meso (e.g. organisations) or macro (e.g. political or policy systems) levels?

We will examine in detail the practicalities of different types of comparative research designs, by following all the hands-on steps:

  1. prior arbitrations and ‘casing’, i.e. the definition of the cases
  2. case selection, through more basic or more advanced strategies
  3. collecting and managing comparative data
  4. comparative data analysis (qualitative, QCA and quantitative options).

Steps 1and 2will be examined in greater detail. Each session allows time for open discussions and interaction.

Day 1
After introducingthe practical and organisational aspects of the course, we will frame comparative research in the broader context of a comparative approach. This means considering some epistemological issues underpinning comparison. Starting from the discussion of comparison as a basic mental operation, we will progress to comparison in the social sciences, then to political science specifically. One core focus will be on the different goals of comparison, with practical examples. To conclude, we will discussa first series of participants’ projects, focusing on the goals pursued (why go for a comparative research design?).

Day 2
We locate comparative research designs within the whole range of possible designs. We present the practical steps of a goodcomparative research design, focusing on the major arbitrations. We also have a first look at Step 1 operations that lie upstream of case selection, such as the formulation of the research question(s) and hypotheses, the correct use of concepts for the purpose of comparison, the number of cases one will be able to manage, and the choice between cross-country or within-country case selection. We conclude by discussing a second series of participants’ projects, with a focus on upstream arbitrations.

Day 3
We continue examiningStep 1operations, and deepen the question of 'what is a case?'within a comparative research design – with an emphasis on core arbitrations such as depth vs breadth and cross-country vs within-country vs within-system casing and case selection. Then we’ll systematically survey all the main options for the core Step 2operation: case selection. We first envisage rather basic or simple strategies of case selection, from very small-N to very large-N, and following different criteria; the pros and cons of each strategy will also be discussed. We conclude by discussing a second series of participants’ projects, with a focus on casingand case selection

Day 4
We turn to more refined strategies, in particular considering time/sequence and multilevel phenomena, and discussing thepros and cons of each. Then we look athands-on tricks of the tradefor collecting and managingdata in a comparative research (Step 3) – including ways to troubleshoot and to makecase selection adjustments as your research develops. A fourth interactive section around participants’ projects willfocus on case selection and data collection/management.

Day 5

We examine different ways to engage in comparative data analysis, envisaging three main families of options:

  1. case-oriented (or qualitative) analyses
  2. Qualitative Comparative Analysis (QCA) for systematic cross-case comparison
  3. statistical/quantitativeanalyses.

We will examine the pros and cons of each, as well as the potential difficulties of sequencing different data analysis techniques in a mixed- or multi-method design. Inparticular, we'lldiscuss the potential of sequencing QCA with single case studies, in small- or intermediate-N designs. In the second part of the morning session, we revisitsome core points – with a focus on the strengths of comparative research designs, but even more on main perils or caveats of comparison. You will become more aware of ways to mis-compare– andavoid it in your own research.

Finally, in an open interactive session, we discuss points still to be clarified, points of debate or disagreements, remaining questions and answers about participants’ projects, etc.

You are encouraged to bring your own research questions and hypotheses,first thoughts and difficulties (if any) in case definition and case selection, and (if applicable) any data you have already compiled. The course is designed to help you make the most appropriate choices in comparative research design. You will be able to reflect and to work on your own project as we follow the sequence of fundamental and thenapplied steps. Wheneverpossible, we’ll use input from participants’ own projects during the interactive parts of each one of the five sessions.

Connections with other courses (see also 'Courses before' and 'Courses after', below):

This course can be taken as a standalone course, but it has been designed as an introductory course, particularly for Summer School courses – in particular Methodologies of Case Studies, QCA and Fuzzy Sets and Mixed Methods Designs (exact coursetitlesmaychange).

Thisis not a specialistQCA course. Some main features of QCA (as an approach & set of techniques) will be presented atintroductory level, but if you want hands-on QCA training,follow Eva Thomann's week-longIntroduction to QCA course, or the two-week QCA course at the Summer School.

The course may also be of interest for scholarsengaged in ‘thick’ observational work (e.g. ethnography, participant observation, interviews) or in in-depth single case studies (using e.g. process tracing), as well as those interested in formalised or statistical approaches (large-N statistical techniques, experiments), especially if their populations and/or samples are not so obvious to circumscribe.

    Day Topic Details
    1 1. Overall introduction (30 minutes)

    Presentation of course, teaching team, course structure, practical organisation, assignments, logistics etc.

    1 2. Fundamentals: the comparative approach I (1 hour 45 minutes)
    • comparison as a mental operation
    • comparison as the basis of experimental science
    • comparison in the social sciences: ‘indirect experimentation’?
    • Mill’s principles in a nutshell: the Method of Agreement, the Method of Difference and the Method of Concomitant Variation
    • Types of comparatives studies and their goals in political science: from singlecases to statistics and from description to patterns
    2 3. Hands-on comparative research design, introduction (1 hour)
    • locating comparative research design(s) vis-à-vis other research designs (typology of research designs)
    • all practical steps of a goodcomparative research design: a bird's eye view of the full sequence
    1 Interactive session I (45 minutes)

    Discussing individual participants’ projects: Why go for comparison?

    2 2. Fundamentals: the comparative approach II (1 hour)
    • comparison as a ‘synthetic strategy’ between case-oriented and variable-oriented research, between Causal Process Observation (CPO) and Data Set Observation (DSO)?
    • comparison, taxonomy and typology-building
    • three contrastedgoals: co-variational analysis vs causal-process tracing vs congruence analysis
    • wrapping up: Why compare?
    2 Interactive session II (45 minutes)

    Discussing individual participants’ projects– upstream arbitrations

    3 4. Hands-on comparative research design, step 1, upstream (1 hour)
    • linking research questions to theories to hypotheses to potential ‘cases’; assessing validity and plausibility; how to make concepts ‘travel’ across different contexts
    • ‘What is a case?’ Choosing the appropriate level(s) of analysis (micro, meso, macro), defining the adequate units of analysis or ‘cases’
    • arbitration: the depth vs breadth trade-off – small-N, intermediate-N and larger-N designs
    • arbitration: cross-country vs within-country or within-system case selection
    • dealing with the time dimension and processes
    3 5. Hands-on comparative research design, step 2. Case selection – basic strategies (1 hour)
    • the single case study as a comparative research design? About counterfactuals
    • binary comparison (most similar cases)
    • binary comparison (most different or contrasted cases)
    • 'most similar systems' designs
    • 'most different systems' designs
    • global (large-N) designs
    4 6. Hands-on comparative research design, step 2Case selection – more advanced strategies (45 minutes)
    • more flexible designs – leaving case selection ‘semi-open’
    • sequencing 'most similar' and 'most different' systems designs (Levi-Faur etc.)
    • multilevel designs (Denk etc.)
    • nesteddesigns (Lieberman etc.)
    • using MSDO/MDSO as a support tool
    4 7. Hands-on comparative research design, step 3. Data collection & management strategies and fine-tuning of case selection (1 hour)
    • the challenge of collecting ‘comparable’ data across cases in cross-national research? How to gain ‘intimacy’ with the cases?
    • how to compile and manage your data? Lessons from the experience (archival, directories, text files, generic data management software, specialist data management software)
    • troubleshooting: what if the initial design appears too ambitious?
    • a critical discussion of some good examplesin political science literature (from steps 1 to 3)
    4 Interactive session IV (1 hour 15 minutes)

    Discussing individual participants’ projects: case selection (following) and data collection/management; wrapping up: good practice& tricks of the trade

    5 8. Hands-on comparative research design, step 4: Methodologies for comparative data analysis (1 hour)
    • case-oriented tools
    • comparative methods strictly defined: from ‘soft’ cross-case comparison to systematic cross-case analysis to variants of QCA (Qualitative Comparative Analysis) and Set-theoretic methods
    • statistical tools
    • comparing the strengths and weaknesses of each tool
    • triangulating, sequencing or mixing data analysis techniques to improve comparison? Some core mixed- and multi-method options.
    5 Interactive session V (45 minutes)
    • concrete comparative data analysis strategies in participants’ projects
    • pros and cons of different options
    • caveats
    • tricks of the trade
    5 9. Conclusion: adding leverage through comparison, and reflecting on how one performs comparison (45 minutes)
    • wrapping up: all the steps of a (successful) comparative research design, with trade-offs and decision points (checklist)
    • linking cases, theory and comparison
    • strengths of comparative research designs, revisited
    • limitations and caveats of comparative research designs, revisited
    • the perils of comparison I: back to concepts and how (not) to overstretch them. Sartori’s lessons – and beyond
    • the perils of comparison II: Galton’s problem, globalisation, diffusion and other nasty issues
    • the perils of comparison III: selection biasand why you should not be afraid of it
    • what should be the qualities of a ‘good’ comparativist researcher?
    5 Interactive session VI (30 minutes)

    Feedback on other projects; open discussion, points still unclear, points of debate, further practical questions & answers about projects, transversal issues, etc.

    3 Interactive session III (1 hour)

    Discussing individual participants’ projects: 'casing' and case selection

    Day Readings
    1

    Lijphart, A. (1975)
    The comparable-cases strategy in comparative research
    Comparative Political Studies 8(2): 158–177

    Lijphart, A. (1971)
    Comparative politics and the comparative method
    American Political Science Review 65(3): 682–693

    English-language version of: Aarebrot, F.H. and Bakka, P.H. (2003)
    Die vergleichende Methode in der Politikwissenschaft
    InBerg-Schlosser, D. and Müller-Rommel, F., (Eds.)Vergleichende Politikwissenschaft: Ein einführendes Studienhandbuch, pp. 57–76
    Wiesbaden:VS-Verlag

    Ragin, C.C. (2004)
    Turning the tables: how case-oriented research challenges variable-oriented research
    InBrady, H.E. and Collier, D., (Eds.)Rethinking social inquiry: diverse tools, shared standards, pp. 123–138
    Lanham, MD:Rowman & Littlefield Publishers, Inc

    Pennings, P., Keman, H. and Kleinnijenhuis, J. (1999)
    Doing research in political science. an introduction to comparative methods and statistics,Chapter 2, pp. 21–40
    London:Sage Publications

    Blatter, J. and Blume, T. (2008)
    In Search of Co-variance, Causal Mechanisms or Congruence? Towards a Plural Understanding of Case Studies
    Swiss Political Science Review 14(2): 315–356

    Excerpts from: Blatter, J. and Haverland, M. (2012)
    Designing Case Studies: Explanatory Approaches in Small-N Research
    Palgrave, ECPR Research Methods Series

    Excerpts from: Peters, B.G. (2013)
    Strategies for comparative research in political science: theory and methods
    Basingstoke:Palgrave

    2

    Pennings, P., Keman, H. and Kleinnijenhuis, J.(1999)
    Doing research in political science: an introduction to comparative methods and statistics,Chapter 3, pp. 41–71
    London: Sage Publications

    Berg-Schlosser, D. and De Meur, G. (2009)
    Comparative research design: case and variable selection
    InRihoux, B. and Ragin, C.C., (Eds.)Configurational comparative methods: Qualitative Comparative Analysis (QCA) and related techniques, pp. 19–32
    Thousand Oaks and London:Sage

    Goertz, G. and Mahoney, J. (2006)
    Negative case selection: The possibility principle
    InGoertz, G., (Ed.) Social science concepts: a user’s guide, pp. 177–210
    Princeton:Princeton University Press

    Peters, B.G. (2013)
    Strategies for comparative research in political science: theory and methods,pp. 58–79
    Basingstoke:Palgrave

    3

    Levi-Faur, D. (2006)
    A question of Size? A Heuristics for Stepwise Comparative Research Design
    InRihoux, B. and Grimm, H., (Eds.)Innovative Comparative Methods for Policy Analysis, pp. 43–66
    New York:Springer

    Denk, T. (2010)
    Comparative multilevel analysis: proposal for a methodology
    International Journal of Social Research Methodology13(1): 29–39

    Lieberman, E.S. (2005)
    Nested Analysis as a Mixed-Method Strategy for Comparative Research
    American Political Science Review99(3): 435–452

    Rohlfing, Ingo. (2011)
    Analyzing multilevel data with QCA: a straightforward procedure
    International Journal of Social Research Methodology

    Schneider, Carsten Q., & Rohlfing, Ingo. (2013)
    Combining QCA and Process-Tracing in Set-Theoretic Multimethod Research
    Sociological Methods & Research, 42(4), 559597

    Excerpts from Rohlfing, Ingo. (2012)
    Case Studies and Causal Inference: An Integrative Framework
    Basingstoke: Palgrave Macmillan

    4

    Excerpts from: Blatter, J. and Haverland, M. (2012)
    Designing Case Studies. Explanatory Approaches in Small-N Research
    Palgrave, ECPR Research Methods Series

    Excerpts from: Peters, B.G. (2013)
    Strategies for comparative research in political science: theory and methods
    Basingstoke:Palgrave

    Excerpts from: Landman, Todd (Ed.). (2003)
    Issues and Methods in Comparative Politics. An Introduction (2nd ed.)
    London: Routledge

    As further reading after the course, one book-length comparative study, displaying good practice, proposed by each course participant as a reference to feed group discussion.

    We will also discuss a selection of specialist textbooks and resources on data collection in connection with participants’ projects.

    5

    Rihoux, B. and Lobe, B. (2009)
    The case for qualitative comparative analysis (QCA): adding leverage for thick cross-case comparison
    InByrne, D. and Ragin, C., (Eds.)The Sage handbook of case-based methods, pp. 222–243
    London:Sage

    Berg-Schlosser, Dirk, Gisèle De Meur, Benoît Rihoux, and Charles C. Ragin
    Qualitative Comparative Analysis (QCA) as an Approach
    In Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques, edited by Benoît Rihoux and Charles C. Ragin, 1–18
    Thousand Oaks and London: Sage, 2009

    Sartori, G. (1991)
    Comparing and miscomparing
    Journal of Theoretical Politics 3(3): 243–257

    Ebbinghaus, B. (2005)
    When less is more: selection problems in large-N and small-N cross-national comparisons
    International Sociology20(2)

    Excerpts from: Goertz, G. (2006)
    Social science concepts: a user's guide
    Princeton:Princeton University Press

    Excerpts from: Goertz, G. (2017)
    Multimethod research, causal mechanisms, and case studies: an integrated approach
    Princeton: Princeton University Press

    Excerpts from: Pennings, Paul, Keman, Hans, & Kleinnijenhuis, Jan (Eds.). (1999)
    Doing research in political science: an introduction to comparative methods and statistics
    London: Sage Publications

    Excerpts from: Yin, Robert K (Ed.). (2003)
    Case study research: Design and methods (3rd ed.)
    London & Thousand Oaks: Sage

    0

    Among those readings, I particularly recommend you buy the following twobooks:

    • For an overall introduction to QCA, including the basic applied protocols:
      Rihoux & Ragin (eds, 2009)
    • For a discussion on strategies for comparative cross-case designs:
      Blatter & Haverland (2012).

    Software Requirements

    No particular software will be used intensively throughout the course, apart from the usual suites (such as MS Office).

    We will discuss the strengths and limitations of different software to compile, store and manage numerical and non-numerical data about a certainnumber of cases (from small-N to larger-N situations)– primarily Excel, Access, SPSS & NVivo, but these software packages will not be used hands-on in the lab.

    Hardware Requirements

    Please bring your own laptop. No specific technical requirements.

    Literature

    Further readings (recommended – not compulsory. A full, more extensive list of other related readings will be made available during the course):

    Bartolini, S. (1993)
    On time and comparative research
    Journal of Theoretical Politics5(2): 131–167

    Becker, H.S.(1998)
    Tricks of the trade: how to think about your research while you're doing it
    Chicago:University of Chicago Press

    Brady, H. and Collier, D.(2010)
    Rethinking social inquiry: Diverse tools, shared standards
    New York:Rowman & Littlefield Publishers

    Byrne, D. and Ragin, C.(2009)
    The Sage handbook of case-based methods
    London:Sage

    Della Porta, D. and Keating, M.(2008)
    Approaches and methodologies in the social sciences: A pluralist perspective
    Cambridge:Cambridge University Press

    George, A.L. and Bennett, A.(2005)
    Case studies and theory development in the social sciences
    Cambridge, MA:MIT Press

    Gerring, J. (2007)
    Case study research: principles and practices
    Cambridge:Cambridge University Press

    Mahoney, J. and Rueschemeyer, D.(2003)
    Comparative historical analysis in the social sciences
    Cambridge:Cambridge University Press

    Przeworski, A. and Teune, H.(1970)
    The logic of comparative social inquiry
    New York:Wiley-Interscience

    Ragin, C.C. and Becker, H.S.(1992)
    What is a case? Exploring the foundations of social inquiry
    Cambridge:Cambridge University Press

    Teune, H. (1990)
    Comparing countries: lessons learned
    In: Oyen, E., (Ed.)Comparative methodology: theory and practice in international social research, pp. 38–62
    London: Sage.

    Recommended Courses to Cover Before this One

    <p><strong>Winter School</strong></p><p>Foundations of set-theoretic and case-oriented thinking and methodology</p><p><span style="color:black">Philosophy and Methodology of the Social Sciences: A Pluralistic Framework</span></p><p><span style="color:black">Tools for the analysis of complex social system: an introduction</span></p><p><span style="color:black">Automated web data collection with R</span></p><p><span style="color:black">Introduction to Qualitative Data Analysis with Atlas.ti</span></p><p><span style="color:black">Introduction to NVivo for Qualitative Data Analysis</span></p><p>Advanced Multi-Method Research</p><p><span style="color:black">Introduction to MAXQDA, a Qualitative and Mixed Methods Data Analysis Software</span></p><p>Process tracing (introductory or advanced)</p><p><strong>Summer School</strong></p><p>Process tracing (introductory or advanced)</p>

    Recommended Courses to Cover After this One

    <p><strong>Winter School</strong></p><p>Methodologies of Case Studies</p><p>QCA and Fuzzy Sets</p><p>Mixed Methods Designs</p><p>Process tracing (introductory or advanced)</p><p>&nbsp;</p><p><strong>Summer School</strong></p><p>&nbsp;</p><p>QCA and Fuzzy Sets</p><p>Advanced Multi-Method Research</p><p>Process tracing (introductory or advanced)</p><p>&nbsp;</p>

    Additional Information

    Disclaimer

    This course description may be subject to subsequent adaptations (e.g. taking into account new developments in the field, participant demands, group size, etc). Registered participants will be informed at the time of change.

    By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach these prerequisite items. If in doubt, please contact us before registering.

    Top Articles
    Latest Posts
    Article information

    Author: Tish Haag

    Last Updated: 01/22/2023

    Views: 5619

    Rating: 4.7 / 5 (47 voted)

    Reviews: 94% of readers found this page helpful

    Author information

    Name: Tish Haag

    Birthday: 1999-11-18

    Address: 30256 Tara Expressway, Kutchburgh, VT 92892-0078

    Phone: +4215847628708

    Job: Internal Consulting Engineer

    Hobby: Roller skating, Roller skating, Kayaking, Flying, Graffiti, Ghost hunting, scrapbook

    Introduction: My name is Tish Haag, I am a excited, delightful, curious, beautiful, agreeable, enchanting, fancy person who loves writing and wants to share my knowledge and understanding with you.