MPhil in Programme Evaluation

BUS5037W Programme Evaluation Coursework
BUS5036S Dissertation (Programme Evaluation)
BUS5050F Dissertation (Programme Evaluation)

The degree consists of two parts: a 120-credit coursework component and a 60-credit research project.

The curriculum as a whole aims to teach students how to:

• Frame and tailor specific evaluation questions to a given programmatic context.
• Develop an evaluation design or assessment methodology suited to a particular set of questions.
• Adapt principles of research design and analysis to the specific assessment of a social programme’s implementation, outcomes and impact.
• Analyse, present and interpret evaluation data and findings to inform programme management and improvement.

We also provide students with an understanding of monitoring (tracking the progress of the programme) and programme theory (the way in which programmes change a problem or people).

BUS5037W and BUS5056W (Coursework year 1 and year 2)

The coursework is delivered part-time over two years via seven weeks of block release (3 or 4 weeks per year). Over the seven weeks of block release, students will cover various coursework modules, comprising a total of 180 hours of contact time.

It is a requirement that students pass all coursework modules to pass the coursework component. Details of the coursework modules are provided below.

Principles of Programme Evaluation

This module provides a systematic overview and introduction to theory-based programme evaluation and its methods. In this module, we work as a class under the guidance of an experienced instructor in evaluation to develop an evaluation approach for a community partner (usually a local Non-Government organisation). We focus on working with a ‘real’ client to understand the logic of programmes and how evaluation tracks this logic. As a class, we explore different evaluation questions and consider questions of programme integrity and strength. Students are also taught the principles of stakeholder relations‚ user-friendly client reports, and the ethics of programme evaluation.

Statistics for Evaluation

In this foundational statistics course, students are taught to identify and apply correct statistical procedures to the quantitative exploration of questions often important in an evaluation context. For example, did a group of programme beneficiaries significantly improve on an outcome after the introduction of an intervention? If so, was there a different rate of change in an intervention relative to a comparison group? Is there statistically significant evidence of a causal effect of the programme on outcomes of interest? The course is structured to scaffold students through the most basic descriptive analysis approaches and the fundamentals of inferential statistics (correlation, t-tests, ANOVA, and simple and multivariate regression with moderation). The course includes both theoretical content and practical laboratory sessions using the SPSS statistical package.

Advanced Quantitative Evaluation Design & Analysis

This module builds on the foundational statistics skills taught in Statistics for Evaluation and extends students’ knowledge even further to cover more nuanced statistical analyses and design elements suitable for more complex quantitative evaluation designs. Design elements include sample size calculations and how to deal with elements of selective uptake, clustering, or attrition in an evaluation design. Analyses approaches include statistical approaches suitable for analysing interrupted time series design, difference-in-difference, propensity score matching, and regression discontinuity designs. The course includes both theoretical content and practical laboratory sessions using the SPSS statistical package.

Qualitative Methods in Evaluation

In this module, students are introduced to qualitative approaches and their use in evaluation. Students are shown how to apply popular evaluation approaches that rely heavily on qualitative data. Sessions will cover the different qualitative frameworks and the types of questions that qualitative evaluation research can successfully address, the key types of qualitative data collection, how to develop appropriate questions, and the correct manner of conducting the interview. The seminars also cover the fundamentals of qualitative data analysis and the reporting of qualitative results.

Alternate Approaches for Complex Evaluations

In this module, we draw on experienced expert practitioners to introduce students to a wider range of different evaluation theories, approaches, methodologies, and views. As such, the module aims to familiarise students with a broad range of alternative methods and approaches that might be useful when dealing with real-life complexity in evaluation. Approaches covered include economic costing evaluation, developmental evaluation, outcome mapping, Africa-centric evaluation, participatory evaluation, realist evaluation and utilisation-focused evaluation.

Monitoring using Programme Theory

Monitoring refers to tracking the progress of a programme. To do this‚ we need to understand monitoring terminology and be able to track programme implementation and outcomes over time. In specific instances, we also need to know about local and global monitoring indicators or the monitoring requirements of funders as well as an appropriate (complementary) evaluation design. In this module, students learn how to develop and apply a plausible programme theory (or an explanation of what works and what does not work in a specific field) to the design and operationalisation of monitoring and evaluation frameworks. Students will learn how to produce appropriate indicators‚ measures and standards for specific programme outcomes and track programme progress against these. Additionally, students will be able to design a monitoring framework for programme implementation by formulating appropriate data collection questions for coverage‚ service delivery and programme organisation.

Research design for Impact Evaluation

In this module, we concentrate on what is required to build a causal (or programme impact) argument by means of research designs, in other words – what kind of research designs establish if the programme, and nothing else, caused observable changes in beneficiaries. Students are taught the different experimental and quasi-experimental impact evaluation designs and how to assess their strengths, limitations and applications in different evaluation contexts. Critical engagement with published evaluations in a combination with taught content provides students with a thorough grounding in core principles of good research and evaluation design.

At the discretion of the Head of Section, modules may be added or withdrawn.

Coursework Delivery Schedule

The coursework delivery schedule is revised annually to allow for 3 to 4 weeks of block-release teaching per year. All block-release teaching will be in person at UCT.

BUS5050H and BUS5150H (Research dissertation 1 and 2)

The research report aims to assess whether or not students have mastered the principles of programme evaluation and are able to apply these to a real-life programme and construct a written research report in the form of a 60-credit dissertation based on this application.

Students work closely with an external programme (or client) chosen by the course convenor. In consultation with their supervisor and the client, students formulate appropriate evaluation questions and levels, conduct a critical literature review, develop a programme theory and revise the programme theory following a series of client consultations, use the evaluation literature to check the plausibility of the programme, select the correct methodology for primary and secondary data collection and analysis, and then present the final design in a coherent class evaluation proposal, collect primary data for the evaluation (interviews, surveys, focus groups, etc.), analyse the primary data and write up the findings coherently in the form of a useful client evaluation report or mini-dissertation.