PPOL5160 Evidence in Public Policy
Instructor
• PhD in Public Policy, Lee KuanYew School of Public Policy, National University of
Singapore, 2011
• Department of Asian and Policy Studies, The Education University of Hong Kong,
2011-2022
• Specializing in social policy, policy process theories, and health policy
• Associate editor, Policy & Society and Journal of Asian Public Policy
• The world’s top 2% most cited scientists in public policy and administration (2021
single year impact)
• One of the only two 4* (internationally outstanding) research impact cases in public
policy and administration in Hong Kong’s Research Assessment Exercise (RAR2020)
• Welcome faculty-student research collaboration and PhD supervision
Who am I?
• MPP core course
• Development of students’ professional skills in policy research
✓familiarize you with key research methodologies commonly used in public policy
research
✓enable you to formulate a general research strategy for a given policy problem
✓enable you to design and implement field work for policy research
✓enable you to conduct evaluation for a particular policy or program
• Prepare you, methodologically, for your PAE
• Gateway for possible doctoral training
Course objectives
• 3-hr lecture-based
• Illustrated with real-world examples
• Class discussion/interaction highly encouraged
• Most readings uploaded to Canvas
• PPT slides to be uploaded after class
• Attendance policy: Three un-approved absences
lead to failure.
• Seek prior approval from instructor and notify TA
Course delivery
Assessment policy
Class participation 10% Continuous
Quantitative policy analysis
(individual)
30% Due on April 5 (Group 1)
/April 2 (Group 2)
Qualitative policy analysis
(individual)
30% Due on April 26 (Group 1)
/April 23 (Group 2)
Impact assessment
(group-based)
20% Due on May 17 (Group 1)
/May 7 (Group 2)
Peer evaluation 10% Due on May 22
• Class #1 Introduction to policy research
• Class #2 Preparing for conducting policy research
• Class #3 Designing policy research: experimental designs
• Class #4 Designing policy research: quasi-experimental designs
• Class #5 Designing policy research: non-experimental designs
• Class #6 Quantitative methods: data collection
• Class #7 & Class #8 Quantitative methods: data analysis (to be delivered by Dr
Zhuoni Zhang via Zoom; two groups to be combined)
• Class #9 & Class #10 Qualitative methods: data collection and analysis
• Class #11 & Class #12 Impact assessment
• Class #13 Ethics and communication in policy research
Course outline
Class #1 Introduction to
Policy Research
• William Jenkins: a set of interrelated decisions taken by a political actor
or a group of actors concerning the selection of goals and the means of
achieving them.
• James Anderson: a purposive course of action followed by an actor or a
set of actors in dealing with a problem or matter of concern.
➢A policy problem may not always be an objective existence.
• Policy process is a problem-solving process.
• All policies are interventions in nature.
➢They want to change the status quo.
➢Bear in mind, maintaining the status quo is oftentimes a policy option too.
What is public policy?
• What is evidence?
➢Information that bears on the truth of a proposition compared to a standard.
➢Information becomes evidence only if it bears on the truth or falsity of the
proposition.
➢All evidence is information, but not all information is evidence.
➢Since we refer to evidence as information that supports a specific proposition by
bearing on its truth, evidence is context-dependent, because it becomes evidence
only by virtue of being relevant as support for a specific proposition, and
relevance is, by definition, a contextual concept.
• The effectiveness of social distancing and mass testing policies in the
COVID-19 response in China vs. the US.
PPOL5160 Evidence for Public Policy
The policy cycle framework
Agenda setting
Policy formulation
Decision making
Policy
implementation
Policy evaluation
Evidence!
✓The nature, size and dynamics of the problem at hand
✓Effective and ineffective interventions to solve the problem
✓The likely positive and negative consequences of the proposed policy option
✓The intended and unintended consequences of the proposed policy option
✓Effective and ineffective modes of delivery and implementation
✓How long the policy will have to run before positive results can be achieved?
✓The resources that will be required to implement the policy
✓The costs and benefits of the proposed policy, and on whom will these costs and benefits fall
✓The sustainability of the policy economically, socially, and environmentally
Evidence needed in policy process

• EBPM helps policy makers and providers of services make better decisions, and achieve
better outcomes, by drawing upon the best available evidence from research and
evaluation and other sources.
Evidence-based policy-making

Policies are not made in vacuum. Factors influencing
policymaking in government.
Adapted from Davies (2004)
Evidence
Experience,
expertise and
education
Pragmatics
and
contingencies
Resources
Policy content
Lobbyists and
pressure
groups
Habits, values
and traditions
Judgement
Political
affiliations and
objectives
Types of evidence
Evidence for
policy
Quantitative
evidence, e.g. cost-
benefit analysis,
statistics, surveys
Descriptive
evidence, e.g.
statistics, expert
opinion, anecdote
Experimental
evidence, e.g.
randomized policy
trials, pilot studies
Existing evidence,
e.g. meta analysis,
systematic review,
confirmed theory
Qualitative
evidence, e.g.
qualitative research,
observational
studies
Adapted from Productivity
Commission (2009)
• Policy research is the application of systematic information structuring,
data collection, and analysis to public sector decision-making.
• The function of policy research is to provide accurate and useful
information required in various stages in policy process.
• Understanding what decision-makers need to know, and why, is a
fundamental part of policy research.
• Policy research may also appeal to other stakeholders such as service
users, service providers, civil society organizations, and the general public.
What is policy research?
Policy research and policy process
Agenda setting
Problem structuring
Policy
formulation
Forecasting
Decision-making
Recommendation
Policy
implementation
Monitoring
Policy evaluation
Assessment
Policy research needed to support policy decisions
➢Yields evidence about the conditions giving rise to a policy
problem.
Problem structuring
(definition)
➢Supplies evidence about future consequences of acting on
policy alternatives, including doing nothing.
Forecasting
(prediction)
➢Provides evidence about the relative value of these future
consequences in solving or alleviating the problem.
Recommendation
(prescription)
Monitoring
(description)
Evaluation
(assessment)
➢ Yields evidence about the present or past consequences of
acting on policy alternatives.
➢ Provides evidence about the performance of the existing policy
in solving or alleviating the problem.
➢Scientific Skills
• Information structuring skills sharpen the analyst’s ability to clarify policy-related ideas and to
examine their correspondence to real world.
• Information collection skills provide the analyst with approaches and tools for making accurate
observations of persons, projects, or events.
• Information analysis skills guide the analyst in drawing conclusions from empirical evidence.
➢Facilitative Skills
• Policy skills refer to the knowledge of policy process.
• Project management skills concern with efficient and effective use of organizational resources to
accomplish research tasks.
• Policy communication skills concern with the effective communication of research findings to key
stakeholders.
Fundamental skills in policy research
Characteristics of research that policymakers find useful…
Research
quality
Conformity
to
expectations
Action
orientation
Challenge to
the status
quo
Impact
Source: Weiss (1980)
Where is policy research conducted?
➢Government agencies
• Ministries, Chief Executive’s Policy Unit (Hong Kong SAR), Congressional Budget Office (USA), Policy
Research Office of the CPC Central Committee, Development Research Center of the State Council,
The Korean Legislation Research Institute
➢Think tanks (semi-government and independent)
• Rand Corporation, Brookings Institution, The Rockefeller Foundation, Chinese Academy of Social
Sciences, China Institute of International Studies, Center for Policy Studies (UK), Institute for Public
Policy Research (UK), The Japanese Institute of International Affairs
➢Academic institutions
➢NGOs (including international NGOs)
• Red Cross, Green Peace, Oxfam
➢Private sector organizations: The Big Four, etc.
➢International and supra-national organizations
• World Bank, WHO, OECD, ILO, Asian Development Bank, ASEAN, APEC.
Key characteristics of policy research
Systematic Decision-relevant
Interdisciplinary Creative
Policy research vs. other types of research
Major objective Client Time constraint
Academic social
science research
Construct theories for understanding
society
“Truth” as defined by the
disciplines; other scholars
Rarely external time
constraints
Policy research Predict impacts of changes in variables
that can be altered by public policy
Actors in policy arena; the
related disciplines
Sometimes deadline pressure
Policy analysis Systematic comparison and evaluation
of alternatives available for public
actors for solving policy problems
Specific person or
institution as decision-
maker
Strong deadline pressure:
completion of analysis usually
tied to specific decision.
Media engagement
Translating policy research into impact
❖Mr Chris Sun, Deputy Secretary for Food and Health of the Hong Kong SAR Government: “It is a pleasure
talking to you…. You have certainly provided me with much food for thought. Perhaps we could meet again when
your second survey is near completion so that I could update you on the development on our side and get a
glimpse of the findings of your research.” (25-1-2015)
❖Food and Health Bureau: “We notice from the press that the results of the second-round telephone survey have
just been released, and wonder if you mind sharing the findings with us? It will be a very useful reference for us in
analysing public views.” (16-3-2015)
❖Mr Peter Tam, Secretary General, Hong Kong Federation of Insurers: “Thanks to you for sharing with me your
survey findings attached. More than 60% of the respondents have no intention to join the voluntary scheme. So
providing sufficient incentives is the key.” In a subsequent email, Mr Tam wrote: “This morning we presented our
industry’s views on the proposed VHIS to the Hon Cheung. I mentioned about you and your expertise.” (21-1-
2015)
❖Mr Chris Sun: “Your finding that those most satisfied with the public system here are more inclined to subscribe to
supplementary private health insurance is interesting, and offers a glimpse of the uniqueness of our health system
and why we have to knock out our own solution.” (28-8-2017)