微信客服：xiaoxionga100
微信客服：ITCS521
ECOS3997-无代写
时间：2024-02-28
ECOS3997 Interdisciplinary Impact in Economics
STREAM 1: Inequality and Policy Evaluation
Lecturer: Dr Juliana Silva-Goncalves
Semester 1 2024
Week 2
Topic: Simple Regression Model for Policy Evaluation
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue
2. Motivation - understand what motivated the research and its goal /
research question
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue
2. Motivation - understand what motivated the research and its goal /
research question
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue (we may refer to additional resources)
2. Motivation - understand what motivated the research and its goal /
research questions
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue (we may refer to additional resources)
2. Motivation - understand what motivated the research and its goal /
research questions
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue (we may refer to additional resources)
2. Motivation - understand what motivated the research and its goal /
research questions
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Article
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 2 / 52
Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 3 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 4 / 52
Background - First Nations Disadvantage & Closing the
Gap Initiative
▶ Across all standard indicators (education, health, employment
outcomes) First Nations Australians do worse than other Australians
▶ In 2007 the Australian Government pledged to Close the Gap:
▶ close the gap in life expectancy within a generation
▶ halve the gap in child mortality rates within a decade
▶ ensure access to childcare in remote communities within 5 years
▶ halve the gap in children’s reading, writing and numeracy within a
decade
▶ halve the gap in year 12 attainment rates by 2020
▶ halve the gap in employment outcomes within a decade
▶ Policies were put in place and billions of dollars spent to achieve those
targets (on average, the government spends more than twice as much
on First Nations Australians than other Australians)
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 5 / 52
Background - Where do First Nations outcomes stand?
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 6 / 52
Background - Where do First Nations outcomes stand?
Life expectancy
Figure 1: Productivity Commission national progress against the target. Baseline:
2005-2007; Latest: 2015-2017
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 7 / 52
Background - Where do First Nations outcomes stand?
Children developmentally on track when commencing school
Figure 2: Productivity Commission national progress against the target.
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 8 / 52
Background - Where do First Nations outcomes stand?
PISA scores
Figure 3: PISA mean performance scores in reading, mathematical and scientific
literacy, by domain and Indigenous status, 2000 to 2018. Left picture: Indigenous;
right picture: non-Indigenous
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 9 / 52
Background - Where do First Nations outcomes stand?
Year 12 attainment
Figure 4: Proportion of people aged 20–24 who had attained a Year 12 or equivalent
(Certificate III or above) qualification, by Indigenous status, 2001 to 2021
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 10 / 52
Background - Where do First Nations outcomes stand?
Employment
Figure 5: Employment rate (people aged 15–64), by Indigenous status, 2021
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 11 / 52
Background - Where do First Nations outcomes stand?
Youth detention rates
Figure 6: National detention rate of young people aged 10–17 years on an average day
and changes in the gap, by Indigenous status, 2011–12 to 2020–21
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 12 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 13 / 52
Motivation
▶ First Nations people make up 25.5% of the Northern Territory’s total
population (compared to 2.8% of the Australian population overall)
▶ Starting in 2007, the Australian Government rolled out ‘income
management’ (IM) for welfare payment recipients in remote
Aboriginal communities in the Northern Territory (NT)
▶ IM followed the publication of a report (“Little children are sacred”)
that called for immediate action to address relatively high rates of
child sexual abuse in remote communities
▶ Report emphasised the need to consider child neglect, alcoholism and
inadequate education as long-term contributors
▶ In response, the Australian Government announced the NT
Emergency Response (NTER) and IM was one of its policies
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 14 / 52
Motivation
▶ Before IM, welfare recipients received payments into their bank
accounts
▶ Under IM, half of each payment was put in a separate account with
restrictions over its use
▶ Funds could only be used towards priority needs, e.g. food, housing,
bills and clothing
▶ Goal: improve child outcomes by increasing spending on priority
needs, and reducing spending on harmful goods (e.g. alcohol,
tobacco)
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 15 / 52
Who was affected?
▶ IM affected most residents in remote communities
▶ Around 55% of adults in NTER communities were under IM by the
end of the rollout period, with around 80 percent having been under
IM at some point during the rollout period
▶ Women and younger adults were more likely to receive welfare
payments (AIHW 2010), meaning pregnant women were
over-represented
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 16 / 52
Research question
RQ. What was the impact of IM on child outcomes?
▶ Focus on birth weight as a proxy for child outcomes
▶ Low birth weight is an indicator of poor newborn health and has
consequences for child developmental outcomes and health outcomes
later in life
▶ Routinely measured —available in hospital administrative data
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 17 / 52
Research question
RQ. What was the impact of IM on child outcomes?
▶ Focus on birth weight as a proxy for child outcomes
▶ Low birth weight is an indicator of poor newborn health and has
consequences for child developmental outcomes and health outcomes
later in life
▶ Routinely measured —available in hospital administrative data
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 17 / 52
Research question
RQ. What was the impact of IM on child outcomes?
▶ Focus on birth weight as a proxy for child outcomes
▶ Low birth weight is an indicator of poor newborn health and has
consequences for child developmental outcomes and health outcomes
later in life
▶ Routinely measured —available in hospital administrative data
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 17 / 52
Link between Income Management (IM) and birth weight
▶ Idea: if IM was successful in increasing food consumption, we expect
an increase in birthweight
▶ 2 potential causes of low birthweight:
1. Short gestational length, which can be affected by mother’s
pre-pregnancy weight, history of prematurity, stress levels during
pregnancy
2. Intrauterine growth restriction, which can be affected by maternal
nutrition, in particular during 3rd trimester
▶ Expectation is that IM could affect (increase) infant birth weight
through intrauterine growth
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 18 / 52
Household Expenditures
▶ Important: Policy only affects recipients spending less than 50% of
their income on priority goods
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 19 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 20 / 52
Background - Policy implementation
▶ IM was rolled out to 73 communities and 10 town camps between
Sept 2007 - Oct 2008
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 21 / 52
Background - Policy implementation
▶ IM was rolled out on a different timeline from other NTER policies
Q. Why is this important / good news for the research?
▶ Because the evaluation will not be affected by other policies that
were put in place at the same time as IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 22 / 52
Background - Policy implementation
▶ IM was rolled out on a different timeline from other NTER policies
Q. Why is this important / good news for the research?
▶ Because the evaluation will not be affected by other policies that
were put in place at the same time as IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 22 / 52
Methodological Approach
▶ Compare outcomes for newborns in communities where IM was
introduced before or during the pregnancy [‘treatment’ group], with
outcomes for newborns in communities where IM was not yet
implemented at birth or was implemented very late in the pregnancy
[‘control’ group]
▶ What could be a problem with this approach, i.e. what could
invalidate the results?
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 23 / 52
Methodological Approach - Discussion
▶ Let’s shorten the sentence outlining the empirical approach to focus
on key aspect:
Compare outcomes in communities where IM was introduced, with
outcomes in communities where IM was not yet implemented
▶ Problem if order of introducing the policy was not random, or not
independent of the factors that affect infant birth weight
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 24 / 52
Methodological Approach - Discussion
▶ What if the policy was first introduced in most problematic
communities (e.g. highest level of alcohol consumption, worst child
outcome indicators, etc)?
→ Sample selection problem
▶ Comparing outcomes (infant birth weight) in communities that
received the policy first and those that received the policy later would
lead to negatively biased results and policy conclusions
▶ For instance, regardless of the impact of the policy, infants in
communities that received the policy first would arguably have lower
infant birth weight than those in communities that did not receive the
policy (yet)
▶ (Invalid) conclusion: Policy worsened child outcomes
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 25 / 52
Methodological Approach - Discussion
▶ What if the policy was first introduced in least problematic
communities (e.g. lowest level of alcohol consumption, best child
outcome indicators, etc)?
→ Again, Sample selection problem
▶ Comparing outcomes (infant birth weight) in communities that
received the policy first and those that received the policy later would
lead to positively biased results and policy conclusions
▶ For instance, regardless of the impact of the policy, infants in
communities that received the policy first would arguably have higher
infant birth weight than those in communities that did not receive the
policy (yet)
▶ (Invalid) conclusion: Policy improved child outcomes
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 26 / 52
Methodological Approach - Discussion
▶ What do the authors say in the article about the rollout of the policy?
1. Rollout was conducted following a pre-defined timeline, though no
information is publicly available explaining the logic of that timeline
2. Present arguments to support that a sample selection problem is
unlikely
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 27 / 52
Arguments to support validity of the approach
1. Rollout did not follow any clear geographic pattern, suggesting that
location-specific characteristics that could affect health and access to
health care were not correlated with the rollout schedule
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 28 / 52
Arguments to support validity of the approach
2. In the year prior to the rollout, birth outcomes in communities that
received IM early were no different from those that received it late
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 29 / 52
Arguments to support validity of the approach
3. Earlier and later adopting communities did not differ in terms of
average birth outcomes, birth complications, or community
characteristics pre-rollout
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 30 / 52
Arguments to support validity of the approach
4. IM did not affect mobility
▶ Scope for residents to move to avoid IM was limited, as eligibility was
determined based on residence just one week after the policy was
announced
▶ Evidence from other another study that IM did not affect mobility
→ Overall, it appears a selection problem is unlikely and that comparing
outcomes for early versus late adopters of the policy will yield valid
results about its impact
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 31 / 52
Other concerns for validity of the approach
▶ Any other concerns not listed / addressed by the authors?
▶ Anticipation effects —people in communities that received the IM
later knew it would happen, and that could have affected their
behaviour
▶ Examples of anticipation effects: create stress in the household,
change purchasing behaviour before IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 32 / 52
Other concerns for validity of the approach
▶ Any other concerns not listed / addressed by the authors?
▶ Anticipation effects —people in communities that received the IM
later knew it would happen, and that could have affected their
behaviour
▶ Examples of anticipation effects: create stress in the household,
change purchasing behaviour before IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 32 / 52
Data
▶ Hospital records on all infants born in the NT from 1996 onwards
▶ Identify infants from NTER communities based on their mother’s
residence
▶ Link the data with public information on date that IM was introduced
in each community, to construct a variable for treatment status
▶ Limit sample to infants born during or shortly after the rollout period,
to include all those who were in their third trimester of pregnancy
during the rollout
▶ Total sample of 1,153 births between 17 September 2007 and 31
January 2009
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 33 / 52
Regression Model
▶ Since rollout is assumed to be ‘as good as’ random, the authors use a
simple regression model to estimate the causal impact of IM on
infant birth weight
▶ Control group consists both of infants born into communities before
IM was introduced, and those born into other communities where it
had not yet been introduced by the beginning of their third trimester
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 34 / 52
Regression Model
▶ Control for seasonal patterns (rainfall), with worse outcomes for
infants born in the wet season
▶ Control for community fixed effects (ηc), controlling for unobserved
location-specific factors that influence birth outcomes
▶ δ: main parameter of interest
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 35 / 52
Mechanisms
▶ Add an indicator of premature birth in the model
▶ If the effect comes through an impact on nutrition during pregnancy,
we would expect it to have a larger effect on intrauterine growth than
gestational length, and hence for the treatment variable to remain
significant when controlling for prematurity
▶ If the effect comes through prematurity, the treatment effect will lose
significance
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 36 / 52
Mechanisms
▶ Add an indicator of premature birth in the model
▶ If the effect comes through an impact on nutrition during pregnancy,
we would expect it to have a larger effect on intrauterine growth than
gestational length, and hence for the treatment variable to remain
significant when controlling for prematurity
▶ If the effect comes through prematurity, the treatment effect will lose
significance
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 36 / 52
Mechanisms
▶ Add an indicator of premature birth in the model
▶ If the effect comes through an impact on nutrition during pregnancy,
we would expect it to have a larger effect on intrauterine growth than
gestational length, and hence for the treatment variable to remain
significant when controlling for prematurity
▶ If the effect comes through prematurity, the treatment effect will lose
significance
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 36 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 37 / 52
Effect of IM on infant birtweight
Model without controls
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 38 / 52
Effect of IM on infant birtweight
Controlling for season variations & time trend
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 39 / 52
Effect of IM on infant birtweight
Controlling for season variations, time trend & community FE
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 40 / 52
Effect of IM on infant birtweight
Controlling for season variations, time trend & community FE
▶ In a model without control variables, average birthweight is around 61
grams lower in the treatment group than in the control group (sig. at
10%)
▶ Controlling for season variations and an annual time trend doubles the
negative treatment effect to 120 grams (sig. at 1%)
▶ Further controlling for community fixed effects increases the
treatment effect to 164 grams (sig. at 1%)
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 41 / 52
Effect of IM on infant birtweight - Mechanisms
▶ What happens when controlling for prematurity? In theory:
▶ If the effect of IM comes through an impact on nutrition during
pregnancy, we expect the treatment variable to remain significant
when controlling for prematurity
→ IM is not linked to prematurity
▶ If the effect comes through prematurity, the treatment effect will
become smaller and lose significance
→ Lower average birth weight is explained by an increase in premature
births
→ IM is linked to prematurity
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 42 / 52
Effect of IM on infant birtweight - Mechanisms
Controlling for season variations, time trend, community FE & prematurity
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 43 / 52
Effect of IM on infant birtweight - Mechanisms
Controlling for season variations, time trend, community FE & prematurity
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 44 / 52
Effect of IM on infant birtweight - Mechanisms
▶ After controlling for prematurity, the treatment effect on birthweight
(intrauterine growth channel) declines by 28% to -119 grams, but
remains sizeable and statistically significant
→ Decrease in birthweight appears to have come through both the
intrauterine growth and the gestational length channels
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 45 / 52
Quantile regression results
▶ A decrease in average biryh weight could be good news (infant health
improvement) if it’s driven by a decrease in heavy babies (due to
lower lower rates of maternal gestational diabetes)
▶ This can be checked by looking at the effect of IM throughout the
birthweight distribution
▶ Method: quantile regression, estimating the effect at different
quantiles of the distribution
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 46 / 52
Quantile regression results
▶ Effect was statistically significant across the distribution, and largest
for newborns with very low birthweight
▶ Shows a negative effect of the policy on infant health
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 47 / 52
Quantile regression results
▶ Effect was statistically significant across the distribution, and largest
for newborns with very low birthweight
▶ Shows a negative effect of the policy on infant health
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 47 / 52
Summary
▶ Did IM improve child outcomes as intended?
▶ Study provides a partial answer, showing that it did not improve one
measure of child health outcomes (birthweight) and does not appear
to have produced the desired change in household consumption
patterns
▶ In fact, IM may have had a net negative impact on newborn health
▶ Possible reasons (untested):
▶ IM may have created an unhealthy food consumption environment for
pregnant women, by setting a low anchor (50% of the income) for
consumption of priority goods, including food
▶ low willingness to comply with the restrictions imposed
▶ stress generated by loss of freedom over consumption decisions
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 48 / 52
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 49 / 52
Discussion
▶ Conclusion of the research is that IM led to a decrease in birthweight,
hence worse child outcomes
▶ Is this enough evidence againts IM? Should the government abolish it
based on this single study?
▶ No. Because:
▶ It could be that a sample selection problem still exists (as
acknowledged by the authors) - IM was not set up as an RCT, and the
logic behind the roll-out is unknown (to the researchers)
▶ This study does not provide the full picture of the effect of IM on child
outcomes, it’s a single puzzle piece (‘partial evidence’ as noted by the
authors)
▶ Other studies should look at longer-term child outcomes (e.g. health,
educational, socioemotional outcomes)
▶ A better picture of the impact of IM is needed to give rigorous,
unbiased policy advice
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 50 / 52
Today’s key points
▶ In some cases, it’s possible to evaluate policies using a simple
regression analysis
▶ Requires making a credible case of no sample selection
problem, including sound arguments and evidence
▶ The study examined today achieves this, but only provides one piece
of the puzzle
▶ We need caution when interpreting findings and giving policy
recommendations based on findings from a single study—perhaps
there was a selection problem unknown to the researchers; study
provides evidence on a single child outcome
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 51 / 52
End of lecture 2!
Compulsory reading for next week:
Long-Run Effects of Catholic Schooling on Wages. (Available on Canvas)
Thank you!
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 52 / 52
STREAM 1: Inequality and Policy Evaluation
Lecturer: Dr Juliana Silva-Goncalves
Semester 1 2024
Week 2
Topic: Simple Regression Model for Policy Evaluation
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue
2. Motivation - understand what motivated the research and its goal /
research question
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue
2. Motivation - understand what motivated the research and its goal /
research question
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue (we may refer to additional resources)
2. Motivation - understand what motivated the research and its goal /
research questions
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue (we may refer to additional resources)
2. Motivation - understand what motivated the research and its goal /
research questions
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Plan
▶ Examine a research article that evaluates the impact of a government
policy using a simple regression analysis, and importantly why this
was possible and yields valid results
▶ We will focus on the article’s:
1. Background - general context of the research linked to an inequality
issue (we may refer to additional resources)
2. Motivation - understand what motivated the research and its goal /
research questions
3. Method - critically examine if it leads to strong results that support the
conclusions of the paper
4. Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 1 / 52
Today’s Article
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 2 / 52
Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 3 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 4 / 52
Background - First Nations Disadvantage & Closing the
Gap Initiative
▶ Across all standard indicators (education, health, employment
outcomes) First Nations Australians do worse than other Australians
▶ In 2007 the Australian Government pledged to Close the Gap:
▶ close the gap in life expectancy within a generation
▶ halve the gap in child mortality rates within a decade
▶ ensure access to childcare in remote communities within 5 years
▶ halve the gap in children’s reading, writing and numeracy within a
decade
▶ halve the gap in year 12 attainment rates by 2020
▶ halve the gap in employment outcomes within a decade
▶ Policies were put in place and billions of dollars spent to achieve those
targets (on average, the government spends more than twice as much
on First Nations Australians than other Australians)
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 5 / 52
Background - Where do First Nations outcomes stand?
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 6 / 52
Background - Where do First Nations outcomes stand?
Life expectancy
Figure 1: Productivity Commission national progress against the target. Baseline:
2005-2007; Latest: 2015-2017
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 7 / 52
Background - Where do First Nations outcomes stand?
Children developmentally on track when commencing school
Figure 2: Productivity Commission national progress against the target.
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 8 / 52
Background - Where do First Nations outcomes stand?
PISA scores
Figure 3: PISA mean performance scores in reading, mathematical and scientific
literacy, by domain and Indigenous status, 2000 to 2018. Left picture: Indigenous;
right picture: non-Indigenous
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 9 / 52
Background - Where do First Nations outcomes stand?
Year 12 attainment
Figure 4: Proportion of people aged 20–24 who had attained a Year 12 or equivalent
(Certificate III or above) qualification, by Indigenous status, 2001 to 2021
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 10 / 52
Background - Where do First Nations outcomes stand?
Employment
Figure 5: Employment rate (people aged 15–64), by Indigenous status, 2021
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 11 / 52
Background - Where do First Nations outcomes stand?
Youth detention rates
Figure 6: National detention rate of young people aged 10–17 years on an average day
and changes in the gap, by Indigenous status, 2011–12 to 2020–21
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 12 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 13 / 52
Motivation
▶ First Nations people make up 25.5% of the Northern Territory’s total
population (compared to 2.8% of the Australian population overall)
▶ Starting in 2007, the Australian Government rolled out ‘income
management’ (IM) for welfare payment recipients in remote
Aboriginal communities in the Northern Territory (NT)
▶ IM followed the publication of a report (“Little children are sacred”)
that called for immediate action to address relatively high rates of
child sexual abuse in remote communities
▶ Report emphasised the need to consider child neglect, alcoholism and
inadequate education as long-term contributors
▶ In response, the Australian Government announced the NT
Emergency Response (NTER) and IM was one of its policies
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 14 / 52
Motivation
▶ Before IM, welfare recipients received payments into their bank
accounts
▶ Under IM, half of each payment was put in a separate account with
restrictions over its use
▶ Funds could only be used towards priority needs, e.g. food, housing,
bills and clothing
▶ Goal: improve child outcomes by increasing spending on priority
needs, and reducing spending on harmful goods (e.g. alcohol,
tobacco)
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 15 / 52
Who was affected?
▶ IM affected most residents in remote communities
▶ Around 55% of adults in NTER communities were under IM by the
end of the rollout period, with around 80 percent having been under
IM at some point during the rollout period
▶ Women and younger adults were more likely to receive welfare
payments (AIHW 2010), meaning pregnant women were
over-represented
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 16 / 52
Research question
RQ. What was the impact of IM on child outcomes?
▶ Focus on birth weight as a proxy for child outcomes
▶ Low birth weight is an indicator of poor newborn health and has
consequences for child developmental outcomes and health outcomes
later in life
▶ Routinely measured —available in hospital administrative data
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 17 / 52
Research question
RQ. What was the impact of IM on child outcomes?
▶ Focus on birth weight as a proxy for child outcomes
▶ Low birth weight is an indicator of poor newborn health and has
consequences for child developmental outcomes and health outcomes
later in life
▶ Routinely measured —available in hospital administrative data
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 17 / 52
Research question
RQ. What was the impact of IM on child outcomes?
▶ Focus on birth weight as a proxy for child outcomes
▶ Low birth weight is an indicator of poor newborn health and has
consequences for child developmental outcomes and health outcomes
later in life
▶ Routinely measured —available in hospital administrative data
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 17 / 52
Link between Income Management (IM) and birth weight
▶ Idea: if IM was successful in increasing food consumption, we expect
an increase in birthweight
▶ 2 potential causes of low birthweight:
1. Short gestational length, which can be affected by mother’s
pre-pregnancy weight, history of prematurity, stress levels during
pregnancy
2. Intrauterine growth restriction, which can be affected by maternal
nutrition, in particular during 3rd trimester
▶ Expectation is that IM could affect (increase) infant birth weight
through intrauterine growth
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 18 / 52
Household Expenditures
▶ Important: Policy only affects recipients spending less than 50% of
their income on priority goods
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 19 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 20 / 52
Background - Policy implementation
▶ IM was rolled out to 73 communities and 10 town camps between
Sept 2007 - Oct 2008
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 21 / 52
Background - Policy implementation
▶ IM was rolled out on a different timeline from other NTER policies
Q. Why is this important / good news for the research?
▶ Because the evaluation will not be affected by other policies that
were put in place at the same time as IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 22 / 52
Background - Policy implementation
▶ IM was rolled out on a different timeline from other NTER policies
Q. Why is this important / good news for the research?
▶ Because the evaluation will not be affected by other policies that
were put in place at the same time as IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 22 / 52
Methodological Approach
▶ Compare outcomes for newborns in communities where IM was
introduced before or during the pregnancy [‘treatment’ group], with
outcomes for newborns in communities where IM was not yet
implemented at birth or was implemented very late in the pregnancy
[‘control’ group]
▶ What could be a problem with this approach, i.e. what could
invalidate the results?
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 23 / 52
Methodological Approach - Discussion
▶ Let’s shorten the sentence outlining the empirical approach to focus
on key aspect:
Compare outcomes in communities where IM was introduced, with
outcomes in communities where IM was not yet implemented
▶ Problem if order of introducing the policy was not random, or not
independent of the factors that affect infant birth weight
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 24 / 52
Methodological Approach - Discussion
▶ What if the policy was first introduced in most problematic
communities (e.g. highest level of alcohol consumption, worst child
outcome indicators, etc)?
→ Sample selection problem
▶ Comparing outcomes (infant birth weight) in communities that
received the policy first and those that received the policy later would
lead to negatively biased results and policy conclusions
▶ For instance, regardless of the impact of the policy, infants in
communities that received the policy first would arguably have lower
infant birth weight than those in communities that did not receive the
policy (yet)
▶ (Invalid) conclusion: Policy worsened child outcomes
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 25 / 52
Methodological Approach - Discussion
▶ What if the policy was first introduced in least problematic
communities (e.g. lowest level of alcohol consumption, best child
outcome indicators, etc)?
→ Again, Sample selection problem
▶ Comparing outcomes (infant birth weight) in communities that
received the policy first and those that received the policy later would
lead to positively biased results and policy conclusions
▶ For instance, regardless of the impact of the policy, infants in
communities that received the policy first would arguably have higher
infant birth weight than those in communities that did not receive the
policy (yet)
▶ (Invalid) conclusion: Policy improved child outcomes
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 26 / 52
Methodological Approach - Discussion
▶ What do the authors say in the article about the rollout of the policy?
1. Rollout was conducted following a pre-defined timeline, though no
information is publicly available explaining the logic of that timeline
2. Present arguments to support that a sample selection problem is
unlikely
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 27 / 52
Arguments to support validity of the approach
1. Rollout did not follow any clear geographic pattern, suggesting that
location-specific characteristics that could affect health and access to
health care were not correlated with the rollout schedule
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 28 / 52
Arguments to support validity of the approach
2. In the year prior to the rollout, birth outcomes in communities that
received IM early were no different from those that received it late
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 29 / 52
Arguments to support validity of the approach
3. Earlier and later adopting communities did not differ in terms of
average birth outcomes, birth complications, or community
characteristics pre-rollout
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 30 / 52
Arguments to support validity of the approach
4. IM did not affect mobility
▶ Scope for residents to move to avoid IM was limited, as eligibility was
determined based on residence just one week after the policy was
announced
▶ Evidence from other another study that IM did not affect mobility
→ Overall, it appears a selection problem is unlikely and that comparing
outcomes for early versus late adopters of the policy will yield valid
results about its impact
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 31 / 52
Other concerns for validity of the approach
▶ Any other concerns not listed / addressed by the authors?
▶ Anticipation effects —people in communities that received the IM
later knew it would happen, and that could have affected their
behaviour
▶ Examples of anticipation effects: create stress in the household,
change purchasing behaviour before IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 32 / 52
Other concerns for validity of the approach
▶ Any other concerns not listed / addressed by the authors?
▶ Anticipation effects —people in communities that received the IM
later knew it would happen, and that could have affected their
behaviour
▶ Examples of anticipation effects: create stress in the household,
change purchasing behaviour before IM
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 32 / 52
Data
▶ Hospital records on all infants born in the NT from 1996 onwards
▶ Identify infants from NTER communities based on their mother’s
residence
▶ Link the data with public information on date that IM was introduced
in each community, to construct a variable for treatment status
▶ Limit sample to infants born during or shortly after the rollout period,
to include all those who were in their third trimester of pregnancy
during the rollout
▶ Total sample of 1,153 births between 17 September 2007 and 31
January 2009
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 33 / 52
Regression Model
▶ Since rollout is assumed to be ‘as good as’ random, the authors use a
simple regression model to estimate the causal impact of IM on
infant birth weight
▶ Control group consists both of infants born into communities before
IM was introduced, and those born into other communities where it
had not yet been introduced by the beginning of their third trimester
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 34 / 52
Regression Model
▶ Control for seasonal patterns (rainfall), with worse outcomes for
infants born in the wet season
▶ Control for community fixed effects (ηc), controlling for unobserved
location-specific factors that influence birth outcomes
▶ δ: main parameter of interest
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 35 / 52
Mechanisms
▶ Add an indicator of premature birth in the model
▶ If the effect comes through an impact on nutrition during pregnancy,
we would expect it to have a larger effect on intrauterine growth than
gestational length, and hence for the treatment variable to remain
significant when controlling for prematurity
▶ If the effect comes through prematurity, the treatment effect will lose
significance
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 36 / 52
Mechanisms
▶ Add an indicator of premature birth in the model
▶ If the effect comes through an impact on nutrition during pregnancy,
we would expect it to have a larger effect on intrauterine growth than
gestational length, and hence for the treatment variable to remain
significant when controlling for prematurity
▶ If the effect comes through prematurity, the treatment effect will lose
significance
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 36 / 52
Mechanisms
▶ Add an indicator of premature birth in the model
▶ If the effect comes through an impact on nutrition during pregnancy,
we would expect it to have a larger effect on intrauterine growth than
gestational length, and hence for the treatment variable to remain
significant when controlling for prematurity
▶ If the effect comes through prematurity, the treatment effect will lose
significance
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 36 / 52
Week 2 Outline
Background - general context of the research
Motivation - what motivated the research and research questions
Methods
Results
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 37 / 52
Effect of IM on infant birtweight
Model without controls
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 38 / 52
Effect of IM on infant birtweight
Controlling for season variations & time trend
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 39 / 52
Effect of IM on infant birtweight
Controlling for season variations, time trend & community FE
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 40 / 52
Effect of IM on infant birtweight
Controlling for season variations, time trend & community FE
▶ In a model without control variables, average birthweight is around 61
grams lower in the treatment group than in the control group (sig. at
10%)
▶ Controlling for season variations and an annual time trend doubles the
negative treatment effect to 120 grams (sig. at 1%)
▶ Further controlling for community fixed effects increases the
treatment effect to 164 grams (sig. at 1%)
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 41 / 52
Effect of IM on infant birtweight - Mechanisms
▶ What happens when controlling for prematurity? In theory:
▶ If the effect of IM comes through an impact on nutrition during
pregnancy, we expect the treatment variable to remain significant
when controlling for prematurity
→ IM is not linked to prematurity
▶ If the effect comes through prematurity, the treatment effect will
become smaller and lose significance
→ Lower average birth weight is explained by an increase in premature
births
→ IM is linked to prematurity
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 42 / 52
Effect of IM on infant birtweight - Mechanisms
Controlling for season variations, time trend, community FE & prematurity
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 43 / 52
Effect of IM on infant birtweight - Mechanisms
Controlling for season variations, time trend, community FE & prematurity
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 44 / 52
Effect of IM on infant birtweight - Mechanisms
▶ After controlling for prematurity, the treatment effect on birthweight
(intrauterine growth channel) declines by 28% to -119 grams, but
remains sizeable and statistically significant
→ Decrease in birthweight appears to have come through both the
intrauterine growth and the gestational length channels
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 45 / 52
Quantile regression results
▶ A decrease in average biryh weight could be good news (infant health
improvement) if it’s driven by a decrease in heavy babies (due to
lower lower rates of maternal gestational diabetes)
▶ This can be checked by looking at the effect of IM throughout the
birthweight distribution
▶ Method: quantile regression, estimating the effect at different
quantiles of the distribution
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 46 / 52
Quantile regression results
▶ Effect was statistically significant across the distribution, and largest
for newborns with very low birthweight
▶ Shows a negative effect of the policy on infant health
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 47 / 52
Quantile regression results
▶ Effect was statistically significant across the distribution, and largest
for newborns with very low birthweight
▶ Shows a negative effect of the policy on infant health
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 47 / 52
Summary
▶ Did IM improve child outcomes as intended?
▶ Study provides a partial answer, showing that it did not improve one
measure of child health outcomes (birthweight) and does not appear
to have produced the desired change in household consumption
patterns
▶ In fact, IM may have had a net negative impact on newborn health
▶ Possible reasons (untested):
▶ IM may have created an unhealthy food consumption environment for
pregnant women, by setting a low anchor (50% of the income) for
consumption of priority goods, including food
▶ low willingness to comply with the restrictions imposed
▶ stress generated by loss of freedom over consumption decisions
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 48 / 52
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 49 / 52
Discussion
▶ Conclusion of the research is that IM led to a decrease in birthweight,
hence worse child outcomes
▶ Is this enough evidence againts IM? Should the government abolish it
based on this single study?
▶ No. Because:
▶ It could be that a sample selection problem still exists (as
acknowledged by the authors) - IM was not set up as an RCT, and the
logic behind the roll-out is unknown (to the researchers)
▶ This study does not provide the full picture of the effect of IM on child
outcomes, it’s a single puzzle piece (‘partial evidence’ as noted by the
authors)
▶ Other studies should look at longer-term child outcomes (e.g. health,
educational, socioemotional outcomes)
▶ A better picture of the impact of IM is needed to give rigorous,
unbiased policy advice
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 50 / 52
Today’s key points
▶ In some cases, it’s possible to evaluate policies using a simple
regression analysis
▶ Requires making a credible case of no sample selection
problem, including sound arguments and evidence
▶ The study examined today achieves this, but only provides one piece
of the puzzle
▶ We need caution when interpreting findings and giving policy
recommendations based on findings from a single study—perhaps
there was a selection problem unknown to the researchers; study
provides evidence on a single child outcome
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 51 / 52
End of lecture 2!
Compulsory reading for next week:
Long-Run Effects of Catholic Schooling on Wages. (Available on Canvas)
Thank you!
Interdisciplinary Impact in Economics Sem 1 2024 Week 2 52 / 52
