Children On Benefit: Who Stays Longest?

Garry Barrett ¹
Department of Economics
University of New South Wales

Frances Krsinich
Moira Wilson
Centre for Social Research and Evaluation
Ministry of Social Development

Abstract

This paper reports on the analysis of administrative benefit data for the cohort of approximately 30,000 children born in 1994 who had contact with the benefit system before age three. We follow each child for between four and eight years and examine the length of their first spell with an adult caregiver receiving an income-tested benefit. The explanatory variables used in the analysis include characteristics of the child and their caregiver(s), as well as the regional unemployment rate and several variables used as proxies for key policy changes. Proportional hazard models, which allow for time variation in the variables as well as censoring, are estimated to gauge the independent association between each observed characteristic and the probability of the child leaving benefit. Median durations for selected combinations of characteristics are calculated, using the estimated proportional hazard models and the empirical survival functions. Departure from benefit marks the end of a benefit spell. This analysis does not go on to examine the quality of the outcome that resulted.

Introduction

Analysis of longitudinal benefit administration data for New Zealand has shown that by the time children born in 1993 turned seven, half had been supported by one of New Zealand’s main social assistance benefits at least once. While this was a transitory experience for many, approximately one in five children in the 1993 birth cohort spent at least five of their first seven years of life supported by a main benefit (Ball and Wilson 2002).

Bivariate analysis of factors associated with long benefit durations highlights having first contact with the benefit system at birth; living with a sole caregiver at first contact; and first appearing with a primary beneficiary who was female, Maori or aged under 20 (Ball and Wilson 2002). But these factors are interrelated. Further, the significance of some may reflect the influence of factors not captured by the benefit data, such as the educational attainment and employment history of the caregivers.

The objective of the more detailed analysis reported here is to estimate independent associations between observed characteristics and benefit durations. We also include new measures of educational attainment and employment history of caregivers in the analysis.

The need to better understand childhood income experiences is not trivial. Recent research highlights links between income relative to needs and access to items basic to child well-being (Krishnan et al. 2002). Early findings from the New Zealand Census-Mortality Study show that children in households with low equivalised income at the 1991 Census had higher than average mortality rates over the ensuing three years (Blakely 2002). Studies carried out in New Zealand and overseas consistently find that low family income matters not only for well-being in childhood, but also for outcomes in later life, although the extent to which this relationship is causal is the subject of some debate (Mayer 2002). While benefit data do not tell us about all of children’s experience of low income, they can provide an indicative view.²

The report is organised as follows. First, we describe the theoretical model underpinning the organisation of the analysis. We then detail the data used, including a discussion of the advantages and limitations of the data for our analysis. The statistical methodology is outlined and then the results are summarised. We conclude by summarising some of the interesting results and point to areas for future research.

More detailed information on all the areas covered by this article can be found in Barrett et al. (2002a).

Theoretical Models

When thinking about the duration of children’s spells on benefit programmes it is not appropriate to apply behavioural causal models, since it is adults, not children, that make the relevant behavioural choices (Jenkins and Rigg 2001:75). The approach we take is to develop descriptive rather than causal models, but use models of adult behaviour to help structure the analysis and select potentially important explanatory variables.

The basic principle underlying the analysis of the duration of adult spells on benefit is that at any point in time an individual either continues to participate or exits a programme, depending on the choices they make, subject to constraints. Where the person is unemployed, models of job search and acceptance are relevant. Where the person is a sole parent, models of partnering as well as models of job search are relevant. Where the person is incapacitated, medical models of time to recovery or mortality might be relevant, as well as job search models that incorporate the implications of current or past incapacity for wage offers and the costs of working.

From the point of view of the child, a range of events might end a spell on benefit. These include the employment of a sole caregiver or one or both partnered caregivers, the partnering of the child’s sole caregiver to an employed partner, the recovery or occupational rehabilitation of an incapacitated caregiver, or the movement of the child to another caregiver who is employed.

The Benefit Dynamics Data

The data source used in the analysis is the benefit dynamics data set, a longitudinal data set built from benefit administration data.³

The data set has some particular strengths as the basis for an analysis of children’s income experiences. It includes the entire population of people who received a first-tier income-tested benefit⁴ over the period covered (January 1993 to December 2001) and holds unique identifiers for all individuals in that population, including the children. These features let us analyse all children having contact with the benefit system, including narrowly defined sub-groups of children, without encountering the problems associated with sampling error. We are also able to analyse the durations of those children who might have a low probability of being included or retained in a longitudinal survey, such as those moving between caregivers (Bradbury et al. 2001:47-48).

In addition, the data set gives a continuous account of changes in status. It accurately dates the start and finish dates of spells on benefit. These features, together with our ability to comprehensively keep track of children, enable accurate calculation of total benefit durations over what can be complex benefit and caregiver histories.

Against these strengths there are some limitations. The quality of the data is highly dependent on the administrative processes that generate them. The administrative origin of the data set also limits its scope. It gives only a partial view of the lives of children and families, confined to those aspects that are important to the administration of benefits.

The Analysis Data

The benefit dynamics data set currently provides longitudinal data over a nine-year window from 1 January 1993 to 31 December 2001. Figure 1 illustrates the manner in which the present analysis utilises this window.

Figure 1 Analysis Data Set: Study Population and Follow-Up Times

We focus on children born within the window of the base data to better understand whole of childhood low-income experiences. The 1994 rather than 1993 birth cohort is used in order to gain a one-year history of the caregiver’s benefit use prior to the child’s first contact.

The study population is children born in 1994 who had some contact with the benefit system by age three, or approximately 28,600 children. We estimate that these children make up close to half the total number of children born in New Zealand in 1994, and more than 80% of the children in that birth cohort who would have contact with the benefit system by age seven (Ball and Wilson 2002). Restricting the study to those with contact by age three allows us to gain a minimum four-year follow-up from the date of first contact. The maximum follow-up is eight years.

The Dependent Variable

The focus of our interest is the length of the first-ever spell on benefit for children in our cohort (that is, those born in 1994 who had some contact with benefit by age three).

A large proportion of children who complete a spell on one benefit or with one caregiver either transfer directly to another benefit or caregiver the same or next day, or return to benefit after only a short period. These short absences make it sensible to define a spell on benefit as the period up to an exit that is sustained for some minimum period. We define this minimum period to be 12 weeks, based on analysis of the rates of return to benefit across different lengths of time.⁵ No attempt is made in this paper to examine the quality of the outcome that results from cessation of benefit.

With this 12-week definition, the first-ever spell accounts for:

• 80% of the total weeks spent on benefit by the study children in the 3.75–7.75 year follow-up;⁶ and
• the total experience of 58% of the study children in the follow-up – this being the proportion with only a single spell in that period in terms of the 12-week definition.

Observed Characteristics

The variables we include capture differences between children in their family composition, and in the demographic and other characteristics of their caregivers. Our methods allow for time variation in those characteristics that potentially vary over time.⁷ This is important, because we allow characteristics such as family structure and the caregiver’s participation in paid employment to vary as the spell progresses. We also include variables that capture key changes in policy settings that occurred over the period, and changes in unemployment rates in the region (or regions) in which the child lives. Partner’s characteristics are included for that portion of the population (approximately 40%) with partnered caregivers.

Family composition is measured using the following variables:

• the partnership status and sex of the primary beneficiary;
  whether there were any other children in the family in specific age groups; and
• the age of the child at their first-ever contact.

The demographic characteristics of caregivers that are captured are:

•  the age of the primary beneficiary; and
   the ethnicity of the primary beneficiary and partner.⁸

We cannot include accurate information on the partnership history of caregivers given the limited window on their lives that benefit data provides. But we include the following variable as an indicative measure of the child’s opportunity for moving to a different caregiver, who may now be employed:

• whether or not there exists a person who has cared for the child at some point in the past on benefit, who is not caring for them at present.

The educational and labour market history of the caregivers is proxied by the following variables:

• the highest educational qualifications on the job-seeker register for the primary beneficiary and the partner; and
• the proportion of time in the first 38 weeks⁹ of the year preceding the child’s first contact spent on benefit by the primary beneficiary and the partner.

Job search activities and the health and incapacity status of caregivers are proxied by the benefit type they received:

• whether they received a sole parent, unemployment, short-term sickness or long-term incapacity-related benefit.

In the absence of more direct measures, such as dates of employment and hours of work, data on the earnings declared by benefit recipients provide the best proxy for the degree to which caregivers participate in paid employment while on benefit. We include:

•  declared earnings of primary and partner.¹⁰

Local labour market conditions are proxied by the inclusion of:

• the region in which the child lived; and
• the quarterly unemployment rate in the region in which the child lived.
  Since unemployment rate data may reflect regional effects apart from (but correlated with) unemployment, we include the dummy variables indicating regional location in an attempt to control for such regional fixed effects.

Policy-Change Variables

We also include variables marking five major policy changes that affected the work incentives and work-related obligations of benefit recipients with young children.¹¹ Indicator variables for each of these were defined as 0 before the key date and 1 from the key date onwards. These policy changes are described below.

1 July 1996 Introduction of Dual Benefit Abatement The dual abatement regime introduced on 1 July 1996 relaxed benefit abatement rates for domestic purposes, widows and invalids beneficiaries. It was aimed at improving incentives for these groups to work part-time in the short term and improving chances of full-time employment and movement off benefit in the longer term.

1 July 1996 taxation and tax credit changes The 1996 abatement change coincided with cuts in tax rates and the introduction of a new Independent Family Tax Credit payable only to in-work families not on benefit (this payment was doubled to reach its full level in July 1997 and has since been renamed the Child Tax Credit). One of the aims of these changes was to increase financial incentives for full-time work for beneficiaries with children (Birch 1996).

1 July 1998 alignment of rates of sickness benefit with rates of unemployment benefit This policy change reduced the amount payable to a person who qualified for a sickness benefit to the equivalent rate of unemployment benefit. This reduced the rates payable to two-parent families, but not sole parents as their rates had already been aligned prior to the reform. Existing recipients continued being paid at their current rate until their circumstances changed.

1 July 1998 taxation changes Further tax rate reductions that increased financial incentives for full-time work were introduced in 1998.

1 February 1999 extension of work-related reciprocal obligations Work-related reciprocal obligations were extended from 1 April 1997, affecting caregivers of very young children from 1 February 1999. From that date spouses and sole parents with a youngest child aged under six years were required to attend an annual mandatory interview to discuss their future employment prospects. Those with a youngest child aged five years could be required to participate in a work preparation activity. A part-time work test was extended to those with children aged six to 13 years, and a full-time work test applied to all spouses and sole parents who had a youngest child aged 14 or more years.¹²

Although the full-time work testing did not apply to the caregivers of the children in the study population, it may have had a signalling effect that increased exit rates.

We include two indicators.

• The first is set to 1 for all children in the study (except those with a sole caregiver on benefits other than Domestic Purposes Benefit (DPB) and Widows Benefit (WB)) from 1 February 1999 onwards, this being the date their caregivers became subject to the mandatory interview requirement and were affected by the changed “signal” associated with full-time work testing of those with older children.
• The second is set to 1 for periods after February 1999 during which the youngest child present was aged six or over (except where the child is with a sole caregiver on benefits other than DPB and WB), these being periods in which their caregivers were subject to extended part-time work testing.

Composition of study population The composition of the study population in terms of the observed characteristics is shown in Table 1.

Unobserved characteristics potentially associated with benefit duration Like all data sources, the data used in the analysis have some important limitations that need to be recognised. First, the data do not contain information on the complete set of factors that potentially influence children’s interaction with the benefit system. For example, the data available provide no information on the aspirations, attitudes, or motivation of caregivers. Some other key characteristics that are either not captured or not captured in enough detail are partnership history; health and incapacity status of both caregivers and children; literacy skills; labour market assistance and training interventions; availability, quality and cost of childcare; local labour market conditions; and benefit levels (independent of behaviour and characteristics of beneficiaries). Therefore, it should be noted that the associations we estimate in this analysis between observed characteristics and benefit durations are only independent of other observed characteristics. If, for example, work experience differs across regions in such a way that the “previous year” proxy for work experience does not fully capture, then the associations we estimate between region and benefit duration will also incorporate the associations between benefit duration and work experience.

Methodology

The longitudinal benefit data we analyse have some characteristic features: they contain censored¹³ observations and most of the measured characteristics are time varying. We use survival analysis¹⁴ techniques specifically designed to take account of these features.

We estimate a proportional hazards model of the form:

where:

is the hazard, or exit rate at time t for individual i
is the baseline hazard¹⁵ at time t
is the proportional effect of characteristic on the baseline hazard
is the measurement of characteristic for individual i at time t.

We also estimate a complementary log-log model of the form:

where:

is the probability that individual i leaves benefit in time interval t, given that they have not already left benefit
, and are as for the proportional hazards model.

For example, an estimated of 0.2 for the characteristic “sole male” means that, all other characteristics being equal, the hazard rate for a sole male is exp(0.2) = 1.22 as high as that of a sole female.¹⁶

This model estimates the proportional effect of each characteristic on the exit rate (the likelihood of leaving benefit) after controlling for the effect of the other characteristics included in the model.

It is important to note that we are estimating the strength and magnitude of associations between different characteristics and length of spell, but we cannot definitively infer the existence or direction of causation with this analysis. For example, the educational attainment of the primary beneficiary is found to be important in explaining the children’s exit from benefit. This may reflect a genuine causal effect, whereby higher education generates better employment opportunities, which enable a carer and his/her children to leave benefit more quickly. Alternatively, it may simply reflect associations induced by unmeasured other factors (such as motivation or perseverance), which explain both educational attainment and benefit duration.

While the two models estimate the same parameters, each one has strengths and weaknesses.

• The Cox proportional hazards method¹⁷ fully incorporates the weekly time variation we have available in the data. Further, the Cox model is a “semi-parametric estimator” and is robust in the statistical sense that no parametric assumption is made about the shape of the baseline hazard function. Indeed, the baseline hazard function is factored out of the model and no direct estimate of the baseline hazard function is produced.
• The complementary log-log model,¹⁸ although very flexible, places restrictions on the baseline hazard (for example, the hazard function is constant within a time-segment). With this model, the baseline hazard function is estimated as a step-function and requires some aggregation of duration times in practice (for example, we divide potential duration into 35 time segments). The advantage of this model is that we directly estimate the baseline hazard function and so are able to produce estimates of median duration for different cases, which is a useful way of interpreting the results.

More technical detail can be found in Barrett et al. (2002a, 2002b).

Results

Independent Associations

Barrett et al. (2002a) give results from incrementally building up the full Cox proportional hazards model. Table 1 below shows the parameters¹⁹ and their associated standard errors estimated in the full model with the complete set of explanatory variables. The composition of the sample is also shown – at the beginning of the spell, and two and five years into the spell.²⁰ Estimates that are statistically significant at the 5% level are in bold. The base categories for each characteristic are indicated by brackets.²¹

Table 1 Sample Composition and Cox Proportional Hazards Model Estimates

 * Due to rounding, percentages may not always sum to totals.
** Bolded values in this table identify estimates statistically significant at the .05% level

 * Due to rounding, percentages may not always sum to totals
**.Bolded values in this table identify estimates statistically significant at the 0.5% level
*** Because this variable relates to the previous year of the partner at first contact, the potential for partners to change means the sample compositions become less relevant after first contact.

* Due to rounding, percentages may not always sum to totals.
** Bolded values in this table identify estimates statistically significant at the 0.5% level

Following are summaries of the estimated proportional effects of the hazard rate of each characteristic, independent of the other characteristics included in the model.

Compared to the base category of sole female, couples with a male primary beneficiary have a 43% increase in the likelihood of leaving benefit, while couples with a female primary have a 40% increase. The estimate for sole male is statistically insignificant.

The age of the primary beneficiary is strongly associated with exit rate, as might be expected. Children of caregivers aged under 20 have the lowest probability of leaving benefit. Compared to this group, the probability of leaving benefit increases up to the 25–29 age group, after which it decreases steadily. There is a 53(61)% increase in the exit rate associated with those aged 20–24 (25–29) years. The increase in exit rate levels off after these age ranges, with an estimated 47(22)% increase for the age ranges 30–39 (40–49). The estimated decrease (-12%) in the probability of leaving benefit for those aged over 50 is statistically insignificant as this is a very small sub-population.

Other children being present gives rise to decreased likelihood of leaving benefit, for each age range. Presence of a child, or children, in the under-two age group corresponds to a 15% decrease in the exit rate. Other children in either the 6–13 age range or the over-14 age range are associated with 10% decreases, while the decreased likelihood of leaving benefit is least marked when there are other children in the two-to-five age range (a 7% decrease in the exit rate).

Whether or not the primary beneficiary transferred from sickness benefit soon after the child’s birth (indicating that receipt of that benefit is likely to have been associated with pregnancy) has no significant effect on the exit rate.

As might be expected, benefit type is strongly associated with the likelihood of leaving benefit. Compared to sole-parent-related benefit, both unemployment and sickness-related benefits are associated with an increased likelihood of leaving benefit: 87% and 33% increases respectively. Unsurprisingly, receipt of a long-term benefit is associated with significantly decreased probability of leaving benefit. The estimated parameter for this category is -1.04, but note that in this extreme case does not give a good approximation to exp()-1, a 65% decrease.²²

Compared with having no formal educational qualifications, any educational qualification held by the primary beneficiary is associated with an increased probability of leaving benefit. The estimated increases range from 8% (for fewer than three School Certificate subjects) to 37% (for a degree). Between these two extremes, three or more School Certificate subjects is associated with a 17% increase; “other” school qualifications with a 19% increase; post-secondary with a 24% increase, form six with a 25% increase and bursary with a 32% increase.

The older the child when they first have contact with the benefit system, the greater their likelihood of leaving benefit. Compared to those in contact at birth, those who first have contact between birth and six months have a 15% increase in the probability of leaving benefit. Between six months and one year there is a 33% increase, between one and two years there is a 41% increase, and first contact between two and three years is associated with a 56% increase in the probability of leaving benefit. There is some possibility, however, that this characteristic could also act as a proxy for environmental changes, as it will be relatively strongly associated with calendar time.²³

Whether or not there has ever been another caregiver associated with the child in their benefit history gives a rough indication of the possibility of moving to another caregiver, who might be off benefit. There is an estimated 5% increase in the probability of leaving benefit associated with this characteristic.

The labour market history of the primary beneficiary is roughly proxied by the proportion of time spent on benefit in the previous year.²⁴ However, this variable may also proxy unobserved characteristics of the primary beneficiary, which are associated with their greater reliance on benefits (that is, it is analogous to a “lagged dependent variable” for the primary beneficiary). Compared with spending all the time on benefit, the less time spent on benefit, the higher the associated exit rate. More than half, but not all, the time is associated with a 24% increase, under half the time with a 51% increase, and no time spent on benefit in the previous year is associated with a 59% increase in the likelihood of leaving benefit. Therefore, the greater the prior labour market experience, or, equivalently, the less their previous benefit reliance, the greater the likelihood the primary beneficiary and their children will exit benefit.

Declared earnings of the primary beneficiary is a good indicator of current participation in paid employment, which we would expect to be strongly associated with the probability of leaving benefit. The results bear this out, with earnings of under $80 associated with a 10% increase, earnings between $80 and $179 associated with a 33% increase, and earnings of $180 or more associated with an 88% increase in the probability of leaving benefit, compared with no earnings.

Three of the 10 non-missing ethnicity categories for the primary beneficiary have statistically significant estimated associations with the likelihood of leaving benefit. The base category is New Zealand European, and compared with this New Zealand Maori is associated with a 14% decrease in the probability of leaving benefit, Samoan ethnicity with a 12% increase, and “Other” ethnicity with a 42% decrease in the probability of leaving benefit. Estimates for the other ethnicities are statistically insignificant.

Other than the “not recorded” and “other” categories for regions, just six of the remaining 15 regions are significantly associated with benefit exit rate. Compared to Auckland, Tasman is associated with the largest decrease in likelihood of leaving benefit (-29%) and Southland is associated with the largest increase (14%). Between these extremes lie Gisborne (23% decrease), Northland (18% decrease), Nelson (16% decrease), and Wellington (11% decrease in the exit rate).

Partner’s characteristics ²⁵
All non-missing categories of partner’s educational qualifications are associated with increased exit rates, compared with having no formal education. The largest increase, of 26%, is associated with Bursary, while the smallest statistically significant increase of 8% is associated with three or more School Certificate subjects. Having fewer than three School Certificate subjects is associated with a 10% increase in the exit rate, a degree with a 13% increase, and post-secondary qualifications with a 20% increase. Note, however, that almost half of all partners have educational qualification missing, so the results are unlikely to be representative of those we would have found with complete data.

Five of the 10 non-missing partner’s ethnicity categories are significantly associated with the likelihood of leaving benefit. Compared with New Zealand European, partners of Indian or “Other” ethnicities are associated with a 40% decrease in the likelihood of leaving benefit, Samoan partners with a 38% decrease, Tongan with a 35% decrease and Other European with a 13% decrease. Partners who are New Zealand Maori are associated with a 5% decrease, but this estimate is not statistically significant due to the small magnitude of the estimate. Note that the results for partner’s ethnicity are quite different from those for primary’s ethnicity – in particular, Samoan ethnicity for the primary beneficiary is associated with a 12% increase in exit rate.

As might be expected, the proportion of time spent by the partner on benefit in the previous year is not as significantly associated with benefit exit rate as for the primary beneficiary. Spending between 0% and 50% of the previous year on benefit is associated with a 9% increase in exit rate, but the other two categories of time spent on benefit do not have significant estimates.

Partner’s earnings are even more strongly associated with leaving benefit than primary’s earnings. Compared to a partner with no earnings, partners earning between $0 and $79 are associated with a 32% increase in exit rate, those earning between $80 and $179 with a 52% increase, and those earning over $180 with a 92% increase. Note that these results are not just acting as a proxy for partnership status, which is already controlled for in the model. An interpretation for this result is currently not obvious, and requires further thought from a policy perspective.

Socio-economic environment
A 1% increase in the unemployment rate is associated with a 1% decrease in the probability of leaving benefit – a nicely symmetrical result, but not statistically significant.

Both the 1996 dual benefit abatement and the 1999 mandatory interview requirement policy changes are associated with an increased probability of leaving benefit – 6% and 12% respectively.

Both the 1996 and the 1998 tax changes are associated with a decreased likelihood of exiting benefit: the 1996 tax change with a 20% decrease and the 1998 tax change with a 29% decrease. These counterintuitive results suggest that the tax change indicators, which apply to all of the cohort still on benefit at the relevant point in time, may be acting as a proxy for other unobserved environmental changes happening around the same calendar time. On the other hand, perhaps the results are not so counterintuitive given the tax changes applied to the whole population, not just beneficiaries with children. Perhaps once there was more financial incentive to apply for relatively low-paying jobs this may have resulted in more competition for these jobs, and beneficiaries, or at least beneficiaries with children, may have had actually less likelihood of being employed in these positions as a result.

The extreme decrease in the probability of leaving benefit associated with the alignment of sickness benefit and unemployment benefit rates in 1998 has a huge standard error and is therefore not at all reliable.²⁶

The 1999 introduction of part-time work testing is associated with a 7% decrease in exit rate, but this result is not statistically significant.

The Baseline Hazard

The complementary log-log modelling enables explicit estimation of the baseline hazard.

The baseline hazard shown in Figure 2 is of interest because it:

• >gives an indication of “duration dependence” – the effect of length of spell on the probability of leaving benefit, and
• lets us calculate expected durations for specific combinations of characteristics.²⁷

Any unobserved heterogeneity in the population will result in the estimated baseline hazard decreasing more than the “true” baseline hazard,²⁸ so we should interpret the slope of the estimated baseline hazard as a lower bound on the slope of the true baseline hazard. This would make an estimated decreasing hazard inconclusive as an indicator of duration dependence. However, as the estimated baseline hazard is increasing over a large range of duration times, we can conclude that the actual baseline hazard is (mainly) increasing – in other words, the likelihood of leaving benefit increases with the length of time on benefit for our cohort of children.

This is perhaps as expected for children’s first-ever spells where those spells begin before the age of three, as the child (and any siblings) age along with the spell duration, getting older and becoming more independent.

Figure 2 Estimated Baseline Hazard

Median Durations for Selected Characteristics

Table 2 gives median durations for different combinations of characteristics, and the estimated proportion expected to remain after 7.75 years (the maximum observation period). Note that the baseline category is not the “average” cohort member, but rather a combination of characteristics which tend to be associated with long benefit durations; hence it has a relatively long median duration.

As a useful reference we also generated a predicted estimate of median duration for the “average” cohort member by combining the model parameters with the sample proportions. The resulting median duration for this average case is 3.5 years.²⁹

Table 2 Median Durations of Selected Cases

Medians for Broad Sub-Groups

The categories for which we report median durations above may be difficult to understand because there are so many characteristics to contend with. Of interest are the median durations of some broadly defined population sub-groups; for example, children of sole females, primaries of Maori ethnicity, of European ethnicity, primaries with no formal education, and so on.

Results from the models estimated do not lend themselves to easy calculation of medians for these types of groups.³⁰ Instead, we have approximated these measures from plots of the empirical survival functions, with the population stratified by the appropriate sub-grouping. While this approach does deal with censoring appropriately, its limitation is that the stratification is defined by the characteristics at first contact, and therefore does not take account of time variation. So, for example, the median duration for sole females will include those children who had first contact with sole females but whose caregiver later partnered and stayed on benefit. Conversely, it will exclude those whose first contact was with partnered carers who later split up such that the primary carer became a sole female.

Despite these limitations, the results are still of interest as a good summary of the actual experience of different sub-groups, where these sub-groups are defined as at the child’s first contact. Note that these measures are from the child’s point of view³¹ and relate to the first spell only.³²

Table 3 Median Durations from Empirical Survival Functions

* Since only selected categories are included, in general these percentages do not add to 100%.

Ranking Characteristics

Of interest is how “important” each observed characteristic is in explaining benefit duration, or, stated differently, what is the level of each characteristic’s “explanatory power” in the model? This information could be used to help focus or prioritise policy work and intervention. We took two approaches to this:

1. We noted the Akaike Information Criterion (AIC) statistic for the full model with each significant characteristic removed in turn. The AIC statistic is calculated as –2 times the value of the log-likelihood function plus two times the number of coefficients estimated. The AIC is used as a measure of the goodness-of-fit for a model which incorporates a penalty for the number of variables used in the model. The AIC statistic is similar to an adjusted-R-squared statistics used in regression analysis, and is used as a model selection criterion. Models with a lower AIC have greater “information content”. The procedure here is to check how the “goodness-of-fit” deteriorates (compared to the full model) when one set of characteristics is excluded from the model: the lower the AIC, the better the model in terms of its information content. The reason for examining the effect of taking each variable out of the full model (rather than adding characteristics to the null model) is the high level of interaction among many of the variables.

2. For each characteristic, we calculated the range of the hazard ratios (that is, the range of the s). While the numbers have some meaning (as a measure of the magnitude of the partial derivative of the hazard rate with respect to this characteristic), this approach has some significant limitations in that there is no accounting for the relative weighting of the population in each category of a characteristic and, given the associations between many of the characteristics, the ranking is very dependent on the characteristics that were included.

Given the limitations of each of these approaches, it is probably most useful to consider both ranking methods together.

Table 4 Ranking Observed Characteristics in Terms of Explanatory Power

*The AIC for the full model is 244949.8. Removal of primary’s education from the model results in the biggest AIC (least “information content”), so primary’s education is ranked as the “most important” characteristic, and so on. Note that the information content of the model is actually (very slightly) less when unemployment rate is included (244949.8), compared to when it is excluded (244949.3).
** All policy change indicators were removed together from the full model when calculating the corresponding AIC.
*** Unemployment rate was not ranked in terms of the hazard ratio range, which is only applicable to categorical characteristics.

Note that both primary’s earnings and partner’s earnings rank highly when we look at the range of the estimated effects on the hazard rates, but somewhere near the middle using the AIC ranking. This is because, although having high earnings is very strongly associated with leaving benefit, there is just a small proportion of the population who have any declared earnings, so in terms of explaining the overall variation in the cohort’s durations this variable is not as important as the estimated parameters might suggest. Note that benefit type is highly ranked using either approach; primary’s previous year on benefit is reasonably highly ranked under both criteria; and primary’s education is the most important characteristic in terms of the “weighted population” AIC measure.

Conclusion

We used nine years of data on the cohort of approximately 30,000 children who were born in 1994 and had contact with the benefit system before the age of three. Using survival analysis techniques we estimated proportional hazards models to:

• obtain estimates of the independent association of each observed characteristic with the probability of leaving benefit;
• estimate expected durations for sub-populations with specified characteristics; and
• estimate duration dependence for the cohort.

We estimated the empirical survival functions in order to obtain median durations for broad sub-groups defined univariately in terms of characteristics at first contact.

This multivariate analysis was the logical next step after that of Ball and Wilson (2002), which gave a bivariate analysis of factors associated with long benefit durations. We provide here a comprehensive set of results for further interpretation, including some that can be used to help evaluate past policy changes. The work has greatly built our capacity in terms of understanding how to structure the data, what methods are appropriate (and their limitations), and knowing how to interpret the results. There remains much scope for further interpreting the results and thinking through their implications for policy. One obvious development would be to go on to establish some measure of the nature of the outcome that resulted from benefit cessation (e.g. did it endure, or did further spells follow?).

Results of Policy Interest

Once education and benefit type have been controlled for, there is very little difference between benefit durations of children of male versus female primary beneficiaries. However, benefit type is one of the most important characteristics in explaining benefit duration, and given that approximately 90%³³  of children on sole-parent benefits had a female primary beneficiary, while approximately 85% of those on unemployment benefits had a male primary, any implications of this result for policy need to be very carefully thought through.

Even after controlling for education and using a proxy for employment history there is still a very strong association between earnings and benefit duration. This suggests that attachment to the labour market is significant in explaining departure from benefit. Whether or not this labour market attachment might be explained by previous employment history is not fully established, given that we are only able to include one year’s previous receipt of benefit as an indicator of employment history. Interestingly, the association with leaving benefit is even stronger for the partner’s earnings, despite controlling for partnership elsewhere in the model.

Over our eight-year window, the probability of leaving the first spell of benefit mainly increases over time, likely reflecting the effects of children becoming older and more independent. While the strong association of primary’s previous time on benefit with benefit duration suggests that in general for adult beneficiaries the probability of leaving benefit may decrease the longer one is on benefit, from the child’s point of view the liberating effects, on the caregiver, of the children ageing outweighs this negative duration dependence, at least during the child’s first spell.

Results Suggesting Further Work

There is still a difference between NZ Maori and NZ European, even after controlling for all the observed characteristics. This raises questions about what is driving this remaining difference. The fact that for Samoan there is a positive association with leaving benefit suggests that, rather than being purely ascribable to labour market discrimination, there is still some unobserved heterogeneity affecting the results. Relevant to the results for the Pacific groups,³⁴  Ball and Wilson (2002) note

…the high levels of mobility across the Pacific Island populations. Many children may have spent periods outside of New Zealand during their early years thus reducing their overall duration of contact.

A rural/urban distinction might also be a significant factor associated with benefit duration, which we would expect to be quite different between Maori and Pacific populations. More thought is required about what relevant characteristics may not be controlled for.³⁵

The decreased probabilities of leaving benefit associated with the 1996 and 1998 tax cuts are not necessarily inconsistent with what we might expect given that tax rates were lowered for the whole population, not just beneficiaries with children. However, we do need to think through whether there is a possibility that other environmental changes happening at around the same time might have contributed to the results and, if so, whether these can be controlled for. Despite these considerations, it is accurate to say that the results give the effect on exit rate of being on benefit after, versus before, each of July 1996 and July 1998. And given that both the estimates are significant (both practically and statistically), these results are of great interest and warrant further interpretation from a policy perspective.

Unobserved Characteristics

There is scope for using a sample survey to collect some of the characteristics not captured by this administrative data. In particular, a sample survey would be a good vehicle for collecting historical characteristics, such as partnership and employment, in more detail. With an appropriately designed instrument it might also be possible to obtain indicators of some less quantifiable characteristics, such as attitudes and motivations. And with the analysis done to date, opportunities for efficient sample design exist by incorporating expected benefit duration as a stratification variable, in addition to using characteristics of sub-populations to define the sampling scheme such that specified accuracies can be ensured for these.

An issue of key policy interest that we have not been able to assess in the current analysis is the effect of benefit levels on benefit exit rates. To test whether there is an association between benefit levels and exit rates it is important to obtain a measure of payments levels that is independent of the behaviour or characteristics of the caregivers involved. The only potential source of variation which is independent of behaviour and characteristics would arise if the inherent assumptions about the relationship between levels of income and well-being that are built into the payment structure do not hold. Differences in “equivalised” payment levels might then induce differences in exit behaviour. So one possible strategy for identifying the effect of payment levels on the exit rate is therefore the non-linearity of the payment scale relative to the “true” equivalence scale. This approach, adopted by Barrett (2001), is a possible area for further work.

Extending the Analysis

An implication of basing our analysis on administrative data is that our results relate to children who are already in contact with the benefit system. To put these results into the context of the entire population we would need to incorporate each sub-population’s differential probability of making that first contact. This requires integration with some other data source, such as the Population Census or Household Labour Force Survey, or the upcoming Longitudinal Survey of Income Dynamics.

There is scope for competing risks modelling by incorporating the “reasons for leaving benefit” data. This is a generalisation of the type of modelling we have been utilising already, and would enable us to identify the factors most strongly associated with leaving benefit for different reasons (work, partnering of caregiver, migration, changed caregivers, etc).

We could examine the experiences of different cohorts. For example, we could analyse a cohort of young adults having their first contact as primary beneficiaries, or a cohort of superannuitants. If analysing populations where it is not possible to incorporate their “birth” into eligibility inside our window of observation, the possibility of left censoring in our data would necessitate some stronger assumptions to be made in the estimation of our models.

Perhaps most obviously, our possible window of observation will get longer, enabling us to look at a larger portion of total childhood experience.

References

Allison, P. (1995) Survival Analysis Using the SAS System: A Practical Guide, SAS Institute Inc, Cary, NC, USA

Ball, D. and M. Wilson (2002) “The prevalence and persistence of low income among New Zealand children: indicative measures from benefit dynamics data” Social Policy Journal of New Zealand, 18:92-117.

Barrett, G. (2001) The Dynamics of Participation in Parenting Payment (Single) and the Sole Parent Pension, Policy Research Paper No. 14, Department of Family and Community Services, Canberra.

Barrett, G., F. Krsinich and M. Wilson (2002a) Full Report on the Multivariate Analysis of Children’s Benefit Experience, unpublished Ministry of Social Development working paper.

Barrett, G., F. Krsinich and M. Wilson (2002b) Technical article, unpublished Ministry of Social Development working paper.

Birch, Rt. Hon B. (1996) Tax Reduction and Social Policy Programme – Details, 19 February.

Blakely, T. (2002) “Anonymous and probabilistic linkage of census and mortality data: the New Zealand census-mortality study” paper presented at the New Zealand Conference on Database Integration and Linked Employer-Employee Data, March 21-22, Wellington.

Bradbury, B., S. Jenkins and J. Micklewright (2001) “Conceptual and measurement issues” in B. Bradbury, S. Jenkins and J. Micklewright (eds.) The Dynamics of Child Poverty in Industrialised Countries, Cambridge University Press, Cambridge.

Jenkins, S.P and J. Rigg (2001) The Dynamics of Poverty in Britain, Department for Work and Pensions Research Report No. 157, London.

Krishnan, V. and J. Jensen and M. Rochford (2002)” Children in poor families: does the source of family income change the picture?” Social Policy Journal of New Zealand, 18:118-147.

Mayer S. (2002) The Influence of Parental Income on Children’s Outcomes, Ministry of Social Development, Wellington.

Wilson, M. (2000) “The policy response to the employment task force and changing patterns of Domestic Purposes Benefit receipt: a cohort analysis” Social Policy Journal of New Zealand, 14:78-103.

Wilson, M. (2001) “Longitudinal Analysis of Administrative Data: The Ministry of Social Policy’s Benefit Dynamics Project” presented at the Ministry of Social Policy Longitudinal Research and Social Policy Seminar, 5-6 April.

Wilson, M (2002) “Children’s experiences of low family income: what could data integration add?” presented at the New Zealand Conference on Database Integration, 21-22 March.

¹ Acknowledgements
We would like to thank Deborah Ball for literature review and researching regional employment data, Simon Chapple, Sarah Crichton, Stephen Jenkins, Ron Lovell and Nicholas Pole for comments on earlier drafts, and Donald Woolford for the paper’s title.

² Receipt of benefit can be used as a rough proxy for low income. Wilson (2002) points out that “If we set a constant real dollar low income threshold at 60 per cent of median annual housing-adjusted equivalent disposable income in 1998, this locates most, but not all, children in families receiving some income from social welfare benefits over a year below the threshold throughout the period of the [benefit dynamics] study. And it locates most, but not all, children in families with income below the threshold in families receiving some benefits.”

³ The data set is held in the secure environment of the Ministry of Social Development’s data warehouse – the Information Analysis Platform (IAP). It does not hold name and address information that could be used to identify individuals, and access is restricted to a small number of research analysts. See Wilson (2001) for more detail on the construction and contents of the data set.

⁴ The New Zealand social security system has three tiers of assistance. The first tier comprises a range of benefits intended to meet “normal” living costs. Second-tier supplementary assistance programmes provide for specific expenses on the basis of need (such as accommodation costs or health-related costs) and are subject to an income test. Finally, third-tier “safety net” programmes exist for special or emergency situations not catered for by first-tier or second-tier assistance. This analysis is concerned with children’s inclusion in the first-tier benefits only.

⁵ We test the sensitivity of the results to this definition by using a four-week minimum period, and find that the results are similar.

⁶ Because we need to leave a 12-week window before concluding that the spell has definitely ended, this reduces the minimum and maximum follow-up times to 3.75 and 7.75 years respectively.

⁷ The only three variables that are not time varying are: “child’s age at first contact”, “the proportion of first 38 weeks of previous year that the primary beneficiary was on benefit” and (where a partner exists) “the proportion of first 38 weeks of previous year that the partner was on benefit”.

⁸ For partnered beneficiaries, both age and ethnicity are likely to be similar between the primary beneficiary and their partner. Of the partner’s characteristics we decided to exclude age, but include ethnicity. Collinearity between primary and partner’s ethnicity does not bias the estimates but can result in high standard errors. However, results from the modeling indicate that statistically insignificant estimates for ethnicity tend to be due to the small magnitudes of the parameters, rather than high standard errors. If we were to include some measure of partner’s age, it would perhaps be of more interest to look at age gaps between primary and partner, rather than age as such. This is an area for further investigation.

⁹ We exclude from this measure the 14 weeks prior to the child’s first contact. The purpose of this is to exclude from the proxy of labour market experience the period in which the caregiver may have received benefit because of pregnancy.

¹⁰ The earnings are not inflation adjusted. Retaining nominal values has appeal in that the bands chosen align with abatement thresholds that applied at different points in the study period, and location above or below these bands may give some indication of the degree of attachment to the labour market.

¹¹ Note that these were not the only policy changes to occur over the period, but they were the major ones that affected incentives and work-related obligations. See Wilson (2000) for an overview of other, more minor, changes in policy and administration that may have also had some impact.

¹² With provision for deferral or exemption in specified circumstances.

¹³ Right censoring occurs if the child is still on their first spell of benefit in December 2001. In these cases we only have a lower bound for the duration, rather than the actual duration.

¹⁴ Also commonly known as duration analysis.

¹⁵ The baseline hazard is the hazard rate for the base category (see Table 1 for base categories).

¹⁶ Sole female is the base category of the characteristic “partnership and sex of primary’.

¹⁷ Estimated using SAS’s PROC PHREG.

¹⁸ Estimated using SAS’s PROC GENMOD.

¹⁹ Rather than report the proportional effect of each covariate on the baseline hazard rates as , we follow the common practice of reporting the directly, which has the advantage that the estimated standard errors relate directly to these estimates. The closer is to 0 (or equivalently, the closer is to 1), the better is as an approximation to .

²⁰ Note that a fixed amount of time into the spell represents different calendar times for different children, as they can enter benefit at any time within a four-year window.

²¹ Except for unemployment rate, all the characteristics included in the analysis are categorical.

²² See the earlier footnote in this “Results” section, which explains our use of this approximation.

²³ Sensitivity testing of the model on successive cohorts (children born in 1995, 1996 and 1997 having contact before the age of three) shows that the pattern across age categories levels off relatively significantly (Barret et al. 2002a).

²⁴ Strictly, only the first 38 weeks of the previous year – see earlier footnote in ‘Observed Characteristics’ section.

²⁵ Approximately 40% of the cohort are partnered.

²⁶ The policy reform may have also had an important impact in assisting families and children from avoiding entry to a benefit. However, the magnitude of the point estimate (and standard error) suggests multicollinearity between this variable and other calendar variables included in the model.

²⁷ Because part of the cohort is still on benefit when we stop observing them (at December 2001) we cannot calculate median durations straight from the data – this is the “censored” aspect of the data referred to earlier. Note that if we ignored the time-varying aspect of the data we could estimate medians using a simpler survival analysis technique than here (Kaplan-Meir methods).

²⁸ The sub-population with a higher hazard tends to leave benefit earlier, leaving behind the sub-population with a lower hazard. Therefore the “average” hazard declines as a result of the heterogeneity, rather than negative duration dependence.

²⁹ This is an approximation in the sense that it uses the sample proportions as at the first contact.

³⁰ The sample composition across the full range of characteristics is different for each sub-group, and can be calculated easily only at particular points in time. This, combined with the time-varying nature of most of the measured characteristics, is why medians are difficult to calculate from the results of the models estimated.

³¹ So they are effectively weighted by the number of children, in comparison to measures from the beneficiary’s point of view

³² Although, because up to 12-week breaks are disregarded, short-term and administrative breaks will not affect the measure.

³³ As at the beginning of spells.

³⁴ Note that Tokelauan and Tongan also have positive associations with chance of leaving benefit, although these results are statistically insignificant.

³⁵ Such as using appropriately defined treatment and comparison groups via an experimental, rather than observational, approach