The Underlying Instability in Statutory Child Protection: Understanding the System Dynamics Driving Risk Assurance Levels

James Mansell ¹
Senior Analyst, Decision Support
Child, Youth and Family
Ministry of Social Development

---

Abstract

A core component of the level of statutory assurance against the risk of child abuse is hazard detection through notifier surveillance, and through risk screening by both notifiers and the statutory child protection agency. Surveillance and screening are undertaken with a great deal of decision-making uncertainty and, as in most public risk-screening decisions, errors are common. In the New Zealand system it was found that increased attention to failed alarms (in the form of child deaths) drove reactive change to increase the level of risk assurance demanded of notifiers and the child protection agency. The combination of high error rates, high-stakes consequences for some errors and the ability to shift interventions thresholds, and competing pressures to meet incommensurable demands (i.e. to save all children from continued abuse, manage within resource constraints and avoid harming innocent families) suggests that deep instability in the underlying system will always be a feature of statutory child protection. The best that can be done is to tackle the main issues that contribute to the instability; specifically, intolerance of errors, lack of ability to defend errors, and the inability to transparently target a defendable and specified optimum level of risk assurance.

---

Introduction

Since 2001 demand for a statutory child protection investigation, as measured by client notifications to Child Youth and Family² within New Zealand, has doubled. The research presented here was initially motivated by a desire to understand the drivers of this sudden surge in demand in New Zealand (see Figure 1).

Figure 1 Client Notifications, January 1992 – December 2005

The received view tends to be that demand is driven by exogenous variables such as changes in demographics (youth population and/or ethnic mix) and/or poorer social outcomes (e.g. increasing poverty, social isolation, etc) driving increased rates of abuse. While this may provide some of the explanation, the evidence presented here demonstrates that the main factors driving the demand surge are endogenous – factors relating to changes in the behaviour of the child protection system itself.

Understanding the system that can drive surges in demand has also provided a wider perspective on the relationship between the particular system pressures that are causing concern in New Zealand and how these are merely symptomatic of deeper issues for child protection in general.

The findings presented in this paper suggest that the underlying problem facing child protection is the instability of the level of risk assurance demanded, and that this instability is fundamental to the nature of the child protection system.

The evidence suggests that this central issue underlies the seemingly disparate range of difficulties that child protection agencies face: surging demand, inability to forecast or manage demand, the inability to respond to criticism for not being responsive enough (e.g. driven by high-profile child deaths), the inability to defend against criticism for being too intrusive (United Kingdom experience), pressure to apply reactive changes to intervention thresholds and continual pressure to risk-manage intake, and becoming more forensic to avoid errors, thereby shifting resources away from effective intervention (CYF 2000a, CYF 2003a, CYF 2003b, E. Munro 1999, Scott 2006).

The figures below show a comparison of increases in notifications in some states in Australia compared with New Zealand. Although it is difficult to make a direct comparison of the numbers, it should be noted that there have been similarly large increases in notifications in each of these regions (AIHW 2006).

Table 1 Care and Protection Notifications in Australia and New Zealand

Financial Year	Australia					New Zealand
	NSW	SA	ACT	NT	Tasmania
1999-00	30,398	15,181	1,189	1,437	422	30,254
2000-01	40,937	9,988	794	1,551	315	29,348
2001-02	55,208	11,203	801	1,605	508	29,561
2002-03	109,498	13,442	2,124	1,554	741	34,065
2003-04	115,541	14,917	5,325	1,957	7,248	44,916
2004-05	133,636	17,473	7,275	2,101	10,788	53,589

The research here was undertaken within a system-thinking framework. This is an iterative process of investigating the nature of the dynamics of the wider system underlying child protection, modelling this, and then testing it against organisational knowledge and through data analysis.

To this end, the next section, “The Nature of the Child Protection System of Intake”, looks into the properties of the risk-screening process in child protection. “System Pressure to Increase Risk Assurance” then reviews the evidence from the New Zealand situation, in particular, the pressures driving calls for increased risk assurance. “The Effects of Pressure to Increase Responsiveness” summarises the observations of changes in demand trends that accrued due to system pressures, and in so doing the proposed model is tested against the evidence and data analysis. The following section, “Child Protection Systems Under Demand Pressures”, summarises the model illustrating the system driving demand pressures that the New Zealand child protection system is currently under. The final section, “The Underlying Instability of Statutory Child Protection”, places the New Zealand child protection experience into a wider context of providing an understanding of the root causes of the difficulties faced by all attempts to provide assurance against the risk of abuse to children through the work of statutory protection agencies.

It is clear from the results – and should be noted at the outset – that the issues facing child protection in New Zealand should not to be taken as a criticism of New Zealand’s child protection agency in particular. The results here suggest that the system of child protection, as practised in many Western countries, will always place unrealistic demands upon their child protection agencies. The underlying system dynamics will always drive instability and so expose the statutory agencies responsible for delivering statutory child protection to unfair and ill-informed criticism and censure. As will become clear, these agencies are only ever part of a much wider system of statutory child protection.

---

The Nature of the Child Protection System of Intake

Providing a level of risk assurance from abuse to children requires that hazards (in this case the risk of continued abuse) be identified and screened before escalation towards a statutory child protection response (investigation/intervention/placement, etc). To understand how demand can surge (in New Zealand since 2001³) one must take a closer look at the first of these tasks, the nature of child protection agencies’ intake and screening process. “Risk assurance” used here refers to the initial risk-screening function. As will become clear, in fact risk screening occurs throughout the statutory engagement with children and their families.

The Risk-Screening Process in New Zealand

The risk-screening process in child protection is a complex system that includes multiple stakeholders. The community, media and government set demands for levels of risk assurance and so influence risk-screening thresholds. Notifiers4 undertake environmental scanning for hazards to children, and escalate these concerns for action when the notifiers consider them to warrant a formal investigation. These concerns can be forwarded to the child protection agency and/or to non-government organisations (NGOs). The child protection agency’s role is to receive and further screen concerns, and to escalate those that are believed to be of sufficient concern to warrant a statutory investigation.

There are, therefore, a number of screening points in the New Zealand child protection context. For example, in 2004 it is estimated that from one million phone calls (and faxes) per annum, 140,000 (14%) were forwarded to intake social workers on the basis of the belief by the receptionist that they might be a substantive intake concern. Of these 140,000, about 35,000 (25%) resulted in a formal notification being lodged. Each notification was typically for more than one child, giving a final tally of 63,000 client notifications in 2004 (i.e. at 1.8 children per notification).

Each individual participant in the intake system – the notifier, telephone service operator and intake social worker – must determine whether to escalate the concern or not. At each of these steps multiple factors may influence the probability of the concern being escalated further into the intake system. The level of risk assurance provided within New Zealand is a combination of the community’s demands, the notifier’s environmental scanning (surveillance) and escalation threshold, and the child protection agency’s own risk-screening and escalation threshold (see Figure 2).

Figure 2 Modelling the Wider System at Intake

Note: TSO = telephone service operator; ISW = intake social worker; FAR = further action required; NFA = no further action.

One of the pervasive features of this system is that all three decision makers (notifier, telephone service operator and intake social worker) screen concerns under conditions of a high degree of uncertainty and potentially serious consequences for making an error.

Decision Making Under Uncertainty

It is well recognised in the social work literature that social workers have very difficult decisions to make (Benbenishty and Chen 2003, Gambrill 2005, Gambrill and Shlonsky 2000).

On the basis of [notification] information elicited some cases are clear-cut. However there are “grey area” cases caused by complex, unclear, ambiguous or unreliable information. Decisions in these circumstances can be characterised as “decision making under uncertainty”. (CYF 2000a)

Caseworkers must distinguish between child neglect, bad parenting and the effect of poverty and they must do this without the aid of accurate assessment tools … Rarely is all relevant information available, hampering problem solving efforts. (Gambrill and Shlonsky 2000)

Assessing risk and identifying child abuse and neglect are difficult tasks … Some mistakes are inevitable because they are due to our limited knowledge. (E. Munro 1999)

Causes of uncertainty include:

• ambiguity about what counts as a reasonable definition of and evidence for a
• concern (particularly difficult for emotional abuse and neglect, for example)
• resistance by people being interviewed
• conflicting information
• lack of information
• time pressure
• limitations on human judgment; that is, the difficulty of integrating a range of different types of information to arrive at an accurate decision.

Notifiers and telephone service operators have similarly difficult decisions to make. They also face difficulties due to a lack of experience and training to help distinguish statutory-level child protection concerns from other concerns.

The uncertainty has three important implications.

• Thresholds for intervention can shift in response to new definitions or pressures. “These professional decision making issues can impact on threshold management” (CYF 2000b).
• “Risk” is not a concept that is distinct from social and cultural beliefs about it. Therefore, demand is not an objective feature of the world but can shift in response to new concerns and beliefs.
• Errors are common and will always be made.⁵ “Since risk assessment is, by definition, making judgements under conditions of uncertainty, there is an unavoidable chance of error” (E. Munro 1999).

Errors in Child Protection Decision Making Have High Stakes and are Symmetrical

When errors are made they can be of two sorts: missing a concern (failed alarm) and over-reacting (false alarm), and there can be high-stakes outcomes for both sorts of error. For a failed alarm (a false negative or Type I error), the worst case is an avoidable child death. In the best case, the family is re-notified a few days later and the concern is caught without too much further harm to the child. False alarms (a false positive or Type II error) are investigations where no harm or abuse was substantiated. These can also be highly destructive.

It is hard to exaggerate the extent of rage, humiliation and intense fear felt by many parents who are subject to child protection investigations where the concerns are not substantiated. (Scott 2006)

Forwarding low-risk concerns for investigation may also indirectly affect more needy children by mis-targeting resources – concerns that are more serious are not given the full attention required, making them vulnerable to failure. In either case, the consequences of an error (failed alarm or false alarm) in decision making can be damaging for the child and their family.

There can also be serious consequences for the social worker, the child protection agency, notifiers and other stakeholders involved. All these stakeholders may feel shame and doubt regarding their performance, and be the subject of sustained criticism and even formal censure.

---

System Pressure to Increase Risk Assurance

Research⁶ has revealed that from the late 1980s until the present there has been increased pressure to raise levels of risk assurance through expanded surveillance and lowering screening thresholds. This pressure is due to increased intolerance of the risk of child abuse, which in turn is driven by high-profile attention to abuse within New Zealand. The abuse-caused deaths of children who were previously known to agencies caused an intense level of media attention in 1988, 1994, 2000 and 2003 (see Box 1).⁷

Box 1 Four Main Periods of Media Attention to Child Deaths and Related Reports 1988: “New Zealand had its first high profile child death review, Dangerous Situations: Independent Inquiry on the Death of a Child. This focused public and professional attention on child abuse in a way not previously experienced. Reactions included disbelief, horror, blame and accountability, and led to an intense questioning of child protection practice.” (Smith 1998) 1994: The death of Delcelia Whitika (1991) and Craig Manukau (1992) both fuelled a “blaze of publicity” when reports were released in 1994 (McLoughlin 1994). June–August 2000: Leading up to August 2000 there were high-profile stories on several child deaths – the death of Lillybing (June); the child-death report on James Whakaruru (released 25 June); an editorial article “We must stop the slaughter” (30 July); and intense coverage of other child deaths: Hoana Rose Matiu (15 August) and Sade Trembath (28 August). September–December 2003: Perhaps the most intense period of attention was in late 2003. The first high-profile story regarding the death of Coral Burrows reached national news headlines on 19 September. For the next three weeks coverage continued, including editorials such as “Our children are crying” (20 September) and “Coral’s death must not be in vain” (27 September). CYF was accused of fault in Coral’s death by “ignoring a call” from Coral’s father (18 October), thus sparking a review of CYF’s handling of the call. The resulting Duffy Report identified the intake social worker at the Call Centre who apparently failed to escalate the concern when, with hindsight, it could be seen that this would have been beneficial. A series of feature articles reviewed and criticised CYF’s performance history by listing previous child deaths (1–13 November). The release of several other reports further fed media interest and were taken as evidence of systemic failure on the part of CYF and other agencies. These included the findings of the Baseline Review (released in September), the results of a UNICEF report that indicated New Zealand had one of the worst rates of preventable child deaths in the OECD (September), the Children’s Commissioner’s report into the deaths of Saliel Aplin and Olympia Jetson (November), and the release of CYF’s case review regarding Kelly Gush (December).

All four periods (and there were probably others) displayed increased intolerance by the public and other key stakeholders (Children’s Commissioner, media, the government of the day, and Child, Youth and Family itself) to child abuse and especially to the risk of failure to intervene.

A qualitative analysis of the content of the media articles in mid-2000 and late 2003 and of the recommendations made in reports in these periods found that the following messages dominated.

• New Zealand has one of the highest abuse rates in the OECD.
• Some of the responsibility for New Zealand’s “epidemic” of harms to children is a failure on the part of the community to take responsibility and notify concerns.
• There is a lack of cooperation between agencies (including Child, Youth and Family, the Accident Compensation Commission, health professionals and the police), leading to missed opportunities to recognise risks to children.
• Agencies and the public at large need to become more responsive and escalate any potential concerns to avoid further child deaths.
• The functioning of Child, Youth and Family and other agencies is failing children at risk of abuse.

These messages translated into calls by key stakeholders to increase screening, widen the net to capture all concerns, lower thresholds and, when in doubt, lean towards false alarms, in order to mitigate the risks of further errors of the failed-alarm type.

Police, the Accident Compensation Commission, other agencies and health professionals were encouraged to be more vigilant regarding risks to children and escalate concerns for appropriate action. Recommendations to the Commissioner for Police encouraged increased surveillance of risks to children and escalation of these for further action. “Consider the desirability of instructing all staff to complete family violence reports (Pol 400) [and] to also record an assessment of the risk to children who witnessed family violence” (CYF 2003a).

Child, Youth and Family came under intense pressure to improve intake processes in order to capture all child abuse concerns. This resulted in the following recommendations.

• Try harder not to lose any concerns forwarded by notifiers. “The Department [CYF] must ensure that systems for receiving notifications are robust and offer no opportunities for notifications to receive no response” (CYF 2003b).
• Improve intake standards and processes, and emphasise the role of the Call Centre. “With the continued implementation of the national Call Centre, the Department continues to develop the system for receiving notifications. ... The Department is considering the [Duffy Report] recommendations which are aimed at enhancing intake systems” (CYF 2003b).
• Tell social workers to capture all potential concerns, and to lodge all siblings. “A reminder to social workers to include all at-risk siblings in notifications” (CYF 2003a).
• When a notification is in doubt, it nevertheless should be lodged and recorded as a formal notification. “CYF received a new notification on 3 September 2001. This information should have been recorded as a new notification, it was not ... The social worker and the supervisor questioned whether this information was new information or whether it related to historical information” (CYF 2003a).

These reactions suggest that the main response in New Zealand was, when in doubt, to err in favour of escalation for fear of missing a real concern. To err on the side of caution through increased surveillance and screening leads to an increase in the number of concerns escalated by notifiers. Child, Youth and Family intake social workers also responded by lowering their thresholds for intake due to pressure to forward concerns when in doubt for a formal statutory investigation.

This led to increased statutory-level risk-screening (through investigations) by the department.

---

The Effects of Pressure to Increase Responsiveness

Child, Youth and Family’s social work data were disaggregated and examined to investigate changes in New Zealand from 2000 to 2004, to see if these aligned with our understanding about the nature of the system and pressures placed on it. That the demand for increased responsiveness drove the surge in demand is well supported by the client notifications data.⁸

Correlation between Media Attention and Notification Spikes

Analysis of weekly notification spikes from 2000 to 2004 (Figure 3) revealed that the level of media attention⁹ each week was found to correlate with residual spikes in notification rates (controlling for trend and seasonality). Weeks with high levels of media attention were significantly likely to have a higher notification rate.

Figure 3 Media Attention and Notification Spikes

Further analysis revealed two periods of extreme growth when observing the residual spikes in notification rates since 1997: one in August 2000 and one in December 2003 (Figure 4). These two spikes in notification rates occurred in the months immediately following the most intensive periods of media attention (June and July 2000, and October to December 2003) and the many recommendations made in November 2003 by CYF (see previous section).

Figure 4 Residual Spikes in Notification Rates since 1997

Widening the Net to Lodge Siblings at Risk

While distinct client notifications rose 89% between August 2001 and August 2004, the number of distinct notifications rose by 58%. The number of clients (typically siblings) who are lodged against each notification increased from 1.5 to 1.8 over the same period (Figure 5). The increase in average number of clients per notification accounted for the 31 percentage-points difference.¹⁰

Figure 5 Number of Clients per Notification, 1997–2004

Trying Not to Lose Forwarded Concerns

A fax machine was introduced in an effort to ensure that anybody not able to reach Child, Youth and Family on the phone would have an alternative route. This increased responsiveness had an unforeseen effect of lowering response thresholds, in that faxes were found to have a very low threshold for being lodged. Roughly 95% of faxes get lodged, whereas only 20–25% of phone calls get lodged. This probably explains the low “further action required” (FAR) rate for faxes.

Phoned client notifications rose by a small amount (20%) compared with faxes (300%) from 2001 to 2005. Faxed notifications now account for over half of all client notifications lodged.

Shift to Professionals Increasing Environmental Scanning

Since July 1997, Child, Youth and Family has recorded the type of notifier making the notification. Nearly all notifier types increased from their average of 1998/2000 to 2005 (see Table 2).

Table 2 Notifications by Type of Notifier, 1992–2005

Note: PFV = Police Family Violence

Between 1998 and 2001 there was a shift in the pattern of notifier behaviour (Figure 6). All professional notifier groups and neighbours were increasing, but this increase was offset by a decrease in notification rates from family members during this period. Notifications from family and whanau also grew (14%), but at a lower rate and only after falling between 1998 and 2001 (by 32%). There has been a shift in the contribution of family and whanau notifications. In 1998 they accounted for 32% of all notifications, whereas by 2005 they accounted for only 15%.

Figure 6 Notifications by Notifier Type (12-month moving average)

The Emergence of a New Class of Risks to Children: Police Family Violence (PFV) Notifications

In 1995 research emerged suggesting that children who witness family violence are at increased risk of physical abuse themselves and may suffer a range of social and mental health problems (Fielding and Taylor 2001).

One response from Police, Child, Youth and Family and the National Collective of Women’s Refuges was to introduce tools to be used by Police to estimate the risks to children present in family violence situations. From July 1999 to January 2000 there was a pilot programme in six sites where a Safety Assessment Form was to be completed by Police attending family violence incidents where there were children present. Police then had three options:

• refer to community support services (such as Women’s Refuge)
• notify Child, Youth and Family
• undertake urgent action under the Children, Young Persons and Their Families Act 1989.
• The evaluation results found that most forms went to Child, Youth and Family.

The Safety Assessment Form was replaced by a modified police form called the POL400. The use of the POL400 to collect information about risks to children in family violence situations, then forward this to Child, Youth and Family, was gradually rolled out across New Zealand over the subsequent five years.

A closer look at rates of Police Family Violence (PFV) notifications (via POL400 forms) reveals wide regional variation in the number forwarded to Child, Youth and Family. Increases in PFV client notifications within each region can be sudden and large. Subject matter experts within the Police and Child, Youth and Family pointed out that in all instances, regions with high levels of PFV notifications were areas that had, just prior to the increase, appointed individuals with a particular interest in family violence (usually Family Violence Response Coordinators). In other cases spikes in notification rates coincided with the introduction of a Child, Youth and Family pilot project to improve the information on the POL400 family violence form. The roll-out has been incremental and is due to the influence of local individuals or local initiatives.

Since 2001, more police regions have begun using the POL400 form, leading last year to over 13,000 PFV notifications. PFV client notifications already account for most of the increase (48%) in notification rates observed between 2000 and 2005.

The new PFV notifications appear to be uncovering softer concerns as well as new concerns of a similar level of severity to other notifications. The average FAR rate of PFV notifications (60%) is lower than for other notifier types (84%), suggesting that some concerns being forwarded are of a lower risk. However, some new PFV notifications are uncovering new severe abuse – the actual number of substantiated cases has increased due to PFV notifications, even if the substantiation rate (per investigated notification) has come down.

The still extremely low rates of PFV notifications in some areas in 2006 suggests that there may still be a substantial level of latent demand in terms of both softer concerns and more serious concerns. Figure 7 shows monthly notification rates between 2001 and 2005 for each of New Zealand’s child protection agency branches.

Figure 7 Police Family Violence Notifications, Monthly Rates by Site, 2001–2005

Lower Substantiation Rates and Higher Numbers of Substantiated Cases

Moving to an increased level of screening suggests a higher rate of false alarms would occur, due to a softer array of concerns being investigated. This in turn suggests that FAR rates and substantiation rates would decline across all concerns investigated. Initial indications suggest that this may have indeed occurred, as the proportion of notifications that have been escalated for an investigation (FAR rates) has declined (–8%).

Furthermore, notifications forwarded for investigation have a lower probability of being substantiated with physical, sexual abuse or neglect findings. The biggest fall is for physical abuse findings. This has almost halved – reduced from 0.1 to 0.05. Physical abuse findings might be expected to be the type of finding least likely to be affected by changes in substantiation thresholds at investigation; i.e. there is probably less uncertainty in assessing physical abuse, given its more observable symptoms, than there is for neglect or for emotional abuse. So, conversely, if softer cases are being escalated for investigation, then this is the type of finding where the rate is most likely to reduce.

Table 3 Substantiation Rates, by Investigation Findings, 2000–2005

Increased Re-notification

Increased focus on screening would tend to drive up the rate of re-notification, through:

• greater probability of escalation of historical cases
• increased attention to linking new concerns to historical cases
• the effect of increased attention on screening drawing resources away from intervention, and increasing the probability that concerns will linger and re-surface.

It was found that re-notifications did indeed accelerate from late 2001 and again in November 2003. Re-notifications now make up over half of all client notifications (Figure 8).

Figure 8 Estimated Growth in New to All Legacy Cases (rolling 12-month average)

A Major Change to System Dynamics at Intake

Child-protection risk assurance occurs within a constrained resource environment. At some point, the increased responsiveness will drive demand to a level that hits a capacity ceiling – by driving spare capacity down. As spare capacity diminishes, the pressure to manage demand increases. This may manifest as calls to increase capacity to deal with extra demand and/or feeding back into the level of risk assurance and thereby raising the threshold for investigations and interventions.

When intake is branch-based (e.g. the branch likely to undertake the investigation is also the branch making the intake decision; that is, whether to do an investigation) then demand management is locally driven. Remembering (from the section above, “System Pressure to Increase Risk Assurance”) that uncertainty in decision making implies flexibility; social workers can adjust their intervention threshold to deal with surges in demand.

Interviews with experienced branch-based intake social workers revealed that “gate keeping” when under work pressure was done a lot during the 1990s by referring to other services (for a non-statutory response) so placing a reasonably high threshold for formally lodging and undertaking a statutory investigation.

If the workload is already large and if capacity is believed to be constrained, then the threshold for taking on new cases will rise, not necessarily through explicit management directives, but simply because staff are aware of the available capacity.

Under a branch-based model of intake (pre-2001), when media attention demanded increased responsiveness, this had to be optimised with capacity by raising thresholds so that branches would not be overwhelmed by demand.

Figure 9 Branch Based Intake

Note: Figure 9 is a causal-loop diagram. Arrows represent causal relationships between variables. For example, spare capacity has a causal relationship with demand management. Each causal link has a polarity assigned to indicate how the dependent variable (effect) changes when the independent variable (cause) changes. An (S) indicates that the cause and effect move in the same direction. If the cause increases, the effect increases above what it otherwise would have been. If the cause decreases, the effect decreases below what it might have been. An (O) indicates that as the cause increases, the effect will move in the opposite direction by decreasing. If the cause decreases, the effect will increase.

In August 2000, the notification spike – due to media attention to the death of James Whakaruru – could not last due to constraints on capacity that led branches to increase the threshold for lodging a formal notification. Looking further back, the department had also been under similar pressures for most of the 1990s. The Brown report (Brown 2000) found that the department was seriously under-resourced and demoralised, and had difficulties with recruitment and retention. And, as Smith suggests, social workers were under increased pressure to spend more time at the front end investigating cases rather than case-managing them (Smith 1998). “Demand”, as measured by client notifications, remained a fairly constant 27,000–33,000 per year for the 10 years prior to 2001. The potential for increased lodged and investigated notifications was masked through shifts to thresholds of intervention at branches. As one social worker described it, “We had to undertake gatekeeping so [we] sent people to other [non-statutory] places”.

Branch-based intake is a naturally self-adjusting system, where workload mediates the threshold for intervention. So the effect of increasing demand is not an increase in notification rates, but rather an increase in the threshold for intervention. This was one of the main rationales for introducing a call centre:

The Call Centre was established in response to a range of issues that were affecting the delivery of our intake social work services. The primary concern was that there was an inconsistent approach to intake across sites, with decisions being made about which notifications would be accepted and which rejected not on the basis of the needs of the case, but on the basis of perceived capacity to respond … There is a tension between resource considerations (capacity) and professional decisions (client needs) ... Where resources to investigate notifications are overcommitted, there is a potential for high levels of threshold management to operate as intake. The call centre approach is intended to separate the resource decision from the professional decision. (CYF 2000b)

The Call Centre was established in October 1997 as a pilot project to serve the Auckland metropolitan area. The Call Centre was extended to include Northland in September 1998 and Hamilton in January 2000 (CYF 2000b). It was rolled out to the rest of New Zealand in early 2001 and received over 80% of all client notifications by 2002.

From a system-thinking perspective, the decision to shift from branch-based to predominantly Call-Centre-based intake is a significant change to the child protection system dynamics at intake. Call-Centre-based intake leads to a majority of intake decisions being more decoupled from the consequences of these decisions. The notion of the level of coupling a system exhibits refers to the level and nature of feedback a system has. In tightly coupled systems the feedback is fast, direct, and carries a lot of detail. The more loosely coupled the system, the more the feedback becomes delayed, indirect and less informative. Intake decisions via branches are more tightly coupled to investigations than are Call Centre intake decisions.

Figure 10 Call Centre-Based Intake

In practice, local demand management undertaken at branches through threshold management (lowered levels of risk assurance) meant that real demand levels were less transparent nationally; that is, they did not turn up in the official measure of demand, the notification figures. In system thinking, the raised intervention threshold is a “hidden queue”.

The shift to the Call Centre system diminished the ability of local offices to manage demand via this hidden queue (local intervention-threshold management). As a result, demand levels were reflected in the notification figures and so became more transparent.

Moreover, the nature of demand management has changed. Whereas before, demand management could be undertaken locally through threshold management, this must now be done at a national level. It is not surprising that various recently proposed demand-management initiatives are really attempts to manage demand through raising intervention thresholds again (as branches had previously done locally). In addition, the more transparent recording of demand via notifications rather than the hidden measure (the intervention threshold) allows the child protection agency to initiate a second method of coping with increased demand by requesting increased capacity through increased funding (see Figure 11).

It should also be noted that the move to a national-level cycle of feedback and demand management has increased the delay time in the feedback loop. Demand is now managed in cycles of months and years on a national basis, not in days and weeks on a local basis.

Finally, the decoupling of intake from investigation (via introducing the Call Centre) meant that recorded demand for a statutory response was uninhibited by capacity, and so it could increase through lowered thresholds for intervention. An evaluation of the introduction of the Call Centre to Auckland found that this was the case:

Auckland sites accepted and take calls at the rate of 17 per 1,000 from the target population, against the national average of 22/1,000. Since the Call Centre has been receiving intakes, the figure has increased to 23 per 1000. This demonstrates the effect of separating the professional decision (need to respond) from the resource decision (capacity to respond). A consequence of needs-based assessment at intake is that the capacity issues (ability to respond to intake) have been shifted to the investigation and assessment phase and have had a significant impact on workload. (CYF 2000b)

Client notifications accelerated when the Call Centre was rolled out nationally in 2001. Recorded demand was able to accelerate further due to the intense concern in September 2003, which forced individuals and the department to lower thresholds further as an attempt to mitigate further child deaths (see Figure 11).

Child deaths might have been the trigger for increased surveillance and screening, but the Call Centre was the element that enabled increased demand to be manifested as increased lodged notifications being escalated towards statutory investigations rather than directed elsewhere.

Figure 11 Introduction of Call Centre and Notification Rate Surge

---

Child Protection Systems under Demand Pressures

The New Zealand child protection agency is under demand pressure, as are a number of other Western child protection agencies. The model that explains this – that demand pressures stem from reactive change to calls for increased risk assurance, through increased surveillance and responsiveness to concerns within the community – is supported by the evidence, including:

• correlation between high-profile media events and subsequent spikes in notification rates
• sudden 300% rise in lodged faxed concerns
• increase in the average numbers of siblings per notification
• lowered FAR and substantiation rates and increased re-notifications
• exponential growth of Police Family Violence notifications and the finding that this was highly deterministic (based on changes to processes and behaviours)
• lowered threshold for intervention at the Call Centre
• sudden surge in demand correlating with the roll-out of the Call Centre
• lack of demand surges during the 1990s under similar pressures, but where demand was constrained by branch-based intake
• that all of these trends occurred at key inflexion points, where intolerance to child abuse and calls to be more responsive were found to be greatest.

The trends observed in the data are insufficient on their own to confirm that the system model given above is correct. There may be competing explanations for each of the observations.

The evidence suggests that the demand surge was driven by increased community intolerance causing reactive changes to behaviours and processes that in turn led to the untapping of previously latent demand. Increased responsiveness manifests itself as new concerns being observed and escalated by notifiers. The threshold for capturing and formally lodging concerns as “client notifications” has also been raised due to the child protection agency responding to recommendations to increase risk assurance. The child protection agency tightened intake procedures, demanded a greater level of formal lodgement and investigation (of “all notifications when in doubt” and of “all siblings”), and removed the workload bias (by shifting to the Call Centre), thus decoupling intake decision-making from capacity considerations.

Not surprisingly, increased responsiveness had the effects observed: increase recorded demand, lower substantiation rates due to lower thresholds for lodgement and investigation, and increased re-notification. There were interactive effects and multiple causes for most of the observed increases in demand variables.

However, the preponderance of evidence, when linked to the deeper understanding of the nature of the child protection agency’s risk surveillance and screening, and pressures exerted on this system, does paint a strikingly consistent and coherent picture regarding the direction of changes taking place and the causes of this (see Figure 12).

Figure 12 Causal Influence Diagram: How increased awareness and calls for risk assurance drive changes in behaviour that lead to changed notification, substantiation and renotification rates

Note: CPA = child protection agency; (S) = same direction; (O) = opposite direction.

It is reasonable to conclude that increased responsiveness by notifiers and Child, Youth and Family has led to increased reporting of similar-severity concerns (more true positives) in the community now being recognised, as was evidenced by the identification of new concerns substantiated through family violence notifications in some regions. However, there is also evidence that a greater proportion of softer concerns – escalated cases that do not meet existing substantiation thresholds (i.e. false positives) – are also now being forwarded by notifiers and accepted more frequently by the child protection agency for a statutory investigation.

The results also imply that the notion of demand may be in need of clarification. The level of recorded demand is a combination of the level of risk to children within the community and the level of risk assurance the community demands. In this sense, “demand” is not a wholly objective notion. The level of risk assurance required will dictate the threshold for what will be treated as being of enough concern to warrant a statutory response. Demand as measured by notifications to a child protection agency does not reflect the level of abuse within the community; it more accurately reflects the extent to which that community desires a level of assurance for the risk of abuse to children.

The true level of demand within the community is not observed directly. It is mediated by the current level of risk assurance of the child protection system. The level of recorded demand (i.e. as measured through rates of client notifications) is a function of the level of real demand within the community multiplied by the level of statutory risk assurance demanded by that community.

It is clear from this, and from other evidence, that there is significant potential for “demand” to keep increasing and to unveil further latent demand¹¹ within the community, as follows.

• There is, potentially, latent Police Family Violence demand in some areas that to date have resulted in almost no notifications. These areas could potentially send the department an estimated further 5,000 to 10,000 notifications per year.
• Recent estimates suggest that only 20% of avoidable child deaths in New Zealand are of children known to Child, Youth and Family. Using this as a guide, we might expect that notifications would have to increase by a great deal because of unreported latent demand (Doolan 2005).
• The level of risk assurance demanded could increase due to further increases to intolerance of risks of missing abuse concerns that have been directed towards the department. Since the department only lodges an estimated 20–25% of all calls directed to intake social workers, the portion of phone calls that could be lodged if a lower threshold were adopted might increase.

The surge in notifications should therefore be understood as a reflection of an increased level of risk assurance demanded by the community rather than reflecting increased risk of child abuse in New Zealand. This is not to say that there is no influence by exogenous factors. However, most of the social indicators have improved over this period (MSD 2005), so we might expect exogenous factors to have softened notification rates rather than increasing them.

Changes to child protection agency and notifier behaviour were honest attempts to meet the new expectations of stakeholders that there should be more risk assurance against child abuse in New Zealand. However, these stakeholders had insufficient understanding of the system dynamics underlying statutory child protection that would inevitably frustrate them, and which provide much of the impetus for the criticism levelled at the child protection agency and at notifiers.

---

The Underlying Instability of Statutory Child Protection

The task of providing statutory child protection is inherently unstable, involving as it does multiple conflicting demands. The core dilemma is, “How [do] we answer the paradox of not intervening until it’s too late, and of being overly interfering?” (Lytollis 1996). This often-voiced concern is a consequence of the properties of risk screening itself, and is not due to any fault of the stakeholders acting within a statutory child protection context. Hazard detection and risk screening are an uncertain business and sometimes decisions must be made under conditions of great uncertainty. Risk-screening systems designed to detect hazards are fallible and errors will always be made. Meeting expectations to increase risk assurance and so avoid further failed alarms must also entail increased demand and increased risk of false alarms.

While there might be opportunities to improve accuracy a little, there will always be errors made. This being the case, a second strategy might be to err in favour of further action when in doubt, so notifiers and child protection agencies lower the threshold for escalation towards a statutory response. In risk-screening terminology, erring in favour of false alarms to find true cases is akin to reducing specificity. The specificity of a risk-screening tool is the extent to which risk screening correctly finds true negatives – or more importantly, the lower the specificity the greater the number of false positives.

Expanding risk assurance through decreased specificity allows the capture of more concerns that merit a statutory response. However, this also requires a proportionally greater rate of false alarms and an overall increase in the total number of errors (see Figure 13).

Figure 13 Error Trade-off Curve

Note: This graph shows the expected function of the trade-off between making different sorts of errors. For example:
Point A) If you wish to make the least total amount of error then aim for roughly even numbers of false positives (0.2) and false negatives (0.2) for a total proportion of errors of 0.2 + 0.2 = 0.4. Error bias = 1:1.
Point B) However, if you particularly wish to avoid false alarms (aim for 0.1) then you will need to make many more failed alarm errors (0.4) for a total number of errors of 0.1 + 0.4 = 0.5. This would be the cheap intervention threshold model where we would accept that the state cannot mitigate all risks but targets resources well. Error bias = 55:1 in favour of failed alarms.
Point C)   This is the position of agencies with surging demand, trading-off an increased number of errors to find the maximum number of cases where they could provide help. Error bias = 1:55 in favour of false positives. The costly model with a wide net.

Most changes in behaviour in New Zealand were to decrease specificity – notifiers should forward concerns when in doubt, intake social workers should lodge when in doubt, and lodge siblings as well – thus attempting to capture and screen softer concerns in the hope of detecting the rare chance of an apparently soft concern escalating to a major risk. The unintended consequences of this are that the extra demand increases workloads and there is an increased chance of causing harm to other families, both of which feed back into calls to raise thresholds.

Demand-Driven Retrenchment

Increased risk assurance through lowered thresholds for a statutory response will result in finding genuinely new cases and also investigating a greater number of false alarms. The resulting increase in demand uses up spare capacity and places extra strain on limited resources. This either causes poorer performance because resources are stretched too thin, or some attempt to manage demand or increase capacity.

• As discussed (in the section “The Effects of Pressure to Increase Responsiveness”), demand management at a local (branch) level can be undertaken by raising the intervention threshold, which lowers the level of risk assurance, thereby setting the intervention threshold that lets only as many cases through as can be dealt with.
• At a national level, similar sorts of “demand management” initiatives seek to lower risk assurance by raising thresholds for intervention, and turning back cases thought to be of lower risk, or pushing these in non-statutory directions.
• A third alternative is to request funding increases to increase capacity to cope with extra demand.

Regardless of which strategy is used (and Child, Youth and Family has used all three), dealing with extra demand will inevitably lead to backwards pressure to lower the level of risk assurance. Local and national demand-management initiatives seek to do this directly – though this may not be recognised as such. Demand-driven expansion of capacity will inevitably reach a ceiling, where providers of funding will begin to question burgeoning demand levels and so drive attention towards less costly responses than statutory interventions, so driving increased attempts to manage demand through lower levels of risk assurance (demanding higher thresholds before escalation towards a statutory response).

Retrenchment Driven by Social Cost

The situation is similar when one considers the social costs of statutory investigations and interventions. As well as the desired increase in true positives, increased risk assurance will also drive an increased number of false alarms – a false alarm being a case that is escalated for a statutory investigation or intervention that would not typically meet criteria for this level of response. One estimate might be the number of cases escalated for investigation where there was no substantiation of abuse allegations and no statutory concerns raised on the part of the investigator.

Some commentators are beginning to question how benign statutory investigations really are. The fear is that casting the net wider to capture a greater number of true positives may in fact cause increased harms through increased false alarms:

In my view it is almost certain that our current unsubstantiated child protection investigations are actually increasing the risk of child abuse and neglect for many children. For this I believe we will one day be rightly held morally responsible, as the capacity to cause further harm to families on such a massive scale is so self-evident … We have become so concerned about false negatives in child protection that we ignore the adverse effects of false positives [false alarms]. (Scott 2006)

In the process of detecting true incidents of abuse, large numbers of families suffer the distress of investigation with no help being offered. (E. Munro 1999)

So far, New Zealand has yet to experience high-profile criticism of being overly interfering. In case there is a danger of thinking this overly theoretical, some experiences from the United Kingdom may prove sobering:

Moving the threshold to reduce one type of error automatically increases the other type. Applying this to child protection history, when society was outraged by the death of Maria Colwell [United Kingdom 1973], professionals responded by lowering the threshold for intervention to minimise the chances of missing another child and such extreme danger. This necessarily led to more families with low actual levels of abuse being caught up in the net … [Fearfulness of missing a serious case] ... had the inevitable result of making professionals prefer to overestimate the risk of abuse since the consequences of underestimation were so severe. This in turn provoked a backlash, with critics claiming that social workers were overreacting and interfering in family life. Myers (1994 p4) gives examples of the emotive language used. The child protection system was described as “trampling the rights of innocent citizens” and engaging in “hysterical witchhunts”; social workers were likened to “Nazis”, McCarthyite persecutors and the KGB. (E. Munro 1999)

Child protection may be beneficial to some children through mitigating the risk of continued serious abuse. However, this benefit should be weighed against the risk of social costs to innocent families and other children by forcing them through a statutory child protection response (statutory investigations and statutory interventions). Whatever your views on the level of social cost, if statutory investigations and interventions can be harmful, then we are unintentionally trading off the wellbeing of some children and families to mitigate the risks of abuse for other children. There will be an optimum level where the trade-off of false alarms and harms to innocent children and families will outweigh the benefits of uncovering genuine new cases (true positives). The real question is, what should the trade-off be? Do we investigate five innocent families to find one further new case? Do we investigate 10 to find one? Fifty? Or, as one experienced social worker suggests, 1,000?

That there must be a trade-off is apparent if we think how repugnant the public would find it if all families were subject to investigations to provide 100% risk assurance that we do not miss any new cases. This thought experiment is not wholly absurd: England is considering the introduction of a national database for all children where professionals can enter an indication if they have any cause for concern about a child’s health or development.

Whatever the optima, screening more and more families to find remaining abuse will have a ceiling beyond which it will lead to feedback into calls for lower thresholds for intervention – less risk assurance.

The system dynamics of the risk-screening trade-off are pictured in the causal-loop diagram in Figure 14. Increasing risk assurance tends to identify more true cases (true positives) and decrease the number of cases missed (false negatives). This decreases the amount of abuse and the chance of high-profile cases, so taking pressure off calls to increase risk assurance (left-hand loop).

However, at the same time as more risk assurance provides increased risk assurance to some children (at risk of abuse), the greater number of false positives drives up risks of social costs to other children. In addition, the increased true positives and false positives diminish capacity and thus drive up the pressure to increase capacity or “manage demand” (right-hand loops).

Figure 14 Underlying Child Protection System Instability

Note: (S) = same direction; (O) = opposite direction.

Within a systems-thinking paradigm, both sides have negative-polarity feedback loops. Negative feedback loops tend to produce goal-seeking behaviour. They seek balance, equilibrium or stasis by bringing the state of the system in line with a goal. If there is a discrepancy between the desired state of the system and the actual state, then corrective action will be undertaken to bring the system back into line with the goal. Note here that the goal does not need to be explicit or even known about.

So, in our example, the statutory child protection system seeks three kinds of optimum trade-off:

• the trade-off between the level of risk assurance and level of intolerance to abuse
• the trade-off between the demand (funding and resources) level and risk-assurance level
• the trade-off between harms to children at risk of abuse and those at risk of unnecessary statutory investigations.

Simplifying this, there are two negative feedback loops influencing risk-assurance levels that tend to work against each other. Levels of abuse drive increased risk assurance to reduce the level of abuse, while increased risk assurance destabilises demand and social-cost levels, tending to drive risk assurance back down.

Figure 15 Underlying Nature of Statutory Child Protection System

Note: (S) = same direction; (O) = opposite direction.

To add to this, there is also delay in the feedback for both sides of the loop. The feedback of increased demand into corrective action to manage demand used to have a short delay under branch-based demand-management, but now the delay can extend for months and years as burgeoning notification rates drive national-office demand-management initiatives. Similarly, the awareness of social costs may take months, years or even decades to emerge as a concern driving corrective action to reduce risk assurance.

There is also a delay between altering the level of risk assurance and the impact on abuse within the community. It takes time for increased risk assurance to add value to children at risk of further abuse. For high-profile child homicides, there is delay in publishing child-death reviews.

When there is significant delay in the goal-seeking behaviour of a negative feedback loop, this tends to manifest as oscillating behaviour. Delays cause overcorrecting behaviour that results in continual adjustment around the optimal level.

Figure 16 Predicted Behaviour of System = Oscillation of Risk Assurance Levels

The competing demands between providing risk assurance against abuse to children and the trade-off this creates in increased demand and social costs for other children leads to deep instability in the statutory child protection system. Unfortunately, the statutory child protection agency and notifiers are the meat in this sandwich. They are berated for being unable to forecast demand or for asking for more money, while at the same time they are censured for failing to provide enough risk assurance for child abuse.

In practice it is even more difficult. As was demonstrated with Police Family Violence notifications, in some regions there is clearly over-sampling – high levels of risk assurance and so pressure to manage demand – whereas in other regions there are almost no Police Family Violence notifications, suggesting that we are providing a low level of assurance to children exposed to family violence in these regions. The competing pressures can be coextensive and thus provide a very real dilemma that occurs geographically rather than over time.

The real issue for statutory child protection is the level of instability in the level of risk assurance applied. Managing demand and trying to avoid further high-profile errors are attempts to deal with symptoms of the level of risk assurance adopted – demand and child deaths, respectively. The real problem is not the particular response of the child protection agencies to become more or less responsive. Child protection agencies are merely adapting in a “well-intentioned effort to respond to the unreal expectations of the general public” (E.M. Munro 1999).

The perceived inadequacy of New Zealand’s child protection response is not solely the fault of New Zealand’s child protection agency. Child, Youth and Family merely made conscientious attempts to meet unrealistic and poorly considered demands placed on it by multiple stakeholders, as reflected particularly by the media. The inability of the agency to provide a sound response to errors and its tendency to react to events and external pressure without considering unintended consequences would not have helped matters.

The solution is to focus on the real issue: how to stabilise the risk-assurance level, thereby avoiding reactive changes designed to address symptoms that tend to destabilise the system. The stakeholders in a child protection system must seek to know the level of risk assurance currently applied, and determine the level required and an acceptable rate of failed alarms to the community.

The stakeholders must account for all of the trade-offs of level of resource and cost, and benefits for true positives vs. the disutility for false positives, identifying the optima in each case. Finally, the stakeholders must develop the capability to target towards specified level of risk assurance and defend that target against criticism that it is to high or too low.

---

References

Australian Institute of Health and Welfare (AIHW) (2006) Child Protection Australia 2004—05, AIHW Cat. No. CWS26, AIHW Child Welfare Series, No 38, Canberra.

Benbenishty, R. and W. Chen (2003) “Decision making by the child protection team of a medical centre” Health and Social Work, 28(4):284–292.

Brown, M. (2000) Care and Protection is about Adult Behaviour: The Ministerial Review of the Department of Child, Youth and Family Services, Report to the Minister of Social Services and Employment, Hon Steve Maharey, Wellington.

CYF (2000a) Child, Youth and Family Submission to the Government Reviews of Referrals and Notifications and Placement Services, Main Report, Department of Child, Youth and Family Services, Wellington.

CYF (2000b) Child, Youth and Family Submission to the Government Reviews of Referrals and Notifications and Placement Services: Appendix 1: Response to Notifications, Department of Child, Youth and Family Services, Wellington.

CYF (2003a) Chief Social Worker’s Report into the Involvement of the Department of Child, Youth and Family Services with Olympia Jetson and Saliel Aplin and Response by Department, Department of Child, Youth and Family Services, Wellington.

CYF (2003b) Review of Involvement of the Department of Child, Youth and Family Services with Kelly Gush and Child, Youth and Family’s Response to the Report, Department of Child, Youth and Family Services, Wellington.

Doolan, M. (2005) “Child death by homicide: An examination of child protection actions 1996–2000” Te Awatea Review, 3(July):4–6.

Fielding, K. and L. Taylor (2001) “Assessing risk to children who live with family violence” Social Work Now, 18:39–44.

Gambrill, E.D. (2005) “Decision making in child welfare: Errors and their context” Children and Youth Services Review, 27(4):347–352.

Gambrill, E. and A. Shlonsky (2000) “Risk assessment in context” Children and Youth Services Review, 22(11–12):813–837.

Lytollis, S. (1996) “Making the best of the media spotlight” Social Work Now, 5(December):15–20.
McLoughlin, D. (1994) “Killing our kids” North and South, (April):42–53.

Ministry of Social Development (2005) The Social Report 2005, Ministry of Social Development, Wellington.

Ministry of Social Development, Child, Youth and Family and Treasury (2003) Report of the Department of Child, Youth and Family Services First Principles Baseline Review, No. 5, Ministry of Social Development, Wellington.

Munro, E. (1999) “Common errors of reasoning in child protection work” Child Abuse and Neglect, 23(8):745–758.

Munro, E.M. (1999) “Protecting children in an anxious society” Health, Risk and Society, 1(1):117–127.

Scott, D. (2006) “Sowing the seeds of innovation in child protection” keynote address to the 10th Australasian Conference on Child Abuse and Neglect, Wellington, 14–16 February.

Smith, C. (1998) “Beyond the tip of the iceberg” Social Work Now, 11(December):33–40.

Weeks, A. (1994) A Study of Financial Management Practices in the Children and Young Persons Service in Fiscal 1994, Department of Social Welfare, Wellington.

---

Footnotes

¹ Acknowledgement
Some of the material in this paper was presented at the 10th Australasian Conference on Child Abuse and Neglect, Wellington, 14–16 February 2006.

² New Zealand’s statutory child protection agency.

³ New Zealand is not alone in experiencing surging demand. Personal communications from researchers from Australia and the United Kingdom suggest that demand for child protection response has surged in these countries as well.

⁴ Notifiers are persons from the public, government agencies or NGOs who notice concerns and forward these to the child protection agency. Notification is voluntary in New Zealand; there is no mandatory reporting.

⁵ Note that decision errors should be distinguished from decision mistakes. Mistakes are misapplications of the risk-screening process. (This may result in a good or bad outcome – some mistakes are beneficial.) Errors can arise even when the best decision methods are applied rigorously. There is no guarantee, when making decisions in an uncertain environment, that you will achieve the best outcome – you may have made the correct decision given the information but still have made an error.

⁶ This includes an internal and external document search, the child abuse literature, internal reports and their recommendations, and media stories.

⁷ Note that this is not to say that there have not also been more general societal changes influencing tolerance for abuse. Dr Marie Connelly is currently examining this issue. The focus on media attention is purely pragmatic, from a research perspective, to provide measurable pressures.

⁸ Intake demand is measured by “client notifications”. This refers to all notifications per client. So, for example, a single notification for three siblings equals three client notifications. If one of these siblings gets renotified, then this would be a fourth notification.

⁹ Level of media attention was assessed by a research assistant blind to the notification rates. “High” levels were recorded where there were at least two front page headlines on child abuse and/or Child, Youth and Family in at least two of New Zealand’s three national daily newspapers or more than 4 articles in national and regional papers.

¹⁰ This assumes a linear function between notification rates and notification size.

¹¹ Latent demand includes both cases of a similar severity and cases that might now meet a lower threshold (softer concerns).

---