Youth on Track randomised controlled trial: process evaluation

Lily Trimboli, NSW Bureau of Crime Statistics and Research, August 2019.[1]

Aims: To determine whether the Youth on Track randomised controlled trial is being implemented as intended and whether there are any unexpected consequences of the trial.

Methods: Semi-structured interviews were conducted with 52 stakeholders and available administrative program data were analysed.

Results: While both the processes of referral into, and engagement with, the scheme were perceived to be operating well, some problems were identified with each process, for example, difficulties in contacting or locating the young person. It was noted that participants of the “control group” (Fast Track) are primarily given referrals to, or linkages with, other service providers while young people allocated to the more comprehensive and longer-term Youth on Track intervention receive a variety of support. Stakeholders identified challenges in implementing each intervention. While some stakeholders stated that there were no challenges in referring young people in either intervention to external service providers, others noted a lack of services in the local area, as well as the effects of the service providers’ risk assessments and their waiting lists. Two recurring themes in the stakeholder interviews were the perceived negative ramifications of Fast Track’s shorter timeframe on re-offending risk and concern about the random allocation of young people to an intervention with no consideration of their background, circumstances or needs.

Conclusion: Stakeholders believed that each intervention was being implemented as intended. However, the delivery of the two different interventions and the evaluation and randomisation processes had produced some unexpected consequences.

Keywords: youth, randomised controlled trial, process evaluation, interviews

[5-0210] Introduction

Youth on Track scheme

Youth on Track is an early intervention scheme. It targets young people in NSW aged between 10 and 17 years who are not yet entrenched in the criminal justice system but who have been assessed as having a medium to high likelihood of re-offending. Youth on Track is not a diversionary program; instead, it operates after formal contact with the criminal justice system. The scheme is based on the premise that young people can be deterred from long-term involvement in the criminal justice system by addressing their multiple and complex offending-related needs such as pro-criminal thinking, negative peer associations, mental health issues, substance abuse, disrupted or dysfunctional family life, poor employment prospects, disabilities and disengagement from education.

Therefore, Youth on Track provides targeted, individualised interventions to address the underlying causes of the young person’s involvement in crime.

The three key objectives of Youth on Track are:

1. 

to identify, in a timely way, young people at high risk of continuing in the criminal justice system

2. 

to provide one-on-one case management and evidence-informed interventions targeted to address the individual criminogenic risk factors of the young person, and

3. 

to provide an evidence-informed family intervention to support the family of young offenders to reduce the young person’s contact with police.[2]

Youth on Track was endorsed by NSW Cabinet in late 2012 and commenced operation on 1 July 2013 in three NSW Police Local Area Commands (LACs) — Blacktown, the Mid-North Coast and Newcastle City. The scheme has gradually expanded over the last few years. The Newcastle site grew to include Lake Macquarie LAC in April 2014 and Port Stephens LAC on 2 February 2015. In addition, on 2 February 2015, the Mid-North Coast site expanded to include Manning Great Lakes LAC and the Blacktown site incorporated the two LACs of Mount Druitt and Quakers Hill. On 17 October 2016, Mr David Elliott, the then NSW Minister for Corrections, announced that, in December 2016, the scheme would receive an injection of $14.5 million over three years and be expanded to three new sites — Central West (Orana and Canobolas LACs), Coffs Harbour (Coffs Clarence LAC) and New England (Oxley and New England LACs). These six sites are currently funded until June 2020 with further funding being sought to June 2022, to allow time for completion of the randomised controlled trial and a report back to NSW Cabinet to consider the results of the evaluation and possible state-wide expansion of the scheme.

Following a competitive tender process, the NSW Department of Communities and Justice contracted three non-government organisations[3] to engage eligible young people and their families in the scheme and to co-ordinate casework and intensive evidence-informed interventions tailored to the young person’s specific offending-related needs.

How Youth on Track operates

Figure 1 broadly illustrates how Youth on Track operates. As Figure 1 shows, the scheme consists of six key elements — referral and screening, engagement, assessment, case management, intervention, review and exit planning. There are two referral pathways into Youth on Track. The first pathway relates to discretionary referrals. These are made principally by NSW Police Youth Liaison Officers (YLOs) and also by local primary and secondary schools in the Youth on Track sites. Discretionary referrals apply to young people who have received at least one formal police contact (ie a caution, a Youth Justice Conference or a charge) and who have several risk factors. Some of the relevant offending-related risk factors include truancy, child-at-risk reports, substance abuse, mental health issues, association with peers involved with police, family history of domestic violence, and lower than normal cognitive and academic ability.

The second referral pathway into the scheme relates to compulsory automatic referrals. These are made by the Youth on Track Screening Officer using the NSW Police Force’s Computerised Operational Policing System (COPS) database. On a daily basis, the Screening Officer identifies relevant young people who have had a police contact within the previous 24 hours in one of the scheme’s pilot sites. Automatic referrals apply to young people who have had at least two formal police contacts and who have a 60% or greater chance of re-offending within 24 months; the latter is measured using an actuarial screening tool developed by the NSW Bureau of Crime Statistics and Research (BOCSAR).[4] To be deemed suitable for either a discretionary or an automatic referral into Youth on Track, the young person must be aged between 10 and 17 years, offend or attend school within one of the Youth on Track sites and have never previously been supervised by Youth Justice NSW.

If the Screening Officer determines that a young person is eligible for the scheme, the young person is referred to the provider who then assesses suitability. A number of factors make a young person unsuitable, including if he/she lives outside the service area of the Youth on Track site. A young person is also deemed to be unsuitable for the scheme if, after completing a risk assessment of the young person (and/or his/her family), the provider determines that it is unsafe to work with them. The young person’s existing case manager could decline the young person’s participation in the scheme if the case manager considers it is not in the best interests of the young person. This would occur in cases where the young person already has a number of agencies involved in his/her life.

Within three days of the young person being referred to Youth on Track, the provider begins the process of contact with both the young person and his/her family to offer a service. The provider liaises with NSW Police, local schools, community groups and other stakeholders in order to locate the young person and engage him/her and his/her family in the scheme. The provider must obtain written consent from both the young person and his/her family to participate in the scheme. However, the scheme is ultimately voluntary and a young person may opt out at any time.

To ensure continuity of service, each young person is allocated a dedicated and trained case worker. The key tasks of the Youth on Track case workers are to engage the young person and his/her family; develop an individual case plan based upon the young person’s assessed criminogenic needs; deliver, refer or broker appropriate programs or services to address these assessed needs and to increase pro-social behaviour; regularly monitor the young person’s progress towards meeting the goals of the case plan; and conduct an exit planning process to facilitate the young person’s access to ongoing community supports outside Youth on Track.

When a young person consents to participate in Youth on Track, the case worker must conduct two assessments, using validated tools. Each of these assessments should be conducted within four weeks of the young person consenting to participate in the scheme. The results of each assessment are used to develop the young person’s case plan. The first assessment is to screen for a cognitive disability. Providers typically use the Child and Adolescent Intellectual Disability Screening Questionnaire (CAIDS-Q). This tool does not assess whether the young person has a cognitive disability, but rather it is used to indicate whether the young person should be referred to an appropriate clinician for further assessment. The second assessment is conducted using the Youth Level of Service Case Management Inventory — Australian Adaptation (YLS/ CMI-AA).[5] The YLS/CMI-AA is a structured risk/needs assessment and case management tool, incorporating items that represent static and dynamic risk factors. It is designed to guide the level and types of interventions so that case planning activities can be focused on the appropriate areas of need. The instrument gives some direction for the three basic principles of effective case management that form the foundation of Youth on Track’s casework and interventions, namely, risk, need and responsivity.[6] Completion of the YLS/CMI-AA involves gathering information from various sources and conducting multiple interviews with the young person and his/her family.

The YLS/CMI-AA consists of 47 items organised into eight criminogenic domains — prior and current offending, family and living circumstances, education/employment, peer relations, substance abuse, leisure/recreation, personality/behaviour and attitudes/beliefs. A score is calculated for each domain and an overall score that combines the eight domains can also be calculated. The overall score assists in deciding on the level of service to be delivered, with more frequent and intensive intervention being provided to young people who are at higher risk. For example, a young person assessed at high risk/needs receives a minimum of six hours of support per week and could remain in the scheme for up to 12 months while a young person assessed at low risk/needs receives a minimum of two hours per fortnight and may remain in the scheme for only three months and a young person assessed at medium risk/needs receives a minimum of two hours per week and could remain in the scheme for up to six months. The YLS/CMI-AA also allows the case worker to assess the young person’s existing strengths, such as his/her skills, family and community relationships.

The YLS/CMI-AA assessment is reviewed 12 and 24 weeks after the first assessment and again when the young person exits from the scheme. The case worker also collects social outcome information from the young person on both entry to, and exit from, the scheme. This is information about the young person’s accommodation (eg living in a home with parents or relatives, in independent living, in a rehabilitation service, in foster or kinship care or is homeless), participation in employment (eg in a traineeship or apprenticeship, employed fulltime/part-time, self-employed, in work experience or not employed), participation in education or training (eg attending school, a back-to-school program, a special education program, TAFE, university or a parenting program), and participation in community activities (eg playing sport, participating in volunteer work or programs such as the Rural Fire Service, attending a youth or family centre, and involvement in community groups).

Young people who are referred to Youth on Track may have a range of complex needs that require interventions. With the exception of young people who have low risk levels, case workers should deliver behavioural and family intervention programs to young people in the scheme. Behavioural intervention programs focus on addressing anti-social/pro-criminal thinking and behaviour and are typically based on cognitive behavioural techniques. A standard cognitive behavioural tool employed by Youth on Track case workers is Changing Habits and Reaching Targets.[7] CHART is an offence focused intervention designed specifically for young offenders. It challenges offending behaviour by helping young people to understand the values and beliefs that underpin their offending behaviour, to re-examine their motivation, to re-evaluate the potential consequences of their actions, and to develop problem-solving and consequential thinking skills.

Family intervention programs are designed to change dysfunctional family patterns; improve parenting skills; promote improved relationships with teachers and positive peers; and help parents and children to communicate more effectively and safely, solve problems collaboratively and resolve conflict. These programs should include both the young person and the significant people in his/her life, such as parents and siblings. They typically involve a number of steps — role clarification, identification of the issues or problems and strengths, decisions about what to work on first and what the family wants to achieve, exploration of the issues in greater detail and the strategies to achieve the goals. One family intervention model available for case workers to employ with young people in Youth on Track is “Collaborative Family Work”.[8] This model consists of between six and ten sessions, each lasting between 30 minutes and an hour. It deals with general family dynamics and relationship focused tasks and agreed actions. The sessions are conducted either in the family home or in a neutral area where the family feels comfortable; they generally involve two workers who share the facilitation and positive role modelling for problem solving and collaboration.

Education engagement programs are another type of program to which young people could be referred by Youth on Track case workers. These programs are designed to support young people to continue, or to reengage, with the mainstream school environment. Their objective is to address behavioural, emotional or cognitive issues that may affect the young person’s educational participation. Case workers could also refer a young person into education programs conducted by other non-government organisations or training providers (such as TAFE or community colleges) or to alternative schools in the catchment area. In addition, depending on the needs of the young people, case workers could refer or broker interventions, including accommodation (eg referrals to refuges, specialist homelessness service providers, accommodation support or living skills development), training and employment programs (eg assistance in the preparation of resumes or job applications, role modelling a job interview, referrals to job network agencies that could provide links to employers, training or job-ready programs), health (eg appointments with general practitioners, family planning, sexual health and protection, information and advocacy), drug and alcohol services (eg organising appointments for counselling or support, referrals for detoxification or rehabilitation), mental health services (eg arranging mental health plans, referrals to psychologists or community mental health, support and advocacy), anger management programs (eg RAGE),[9] financial services (eg money management, financial advice), legal services (eg referring a young person and supporting him/her to seek legal advice), recreation (eg linking a young person to an activity conducted by the local Police Citizens Youth Club, sports or recreation clubs) and cultural support programs.

During their interactions with the young people and their families, case workers employ motivational interviewing techniques in order to understand the young person’s perspective, minimise any resistance, and elicit his/her motivation for change. In addition, case workers use core effective practice skills. These include having open and honest discussions to clarify the respective roles of the case worker and the young person, as well as clarifying timeframes for both parties, problem-solving modelling and encouraging pro-social behaviour and values.

As Figure 1 shows, the final element of the Youth on Track scheme is review and exit planning. The case worker reviews the outcomes against the goals identified in the young person’s case plan and develops an exit plan. This occurs with the participation of both the young person and his/her family. The exit plan focuses on strategies for the young person and his/her family to continue to improve outcomes and to reduce the young person’s likelihood of re-offending.

The scheme is supported by a two-tiered governance structure. This consists of a multi-agency Implementation Committee that is responsible for high-level decision making and Regional Governance Committees in each site to deal with local implementation issues.

Previous evaluations

Two external evaluations of the Youth on Track scheme have previously been undertaken.[10] BOCSAR’s research findings were based on an analysis of two relevant databases and the telephone interview responses of 22 young people, 24 caregivers and 38 stakeholders. Both the young people and their caregivers gave overwhelmingly positive feedback regarding both the scheme and their case workers. The young people interviewed were highly motivated to participate in the scheme and perceived that their case plans did not miss any issues. However, most were not clear about some of the elements of their case plan. The vast majority of the stakeholders who were interviewed believed that the scheme is beneficial, innovative and has the potential to enhance the lives of the young people and their families. However, they suggested improvements in the operation of the scheme, most of which should have occurred prior to the scheme’s implementation, for example, negotiating for the establishment of relevant local services; and consulting with, engaging and promoting the scheme among local communities and key agencies.

CIRCA’s evaluation focused on the impact of Youth on Track on social outcomes for young people and their families. It comprised interviews with 18 young people, 18 family members/carers, 10 Youth on Track staff and 15 stakeholders. CIRCA also analysed both the YLS/ CMI-AA data and the satisfaction surveys (conducted by Youth on Track staff) for 44 scheme participants (that is, 22 young people, 16 family members/carers and 6 involving both young people and their family members/ carers). CIRCA found improvements over time in the total YLS/CMI-AA assessment scores and in a number of specific criminogenic domains — education/employment, leisure/recreation and peer relations.[11] However, CIRCA found no improvements in other domains, for example, personality/behaviour, attitudes/beliefs, alcohol and other drug use, and family and living circumstances. As CIRCA noted, their evaluation did not include a comparison group. Consequently, it cannot be determined whether the improvements observed in the various criminogenic domains are due to participation in the Youth on Track scheme or to some other factor(s).

Neither the CIRCA nor the initial BOCSAR evaluation was designed to examine the scheme’s central objective of reducing rates of re-offending by young people. To fill this gap, BOCSAR is currently undertaking a randomised controlled trial to determine whether the scheme achieves this central objective. This methodology compares two groups of people who have been randomly assigned to two different treatments; one group receives the treatment being evaluated (in this case, Youth on Track) and the other group receives a “control” treatment. Since random assignment minimises group allocation bias, this methodology is considered to be the most reliable and robust for evaluating the effectiveness of an intervention.

Youth on Track randomised controlled trial

The key objective of the randomised controlled trial is to determine if young people in the Youth on Track scheme have reduced re-offending rates after participating in the scheme compared to young people in the control group. The control group consists of young people who are eligible for the Youth on Track scheme but who have not been randomly balloted to receive Youth on Track services. In lieu of Youth on Track, the control group receives a shorter, less intensive intervention, referred to as “Fast Track”. Fast Track was developed specifically for the purpose of this randomised controlled trial and was designed to provide only minimal support and intervention for the young person. The randomisation of the young people into the two groups began on 9 August 2017 and will continue until approximately 350 young people have been allocated to each group. Participation in the evaluation is voluntary and young people can withdraw from it at any time.[12] If a young person does not consent to participate in the evaluation but consents to participate in the Youth on Track scheme, he/she is still randomly assigned to either Youth on Track or Fast Track, but his/her information is not used in the evaluation. This was done to avoid any potential for the randomisation process to be manipulated by either the young person or the case worker. The primary measure of re-offending is time (in days) to the first formal contact with the police (ie a caution, a Youth Justice Conference or a charge) after consenting to participate in the evaluation. The minimum follow-up period for young people in each group is 12 months from their date of consent to participate. Changes over time in a number of social outcomes (accommodation status, engagement in education, employment and community activities) will also be compared across the two groups of study participants.

The case workers who deliver the Youth on Track intervention also deliver the Fast Track intervention. This is done to maintain consistency and ensure that the staff involved in each intervention have had the same training in motivational interactions, core effective practice skills and working with young offenders. For Fast Track participants, the case worker completes a shortened version of the YLS/ CMI criminogenic assessment, namely, the Youth Level of Service/Case Management Inventory: Screening Version (YLS/CMI:SV).[13] Following this assessment, the case worker, together with the young person, formulates a plan of action. This includes goals for the young person and the services needed to address the higher risk criminogenic domains. The case worker facilitates referrals to appropriate external services and programs over four face-to-face case management interactions with the young person and/or his/her family and one additional case conference if required; these interactions occur within a period of six weeks. Case management responsibilities for Fast Track participants include making telephone contact with referral agencies, reminder telephone calls to the young person and/or his/her family, and completing referral forms. Youth on Track staff do not provide Fast Track participants with any direct offence-based behavioural or family interventions. Limited brokerage funds are also available for Fast Track participants.

Figure 2 outlines the steps involved in the random ballot process, summarising a young person’s progress through either Youth on Track or Fast Track, and highlighting the differences between the two interventions.

This randomised controlled trial has received ethics approval.[14] In addition, prior to the commencement of the study, staff from BOCSAR and Youth Justice NSW conducted 18 face-to-face consultations with relevant stakeholders in each of the six sites. These consultations, which occurred over a four-week period in April 2017, involved Youth on Track case workers and managers, Aboriginal community members and service providers. During these consultations, stakeholders were informed of the advantages and disadvantages of a randomised controlled trial, the proposed experimental design and the elements of the shorter Fast Track intervention. Perceived risks identified by stakeholders were discussed, and where possible, strategies were developed to mitigate these risks. This included allowing case workers to set up case conferences to ensure continuity of care of Fast Track clients when they exit the intervention, informing the Children’s Court of the evaluation and establishing an Evaluation Advisory Committee to discuss and address any issues that arise during the evaluation.

Furthermore, since this is the first randomised controlled trial of a youth justice intervention in NSW, BOCSAR simultaneously undertook a process evaluation to complement the outcome evaluation. The key objectives of the process evaluation were to determine whether the randomised controlled trial is being implemented as intended and whether there are any unanticipated consequences of the trial.

Research Aims

In consultation with the Youth on Track Unit, NSW Department of Communities and Justice, BOCSAR’s process evaluation was structured around the following questions:

1. 

Are the referral and engagement processes into the scheme operating efficiently/effectively?

2. 

What types/levels of support are Fast Track participants receiving? How does this differ from the support provided to Youth on Track participants?

3. 

Are there any major challenges in implementing Fast Track and Youth on Track?

4. 

Are there any challenges in referring Fast Track and/or Youth on Track participants to external services?

5. 

Is each intervention being delivered as intended?

6. 

Is the delivery of Fast Track/Youth on Track producing any unexpected consequences (either positive or negative)?

7. 

Is the evaluation (and the randomisation process) producing any unexpected consequences (either positive or negative)?

Method

To address these issues, semi-structured interviews were conducted with 52 stakeholders across the six sites. The stakeholders comprised the Youth on Track staff (managers and case workers; n=27); staff associated with each of the two discretionary referral sources, namely, Youth Liaison Officers and school-related staff (n=13); and representatives of various service providers to which the young people were referred by the Youth on Track staff (n=12). These interviews were conducted in two phases. The first phase occurred in November 2017. During these interviews (n=21), it became clear that many Youth on Track staff had had limited direct experience with Fast Track participants and were therefore not in a position to outline any associated issues. As a result, interviews were suspended until mid-2018 when the second phase of interviews occurred (n=31). However, of the 52 stakeholders interviewed over the two phases, 21 (40.4%) had no or limited experience with, or knowledge of, the Fast Track intervention. Youth Liaison Officers, school-related staff and service providers comprised about 86% of these stakeholders.

The stakeholder interviews were supplemented with an analysis of available administrative program data extracted from the Youth on Track database.

Results

Characteristics of Youth on Track participants

Of the 708 young people who consented to participate in the Youth on Track scheme between 9 August 2017 and 24 July 2019, 145 (20.5%) were ineligible for the evaluation. The vast majority (525, 93.2%) of the remaining 563 young people consented to participate in the evaluation. Table 1 shows the characteristics of these 525 young people who consented to participate in both the Youth on Track scheme and the evaluation.

As Table 1 shows, between 9 August 2017 and 24 July 2019, the New England site accounted for the lowest proportion of young people who consented to both receive the Youth on Track service and to participate in the evaluation (14.1%) while the Blacktown site accounted for the highest proportion (20.6%). In the first 24 months of the randomised controlled trial, 45.7% of the young people who consented to participate in the scheme were randomly allocated to the shorter Fast Track intervention. However, the rate differed by site, ranging from 33.8% in the Coffs/Clarence site to 56.0% in the Hunter. It is expected that there will be a fairly even distribution of young people randomly assigned to each of the two interventions across the six sites once a sufficient sample size is reached.

Table 1 shows that, across the six sites, the majority of participants were male (75.0%) and aged between 14 and 17 years (71.0%) with an average age of 14.53 years. While 44.8 % of the total number of young people who consented to the scheme were recorded as being Aboriginal or Torres Strait Islander, this varied across the sites and, for one in five young people, their indigenous status was not known/recorded. In three sites, at least half of the participants were Aboriginal or Torres Strait Islander — Central West (63.6%), Mid North Coast (53.9%) and New England (62.2%).

About three in five (59.2%) of all young people had been referred to the scheme via the automatic referral pathway of COPS, and a further 32.0 % were referred by YLOs. Less than 9% of young people were referred by staff of the Department of Education. Three in ten (30.3%) young people had no police cautions prior to their involvement with the Youth on Track scheme. The remaining 69.7% of participants had either one (29.7%) or at least two prior cautions (40.0%), with the average being 1.39 prior cautions per young person. By contrast, the majority (62.7%) of participants had no prior charges, with the average being 0.98 prior charges.

Table 2 shows, for each intervention type, the number of young people referred to different types of services provided by external agencies.

As Table 2 shows, young people in each intervention were referred to a variety of external services by case workers. Consistent with the operating rules of the two interventions, compared to young people in the longer Youth on Track intervention, no young person in Fast Track received family interventions (13.1% vs 0%, p < .001) and much fewer young people in Fast Track received behavioural interventions (66.0% vs 0.9%, p < .001). Although significantly more young people in the longer intervention were referred to drug and alcohol services compared to Fast Track (16.5% vs 9.4%, p = .027), similar proportions of young people in each intervention were referred to a number of services, including education (about 30%), financial (about 6%), health (about 7%), employment services (13.6% of Youth on Track participants vs 16.1% of Fast Track participants) and family support (15.5% vs 18.3%, respectively). Table 2 also shows that young people in Youth on Track were referred to a greater number of different services than those in Fast Track.

Stakeholder interviews

Are the referral and engagement processes into the scheme operating efficiently/effectively?

About 35% of the 37 relevant stakeholders (Youth on Track Managers and case workers, and YLOs) stated that both the referral and engagement processes into the scheme were operating well. Stakeholders also noted that, with discretionary referrals, young people have some understanding of the scheme, the role of the case worker and the purpose of the case workers’ call because YLOs and school staff provide them with this information. It is, therefore, easier for case workers to contact these young people. Some stakeholders noted that these young people are more willing to participate in the scheme. Discussions with the YLOs or school teachers also give the case workers an insight into the young person’s needs, background, family history and perhaps suggestions for priority areas for the young person. Conversely, for young people referred to the scheme via the automatic referral pathway, case workers have no background information about them or their needs prior to the first phone contact. Some stakeholders noted that it takes more effort from the case workers to explain the scheme and the benefits of participating to young people referred via the automatic referral pathway.

Despite the perception of some stakeholders that the referral and engagement processes were operating well, a number of problems were identified with each process. One in four of the 31 comments made regarding the discretionary police and school referrals related to a lack of appropriate staff, for example, “referrals come to a standstill when the NSW Police Screening Officer is on leave”, the lack of a consistent YLO in a specific Police District and the long recruitment process to replace a YLO. A further three in five comments dealt with school-related issues, namely, that referrals from schools often do not meet the scheme’s eligibility criteria (29% of comments), only some schools or teachers refer young people to the scheme (12.9%), schools make few referrals (9.7%) and the school term structure affects the flow of referrals (6.4%).

Stakeholders also noted problems with the scheme’s engagement process. More than half of the 48 comments related to difficulties in contacting or locating the young person (25%, eg some young people and families are transient, phones are disconnected or are not answered, phones are lost or stolen, a young person living in a remote bush location may have neither a phone nor mobile phone access), information regarding the young person being incorrect (14.6%, eg phone numbers change, addresses do not correspond to the young person) and issues associated with cold-calling the young person (14.6%). About 8% of comments noted that the engagement of young people is difficult because the program is voluntary.

Suggested improvements to referral and engagement processes

Five (13.5%) of the 37 stakeholders stated that no improvements were required to either the referral or the engagement processes into the scheme. About 71% of the 28 comments made regarding improvements to either the school or the police referral process suggested increased engagement and communication between Youth on Track staff and the referral sources. Examples of these suggestions included that the Youth on Track Unit conduct workshops with school staff who make referrals to ensure that referrals are successful rather than being screened out, case workers undertake further “road-shows” to speak with school principals and support staff who could potentially make referrals, case workers liaise with schools to find where a young person is living, case workers attend school meetings and provide updates, and case workers improve their relationships with some YLOs and conduct joint visits in order to engage young people. About one in ten comments related to improving the accuracy of the information about the young people referred.

Of the 34 comments made regarding general improvements to the referral and engagement processes into the scheme, one in five was to modify the scheme’s eligibility criteria, for example, allowing referrals for children with “at risk” notifications and for school students displaying violent behaviours even if the police have not been involved. Another comment was to decrease the threshold for automatic referrals from COPS.[15] An associated suggestion, accounting for 15% of the comments, was to increase the referral pathways to allow referrals from other agencies or sources. One in four comments dealt with increasing communication and community awareness of the scheme. The strategies suggested to achieve this included holding round-table conferences to identify other services already involved with the young person in order to avoid duplication and to determine gaps in service, having telephone referrals to allow for discussion of any concerns regarding the young person or to discuss the best fit for that young person, the Youth on Track team becoming better known in the local community and breaking down barriers through community awareness, increasing promotion of the scheme within the community, and the local Youth on Track team educating agencies about the scheme’s eligibility criteria. About 12% of the comments suggested modifications to the referral form to incorporate more information about the young person, for example, his/her cultural identity, drug and alcohol issues, mental health issues, domestic violence background, family environment, trauma, possible weapons, whether there are any dogs at the residence and who else may be in the house. It was suggested that some of this information could help to assess the case workers’ safety when visiting the young person’s house.

What types/levels of support are Fast Track participants receiving? How does this differ from the support provided to Youth on Track participants?

Stakeholders commonly reported that Fast Track participants were referred to, or linked with, external service providers in the local areas (34% of 195 comments) and engaged in activities provided by external agencies/programs (10.3%). These included housing, local mental health services, drug and alcohol services, education, youth employment programs, legal services support and family support. Stakeholders also reported that Fast Track participants were engaged in goal-setting activities and developing action plans (8.7%). In addition to providing case work or support (6.1%), case workers working with Fast Track participants provide education-related support (4.6%, eg accompanying a young person to a meeting about returning to school following suspension or assisting a young person to enrol in a TAFE course) and court-related support (4.1%, eg informing a young person about a typical day in court, providing information about navigating the criminal justice system).

The main difference between the types of support or interventions that case workers and agencies provide to Youth on Track and Fast Track participants is that only Youth on Track participants receive CHART and the family collaborative work interventions. By contrast, for the most part, Fast Track participants are referred out to, or linked with, external service providers in the local area. One concern noted by a stakeholder was that “case workers feel as though young people [in Fast Track] are being palmed off”.

Are there any major challenges in implementing Fast Track and Youth on Track?
Challenges in implementing Youth on Track

Many of the 141 comments made by 48 of the 52 stakeholders regarding the challenges faced when implementing the longer Youth on Track intervention related to engagement (17%) and the young people’s background and characteristics (12.1%). Examples of the challenges in engaging the young person included not only securing the young person’s initial engagement, but maintaining that engagement and motivation over time. It was also observed that families of some young people are not prepared to engage, or participate on a regular basis, in the structured family collaborative work. While families are prepared for case workers to work with their young person, they baulk at the implication that their parenting is being questioned, distancing themselves from any responsibility for their child’s behaviour. Some case workers remarked that they undertake family work informally rather than in the structured manner required by the intervention. Examples of the challenges associated with the young person’s background and characteristics included being entrenched in the criminal justice system; having multiple or complex issues such as long-term drug or alcohol or mental health concerns; having families who are not supportive, provide minimal parental guidance or are negative role models; some young people believing that they had no problems or not believing that their attitudes, beliefs or behaviours contribute to their offending; the difficulties of working with homeless young people or young people when they leave out-of-home care; and some young people not being willing to accept the requirements of the intervention. It was noted that the younger age group of 10–13 years have additional challenges associated with their greater susceptibility to being influenced by their peers and the limited services/ activities available for their age group.

Other challenges noted by stakeholders in implementing the Youth on Track intervention included the restrictions associated with the intervention’s requirements, parameters or resources (9%, eg referral pathways into the scheme being too narrow, the inability to deliver some anger management programs and living skills programs that some stakeholders believe to be more suitable, not having the scope to run programs in schools where further connections could be established); the voluntary nature of the scheme (7.1%); locating the young person due to incorrect telephone numbers or addresses or the lack of a fixed address (7.1%); the skills, experience and quality of the Youth on Track staff required for effective service delivery (7.1%); family-related issues (6.4%, eg the lack of involvement or support from some parents/carers and the parents’ own problems with, for example, addictions); a lack of services or referral sources (5.7%); community understanding of the scheme’s aims and services (5%); and the impact of their past negative service history on young people and their families (4.2%).

Challenges in implementing Fast Track

Some of these same challenges were also reported in relation to the implementation of Fast Track. For example, of the 198 comments made by 41 stakeholders, reference was made to the difficulties associated with engagement (7.1%), building rapport (6.1%), a lack of services or capacity in the area including lengthy waiting lists (7.6%) and challenges linked to the characteristics of the young person (4%, eg some young people have difficulty trusting people). However, about 31% of the comments noted specific challenges when implementing Fast Track. These included the ramifications of the intervention’s short timeframe, for example, being unable to include everything in the six-week time period, goals requiring longer than the allocated time to be completed, the case worker not being able to assist the young person through their legal process because their court matter is not finalised within the timeframe, and the difficulties of completing a handover to another organisation within the short timeframe. These concerns accounted for 24.2% of the stakeholders’ comments. An additional time-related challenge for the case workers is the need to maximise each contact with the young person. It was observed that the case workers must quickly locate the young person, obtain his/her early consent to participate in the scheme, build rapport and a relationship, engage him/her, immediately identify and prioritise his/her criminogenic needs, transfer those needs into a functional case plan and address them while simultaneously communicating to the young person both the timeframe of the intervention and the need for haste. To utilise the time available in the best way possible, it was noted that case workers need to be organised, structured, highly goal orientated and goal-specific from the outset.

Just over 7% of comments referred to the Fast Track’s restricted parameters, for example, case workers not being able to offer the young people any behavioural interventions or family interventions; not being able to help directly with the young persons’ issues; not being able to get to know the young person deeply; and the case plan for Fast Track clients being a “to-do” list rather than a working document as it is for clients of the Youth on Track intervention.

Challenges in implementing Youth on Track and Fast Track concurrently

Stakeholders were also asked whether there are major challenges in the implementation of both Youth on Track and Fast Track concurrently. Nine (17.3%) stakeholders stated that there were no challenges in implementing the two interventions simultaneously. Of the 34 stakeholders who indicated that there were challenges, about one in ten (9.6%) of 114 comments dealt with the difficulties for case workers, for example, case workers struggle to explain to young people and their families that they could be allocated to either a six-week or a 12-week intervention, it is stressful for case workers to work with both interventions and it is difficult for case workers to transition from working with a client in Youth on Track to a client in Fast Track. An associated challenge was time management for case workers, this accounted for 5.3% of the comments made.

About 12% of comments dealt with the issue of confusion, both within the community and amongst the young people and their families. Comments included that people do not understand the difference between Youth on Track and Fast Track, people assume that case workers only deliver one intervention rather than both, people living in the same street whose young people are involved in the scheme may have a different understanding of the scheme because some young people are in the longer intervention and others are in the shorter intervention.

Other challenges of simultaneously implementing both Youth on Track and Fast Track included the difficulties for Youth on Track managers (10.5%, eg ensuring and monitoring intervention fidelity and the fair allocation of caseloads to case workers); case workers giving priority to clients in the Fast Track intervention because of the limited time available whereas they have more time and freedom to interact with clients in Youth on Track over three to 12 months and incorporating more into the shorter timeframe (10.5%); the challenges associated with the randomisation process (6.1%, eg making caseloads unpredictable, neither case workers nor young people knowing to which intervention they will be assigned); disappointment and jealousy that other young people are receiving a different intervention (3.5%); whether the intervention is suitable for the young person (3.5%, eg a young person in Fast Track may have as many needs as a young person in the Youth on Track intervention and could benefit from further contact with the scheme); and the difficulties of explaining the two interventions to the young people, their families, the community and referral sources (2.6%).

Are there any challenges in referring Fast Track and/or Youth on Track clients to external services?

Of the 27 Youth on Track staff who were interviewed, 55.5% stated that there were no challenges in referring Youth on Track participants to external service providers, and 40.7% made the same comment regarding Fast Track participants. About 15% of stakeholders noted that the challenges in referring Fast Track participants were the same as those for referring Youth on Track participants. However, a lack of services in the local area was commonly noted, accounting for about one in five of the comments made in relation to referring Fast Track participants (n=48 comments) and Youth on Track participants (n=46 comments). Some of the services that stakeholders reported to be lacking for young people in both interventions include mental health services and programs, drug and alcohol services, alternative education options for young people who are not comfortable in the traditional school setting, transport options and accommodation including emergency and short-term housing. Local services were considered to be particularly critical to the success of Fast Track because, as one stakeholder noted, it is “a small, sharp program over six weeks”. Reference was also made to the large distances to be travelled to reach services and the inflexible hours of operation of some programs. A second common challenge in referring young people to external services that was reported by Youth on Track staff (accounting for about 15% of their comments) was the impact of the service providers’ risk assessments, for example, some providers require a case worker to accompany a young person who the service has assessed as being too high risk, some services do not accept complex young people or young people who have a high level of criminogenic need, and mental health services triage a referral to determine if it is safe for their clinicians to see the young person. Service providers’ lengthy waiting lists comprised about one in seven of the staff’s comments regarding each intervention.

The key difference in the challenges reported by Youth on Track staff in referring young people in each intervention focused on Fast Track’s timeframe (29.2% of 48 comments). Stakeholders noted that a case worker could spend several weeks attempting to negotiate for a young person to be allowed to return to school without achieving an outcome within the required six-week period, a young person could be exited from the intervention before the school term begins, referrals to psychologists or psychiatrists may not be accepted within the timeframe, and referrals must be left in the hands of the young people or their families to follow-up.

Strategies to overcome challenges in referring young people

There was overlap in the strategies employed by Youth on Track staff to overcome these challenges. Two of the main strategies used were assisting service providers through education or additional support and utilising existing relationships or establishing positive relationships (by, for example, attending inter-agency meetings and establishing partnerships with other community programs). These two strategies together accounted for 80.9% and 71.0% of the 42 and 38 comments made regarding Youth on Track and Fast Track participants, respectively. Examples of how Youth on Track staff assist service providers were by offering them information about successful strategies that Youth on Track staff have employed to manage young people’s difficult behaviour, educating the service providers about the different types of risk and risk management, case workers accompanying young people to appointments to build a bridge with the service provider, Youth on Track case workers providing input about the young person so that the intervention can be tailored to the young person, conducting informal case conferences by telephone with staff of the external service, and making warm referrals rather than simply expecting the young person to attend an appointment. Other strategies used by Youth on Track staff to overcome challenges in referring clients included contacting a different service (9.5% of comments made regarding Youth on Track participants and 5.3% of comments made regarding Fast Track participants) and being knowledgeable about local services (4.8% and 7.9%, respectively).

Most of the 37 relevant stakeholders stated that neither Youth on Track nor Fast Track participants are given priority by external service providers, rather they are treated the same as any other client referred to the provider.

Are Fast Track and Youth on Track being delivered as intended?

The vast majority of the relevant stakeholders stated that both Fast Track and Youth on Track were being delivered as intended. For Fast Track, some case workers noted “we are doing what we were instructed to do”.

Stakeholders perceived that the specific elements of the Youth on Track intervention were crucial for the intervention to be delivered as intended, for example, CHART, YLS and the family collaborative work. Comments regarding these elements comprised 34.5% of the 142 comments made by the relevant stakeholders. About three in ten (30.3%) comments referred to relationships, either relationships between the case worker and the young person (17.6%, eg case workers building rapport, trust and good relationships with the young people and their families), or relationships between the Youth on Track staff and stakeholders/agencies such as schools, YLOs, local organisations (6.3%) or community engagement, communication and support (6.3%). About 15% of comments noted that the characteristics of the individuals involved are key aspects of the intervention for it to be delivered as intended, namely, the quality of the case workers (9.1% of comments) and the attributes of the young people such as their level of engagement and their motivation to change (6.3%).

The relevant stakeholders noted that, for Fast Track to be delivered as intended, it was necessary for case workers to make each interaction with the young person as thorough as possible, with speed and timeliness being a priority (30.9% of 84 comments). As for the more comprehensive intervention, stakeholders also referred to the elements of Fast Track (19%, eg identifying the highest risk and most intensive needs, the more directed case plan, some brokerage) and case workers’ relationships with, and referrals to, other services/stakeholders (19%, eg having referral avenues; case workers knowing the local services to which young people could be referred, establishing relationships with other services and maintaining communication with the referral sources).

Is the delivery of Fast Track/Youth on Track producing any unexpected consequences (either positive or negative)?

Although 35% of the stakeholders interviewed stated that the delivery of Fast Track and Youth on Track did not produce any unexpected consequences, either positive or negative, others disagreed, with Fast Track comprising most (71%) of the 31 comments about unexpected negative consequences of the scheme’s delivery. One stakeholder remarked “the negative is Fast Track”. Comments included that Fast Track is not sufficient to achieve change in the young person (35.5% of comments) because it is too short, there is insufficient time to get positive outcomes with some clients, complex young people who need more than Fast Track have been allocated to the intervention and young people have re-offended after completing Fast Track. Other unexpected negative consequences of the scheme reported by stakeholders were that families are angry and disappointed when allocated to the Fast Track intervention (16.1%) and that Fast Track is simply a referral service (6.4%). Only a minority of comments (about one in ten) noted that the delivery of the scheme has had a negative impact on other services, for example, other agencies are providing an extended service so that young people can complete the goals that they are not completing with the Youth on Track case workers.

Two in five of the stakeholders’ 22 comments about unexpected positive consequences of the delivery of the two interventions related to the positive relationships and collaboration either created or enhanced between Youth on Track staff and local agencies, programs and staff. For example, stakeholders remarked that Youth on Track staff have strengthened their relationships with the police and PCYCs; new networks have been developed with job support agencies, the National Disability Insurance Scheme, local sports clubs; and Youth on Track case workers attend Youth Justice Conferences. Stakeholders noted that Youth on Track is another service in the local community (22.7% of comments). About one in five (18.2%) comments referred to some unexpected positive consequences associated with Fast Track, such as young people beginning TAFE courses and case workers placing young people into programs or school. Fast Track was observed to be effective for some young people, particularly those who are efficient in their everyday life, who are motivated, who have family support, who have transport, who can continue to engage with services after they exit Fast Track, who are at the lower end of the needs spectrum or who have issues that can be resolved within the intervention’s six-week timeframe. It was also noted that Fast Track is structured and focused in terms of how it is delivered and its timeframe. This was perceived to have a number of advantages, including making it possible to predict the duration of a young person’s engagement in the intervention, making it easier for the scheme’s managers to both allocate caseloads to the case workers and also to monitor those caseloads. Some comments referred to the fact that Fast Track does not create dependence on the case worker, it maintains the young person’s motivation and it enables case workers to assist more young people through shorter waitlists.

Is the evaluation (and the randomisation process) producing any unexpected consequences (either positive or negative)?

Of the stakeholders in a position to comment, two in five stated that there were no unexpected consequences of either the evaluation or the randomisation processes. In addition, most of the 28 stakeholders who had been involved with the Youth on Track scheme when randomisation commenced stated that the trial had no impact on the referral process. One in three of these stakeholders also noted that the vast majority of young people and their families consented to participate in the evaluation. Several stakeholders stated that consenting to participate in the evaluation went hand-in-hand with consenting to participate in the scheme.

Although all program staff who were interviewed reported that the evaluation requirements were being implemented as intended, three in ten (30.3%) of the 76 comments made identified challenges when gaining consent to participate in the scheme because of the evaluation. Staff referred to not being able to guarantee which of the two interventions the young person would receive prior to giving consent; case workers being required to explain both interventions as well as the evaluation process; difficulties in explaining the randomisation/evaluation processes; the limited capacity of the young people and their families to understand the randomisation process/evaluation; consent sometimes being conditional on the young person being allocated to the longer intervention; and case workers having to manage the disappointment of the young person and their family when allocated to Fast Track.

Several (n=81) comments were made about the consequences of the evaluation and the randomisation processes. Three-quarters of these comments were negative, but stakeholders stressed that the consequences were not unexpected. About one in five of the 61 comments regarding negative consequences related to the perception that young people may be allocated to an intervention which is not suitable for them. For example, it was observed that some young people had been allocated to Fast Track but would have benefited from the longer intervention, and conversely that some young people only need six weeks but had been assigned to the longer Youth on Track intervention. Other comments dealt with the difficulties associated with the randomisation process (11.5%, eg some conversations about randomisation have been unpleasant, and some young people and their families have refused to participate in the scheme because of the randomisation), the ramifications of Fast Track’s time limits (11%), the difficulties of gaining young people’s consent to participate in the program if they had previously been allocated to Fast Track and were re-referred to the scheme after having re-offended (9.8%), the frustrations of not knowing to which intervention the young person will be allocated prior to the initial home visit (8.2%), the fact that some young people who complete Fast Track re-offend (8.2%), and the possibility that the integrity of the program or the evaluation could be damaged (4.9%).

Although only 11.1% of the consequences of the evaluation and randomisation processes were perceived to be positive, almost three in five (55.5%) of these comments noted that Fast Track is suitable for some young people.

General comments

The vast majority of the 52 stakeholders who were interviewed believed that there is a need for the Youth on Track scheme. Some stakeholders asserted that the longer Youth on Track intervention was “great”, “fantastic” or “brilliant”. In fact, one in five (19.6%) stakeholders stated that all aspects of this intervention are working well. Conversely, some stakeholders stated “Fast Track has no place” or “I do not like Fast Track”.

Although one in four stakeholders stated that the Youth on Track intervention required no improvements, the remaining stakeholders suggested improvements in its operation. About two in five (39.0%) of the 77 comments referred to modifying various elements of the intervention. This included modifying the specific interventions (14.3%), for example, changing the method of delivering the structured family intervention, delivering some behavioural interventions to young people in group settings rather than individually, and increasing the flexibility in delivering CHART. Modifications were also suggested to the eligibility criteria (10.4%, eg focusing on earlier intervention, before young people have had a formal police contact and accepting young people at risk into the scheme), the parameters (10.4%, eg making the scheme mandatory rather than voluntary and making participation in Youth on Track a bail condition) or the referral pathways (3.9%).

Suggested improvements in operation of Youth on Track

One in five (20.8%) of the suggested improvements in the operation of the Youth on Track intervention related to improving communication, dissemination of information and collaboration between Youth on Track staff and local agencies. The suggestions included informing relevant stakeholders of the progress being made by the young people, perhaps through case co-ordination meetings; involving schools more directly in the delivery of the scheme, for example, in the development of the young person’s plan so that the school could support the young person and celebrate the successes; or offering some programs in the local schools that directly relate to criminogenic needs (eg “Love Bites”).[16] Although only accounting for 3.9% of the suggestions, a related suggestion to collaboration and communication was to increase community capacity, for example, increasing community awareness of the scheme, and reaching out to the local Aboriginal communities to establish and extend trust.

Other suggested improvements were to increase resources (10.4% of comments, eg increase the number of case workers in order to decrease the waitlist, appoint a family therapist as part of the Youth on Track team), improve administrative aspects of the scheme (7.8%, eg streamlined documentation) and analyse the engagement and disengagement processes to determine what is working and what is not working (3.9%).

Suggested improvements in operation of Fast Track

Stakeholders also suggested a number of improvements to the operation of Fast Track. Almost two in five (37.9%) of the 66 comments related to changing and clarifying elements of the intervention. The most frequent of these suggestions, accounting for one in five of the comments, was to increase the length of the program, perhaps to eight or 12 weeks instead of the current six weeks. One stakeholder noted that this would enable the case worker to build some rapport with the young person, encouraging the young person to open up. This, in turn, would be beneficial for both the case worker and the young person, facilitating the young person’s engagement with more services. The additional time would also allow the case worker greater flexibility to work with the young person rather than being completely task-focused in order to accomplish all that is required within the six-week timeframe. Another suggested change to elements of the intervention was to allow an unlimited number of face-to-face interactions between the case worker and the young person for the duration of the program, rather than restricting the number of interactions to four. One stakeholder asserted that even six weeks would be sufficient if the case worker could meet with the young person every day, noting that working intensively with a young person is different to one meeting a week. Other suggested changes to elements of the shorter intervention were to clarify the expectations of both the case worker and the young person for the duration of the intervention, beginning the six-week period of the intervention three weeks after consent to take into account the difficulties often associated with contacting the young person, and formalising a case conference with the service providers to which the young people have been referred so that case workers know whether the young person has engaged with those providers.

Nine per cent of the suggested improvements to the operation of Fast Track involved modifications to the intervention’s eligibility criteria. This included restricting Fast Track to young people with specific characteristics, such as those with fewer needs, those with minimal interactions with the police/court, those with supportive families, or to young people aged between 14 and 17 years who have fewer service referral pathways. About 6% of the suggested improvements to the operation of Fast Track related to increasing access to services and having a broader range of services to which young people could be referred. Stakeholders also stressed the need for case workers to have the discretion to decide which intervention a young person receives; this accounted for 13.6% of comments.

Discussion

The main purpose of this study was to obtain stakeholder perceptions about whether the Youth on Track randomised controlled trial was being implemented as intended and the challenges associated with the scheme’s implementation. Interviews were conducted with 52 stakeholders, including Youth on Track staff, Youth Liaison Officers, school-related staff and representatives of various service providers.

The relevant stakeholders were unequivocal in stating that both the Youth on Track and Fast Track interventions were being implemented as intended. In addition, young people are engaging well in each intervention, with the randomisation process having little impact on engagement rates. Several stakeholders went so far as to state that participation in the evaluation went hand-in-hand with participation in the scheme. This is corroborated by the administrative data that shows that 92.5% of eligible young people consented to participate in the Youth on Track evaluation.

Stakeholders highlighted two very distinct differences between Youth on Track and Fast Track:

1. 

program length; and,

2. 

the types of interventions that can be offered.

While these differences were a design feature of the randomised controlled trial, stakeholders stressed that these two elements are problematic for Fast Track participants. Youth on Track staff, in particular, claimed that, Fast Track does not allow sufficient time for the case workers to build rapport with the young person or to deliver services that could produce change in the young person. The second key difference between Youth on Track and Fast Track highlighted by stakeholders was that young people allocated to the former receive a variety of interventions, in particular CHART and the family collaborative intervention. On the other hand, stakeholders noted that, due to its parameters, case workers are not permitted to provide either family or behavioural interventions to young people allocated to Fast Track. This is supported by the data held in the Youth on Track administrative database. The latter confirmed that no young person in Fast Track received family interventions and less than 1% received behavioural interventions. Furthermore, stakeholders reported that Fast Track participants are primarily referred to, or linked with, other service providers, for example, housing, local mental health services, drug and alcohol services, education, legal services support and family support.

While two in five Youth on Track staff reported no challenges in referring Fast Track participants to external services, others noted a lack of services in the local area and lengthy waiting lists, particularly in rural areas. The latter issue is relevant to participants of both Youth on Track and Fast Track, however, its impact is magnified for young people on Fast Track because often appointments that are arranged by the case workers cannot be scheduled within the six-week period of the intervention.

Many stakeholders, particularly the Youth on Track staff, also voiced concerns over the random allocation of young people to an intervention with no consideration of the young person’s needs. This issue dominated stakeholder comments. Most of these stakeholders were concerned about the negative impact of randomly allocating young people with multiple and complex needs to the shorter intervention, particularly if the young person has insufficient family support to assist them after the case worker exits them from the intervention.

Although most stakeholders had a negative perception of Fast Track, some stakeholders conceded that it had benefits. They noted that some positive outcomes have been achieved in Fast Track’s limited timeframe and acknowledged that the intervention accelerates case workers’ progress through the waitlist of young people who have been referred to the scheme. It was also recognised that the shorter intervention enables at least some of the scheme’s elements to reach a greater number of young people who otherwise may not have received any services. In addition, a few stakeholders conceded that a randomised controlled trial is the best evaluation methodology to determine which of the two interventions is the most effective in reducing re-offending amongst this target group of young people. They accepted that, until the research was completed, it was necessary for young people to be randomly allocated to one of the two interventions rather than allocation being based on an assessment of the young person’s needs, backgrounds and circumstances.

BOCSAR’s outcome evaluation which will be completed in 2021 will compare re-offending rates amongst young people who were randomly allocated to each intervention and determine whether the longer, more comprehensive Youth on Track intervention confers a benefit in terms of reducing a young person’s likelihood of re-offending. It will clarify whether some stakeholders are correct in their assertion that young people who are referred to Fast Track are more likely to re-offend because the time available in the intervention is insufficient to make a positive impact on the young person’s life and external services are less accessible. Although stakeholders expressed concerns about randomly allocating young people to an intervention, the randomisation process itself appeared to have little impact on engagement rates, and both interventions were largely implemented as intended. This information will be invaluable when interpreting findings from the outcome evaluation. However, more broadly these findings demonstrate that randomised controlled trials can be successfully employed to evaluate criminal justice programs and interventions without compromising the integrity of the programs and interventions being delivered.



[1]  This paper was originally published as L Trimboli, Youth on Track randomised controlled trial: process evaluation (Issue Paper No 141), NSW Bureau of Crime Statistics and Research, Sydney, 2019.

[2]  NSW Department of Justice, Youth on Track Service Performance Framework, 2018, p 2 at www.youthontrack.justice.nsw.gov.au/Documents/yot-performance-framework.PDF, accessed 9 October 2019.

[3]  The relevant organisations are Mission Australia servicing the Blacktown, Central West, Hunter and Mid-North Coast sites; Social Futures servicing Coffs Clarence; and Centacare NSW servicing New England. See D Elliott, Minister for Corrections, Emergency Services and Veterans Affairs, NSW Government, Innovative program gets Youth on Track, media release, 17 October 2016 at www.justice.nsw.gov.au/Documents/Media%20Releases/2016/Innovative-program-gets-youth-on-track.pdf, accessed 9 October 2019.

[4]  GRAM or Group Risk Assessment Model is an actuarial risk assessment instrument whose scores are indicative of a defendant’s risk of re-offending within 24 months of an index offence. (See N Smith and C Jones, “Monitoring trends in re-offending among adult and juvenile offenders given non-custodial sanctions”, Crime and Justice Bulletin No 110, NSW Bureau of Crime Statistics and Research, 2008 at www.bocsar.nsw.gov.au/Documents/CJB/cjb110.pdf, accessed 9 October 2019; N Smith and C Jones, “Monitoring trends in re-offending among offenders released from prison”, Crime and Justice Bulletin No 117, NSW Bureau of Crime Statistics and Research, 2008 at www.bocsar.nsw.gov.au/Documents/CJB/cjb117.pdf, accessed 9 October 2019.) Technically, GRAM is a logistic regression model that predicts reconviction on the basis of a number of offender and offence characteristics. The GRAM score is calculated on the basis of various individual-level static risk factors, including the defendant’s age, gender, Aboriginal or Torres Strait Islander origin, prior criminal history and current offences. One application of the risk assessment instrument is to screen individuals who come in contact with the criminal justice system for further intervention, identifying which individuals would benefit most from referral to specific programs that are designed to reduce their likelihood of re-offending. For the Youth on Track scheme, a young person’s GRAM score is calculated by the NSW Police Force’s Screening Officer at the time of the young person’s referral to the scheme. A GRAM score of <0.6 indicates that a young person’s predicted risk of re-offending within 24 months is moderate to low; a score between 0.6 and 0.7 indicates a moderate to high risk of re-offending and a score of >0.7 indicates a high risk of re-offending.

[5]  The YLS/CMI-AA is an adaptation of a parent tool (YLS/ CMI) that was developed in Canada. It was adapted to improve its validity and reliability in the Australian context and to accommodate an older age range. See A Thompson and Z Pope, “Assessing juvenile offenders: preliminary data for the Australian adaptation of the youth level of service/case management inventory (Hoge & Andrews, 1995)” (2005) 40(3) Australian Psychologist 207. Use of the YLS/CMI-AA by Youth on Track case workers began on 1 January 2015. Prior to this date, the tool used was the YLS/CMI.

[6]  D Andrews et al, “Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis” (1990) 28(3) Criminology 369 at 374–375 describe these principles in the following ways:

The risk principle suggests that higher levels of service are best reserved for higher risk cases and that low-risk cases are best assigned to minimal service … the effects of treatment typically are found to be greater among higher risk cases than among lower risk cases. This is expected unless the need and/or responsivity principles are violated.

[need principle] The most promising intermediate targets [for correctional rehabilitation programs] include changing antisocial attitudes, feelings, and peer associations; promoting … [family bonds] …; promoting identification with anti-criminal role models; increasing self-control and self-management skills; replacing the skills of lying, stealing, and aggression with other, more pro-social skills; reducing chemical dependencies; and generally shifting the density of rewards and costs for criminal and noncriminal activities in familial, academic, vocational, and other behavioural settings … Less-promising targets include increasing self-esteem without touching antisocial propensity …, increasing the cohesiveness of antisocial peer groups …, improving neighbourhood-wide living conditions without reaching high-risk families …, and attempts to focus on vague personal/emotional problems that have not been linked with recidivism …

The responsivity principle has to do with the selection of styles and modes of service that are (a) capable of influencing the specific types of intermediate targets that are set with offenders and (b) appropriately matched to the learning styles of offenders … Specifically, they include modeling, graduated practice, rehearsal, role playing, reinforcement, resource provision, and detailed verbal guidance and explanations (making suggestions, giving reasons, cognitive restructuring).

[7]  Juvenile Justice NSW, CHART: Changing Habits and Reaching Targets, 2015 at https://justice.nt.gov.au/__data/assets/pdf_file/0003/238188/chart-resources.pdf, accessed 10 October 2019, adapted for use in NSW.

[8]  C Trotter, Collaborative family work: a practical guide to working with families in the human services, Allen & Unwin, Sydney, 2013.

[9]  Re-Navigating Anger and Guilty Emotions (RAGE) is a six-week anger management course targeting young people aged between 11 and 17 years. Each session lasts about two hours. The course incorporates a number of themes, including the different anger styles; identifying the triggers, thoughts, tantrums and troubles ("the four Ts") of the anger cycle; healthy expressions of anger; understanding that guilt is part of the cycle of anger and learning healthy ways of dealing with it; understanding the importance of relaxation, exercise and diet on the state of mind and emotions.

[10]  NSW Bureau of Crime Statistics and Research (BOCSAR), July 2014, unpublished; Cultural and Indigenous Research Centre of Australia (CIRCA), Youth on Track social outcomes evaluation: final report, 2017 at www.youthontrack.justice.nsw.gov.au/Documents/circa-evaluation-final-report.pdf, accessed 10 October 2019.

[11]  CIRCA found no statistically significant change in mean YLS/CMI-AA score for peer relations between the first (mean = 3.1) and second (mean = 2.8) assessment at three months; however, there was a significant reduction between the first (mean = 3.2) and third (mean = 2.5) assessment at six months. See CIRCA, ibid, pp 39–40.

[12]  However, some categories of young people are excluded from the study, namely, a young person referred to the Youth on Track scheme who has had a household member already randomised into an intervention will be allocated to the same intervention as that household member, but only the first-placed young person will be included in the study; if a study participant re-offends during the trial and is re-referred to the scheme, he/she is excluded from the randomisation process and is not counted as a new participant in the evaluation.

[13]  The YLS/CMI:SV is an abbreviated form of the YLS/ CMI 2.0. It was specifically designed to provide an initial screening of risk and need levels in young people allocated to the Fast Track intervention to determine the level and nature of interventions required by these young people. This inventory has eight categories: history of conduct problems, current school or employment problems, some criminal friends, alcohol/drug problems, leisure/recreation, personality/behaviour, family circumstances/parenting and attitudes/orientation.

[14]  Bellberry Application 2017-05-361.

[15]  From a GRAM score of 0.6 to 0.4 (the GRAM, or the Group Risk Assessment Model, score is indicative of a defendant’s risk of re-offending within 24 months of an index offence, a score of less than 0.6 indicates that the risk is moderate to low).

[16]  “Love Bites” is a school-based program targeting students in Years 9 and 10. It is designed to promote respectful relationships for young people and to raise awareness about sexual assault, the concept of consent and domestic and family violence.