Effectiveness study

Behavioral Parenting Training (BPT) groups are the most effective treatment for disruptive behavior problems in young children, and a cost-effective way to prevent their escalation (Menting, De Castro & Mattthys, 2013; Posthumus, 2009). Programs taking a collaborative approach have been shown to be particularly appealing to diverse parents, with high inclusion, motivation, attendance, and satisfaction among parents from diverse cultural and socioeconomic backgrounds (e.g. Menting, De Castro, Wijngaardts-de Meij & Mattthys, 2014).

Nonetheless, existing well-established programs may be further optimized, particularly with regard to cultural fit of program materials to today’s “superdiverse” families and (sub)cultures and for brevity. To this end, the Invest In Play (iiP) program was developed with the goals of utilising core components of existing evidence-based parent programs, co-creating materials for multiple cultural backgrounds, and maximizing efficient implementation with a 12-session program and on-line curriculum access. The question is whether the new iiP parent program is effective in its goals to decrease child behavior problems and parenting stress, provide a good cultural fit for participating families, and be a less costly intervention.

The present study aimed to provide a tentative initial answer to these questions for the first iiP groups given in Denmark. To this end, we compared pre and post measurements form the pilot iiP groups in Denmark to benchmarks for control and intervention groups derived from an individual patient data meta-analysis of European trials of a well-established evidence-based BPT program (Leijten et al., 2018). The present findings are meant as a first building block while establishing larger and more rigorous studies of iiP effects. In addition, it is meant to illustrate how integrated assessment and evaluation in iiP may contribute to a continuous loop of learning and improvement as the program is further developed. 

In this quasi-experimental benchmark study, we tested the following hypotheses:

 - iiP decreases parent reported child behavior problems significantly compared to a control group (see below)
- iiP decreases paren;ng stress significantly compared to a control group (see below)
- iiP decreases parent reported child behavior problems as much as aggregated comparison BPT groups
- iiP decreases paren;ng stress as much as aggregated comparison BPT groups
- Parental sa;sfac;on with iiP is larger than sa;sfac;on with comparison BPT groups at the same loca;on
- iiP costs per group are lower than for comparison BPT groups at the same locaton.

Pre- and post-assessments of the first four Danish iiP parent groups were obtained online as an integral part of the iiP procedure. On both occasions, 25 to 33 parents of 25 families filled out the ECBI child behavior questionnaire, the PSI parental stress questionnaire, and a satisfaction questionnaire.

As a benchmark, aggregated means and standard deviations pre- and post-intervention for comparison BTP intervention and control groups were drawn from Leijten et al., 2018: an individual participant data meta-analysis of 14 randomized trials conducted in Europe, with the same (or harmonized) measures ECBI and PSI, with a total N of 544 to 1622, depending on the measure.

With regards to program satisfaction, ratings by parents in the first four iiP groups were compared to ratings given by parents in two BPT comparison groups held at the same location in the same institute. Parents rated 10 items about program satisfaction on a Likert scale of 1 to 5, see Figure 3 in the results for the items.

First estimates of costing were provided by the treatment institute, based on scheduling of sessions and facilitator hours.

iiP reduces child behavior problems significantly (a mean decrease of .65 standard deviations), in comparison to the aggregated control groups (.27). The magnitude of the iiP decrease in child behavior problems is equal to the mean decrease in aggregated comparison BPT groups
(.64).

iiP reduces parenting stress significantly (a mean decrease of .85 standard deviations), in comparison to the aggregated control groups (.18). The magnitude of the iiP decrease in parenting stress (.85) is larger than the mean decrease in aggregated BPT groups (.37)

Total program satisfaction was high for both iiP and the comparison BPT. Total program
satisfaction was higher for iiP than for the comparison BPT, which is particularly due to
pronounced differences in satisfaction with the video materials and relevance to daily life.

Time for delivering iiP was estimated at 22 hours per family, estimated from 2 group leaders each utilising 6-7 hours a week for 12 weeks for a group of 7 families (14 parents). Time for delivering the comparison BPT at the same institute was estimated at 43 hours per family, estimated from 2 group leaders each utilising 9-10 hours a week for 16 weeks for a group of 7 families. Thus, costs at this specific institute in Denmark were estimated to be 48% lower for iiP than for the comparison BPT.

The present ongoing study aimed to provide a tentative initial indication of the effects of the very first iiP groups given in Denmark. To this end, we compared pre and post measurements form these pilot iiP groups in Denmark to benchmarks for control and intervention groups derived from an individual participant data meta-analysis of European BPT trials (Leijten et al., 2018).

These four pilot iiP groups in Denmark effectively reduce child behavior problems and parenting stress, in comparison to aggregated active and passive control groups from 14 European trials. The magnitude of these reductions is comparable to the comparison BPT for child behaviour problems, and considerably larger than the comparison BPT for parental stress, whilst parents in the iiP groups were somewhat more satisfied with the program materials, fewer sessions were needed, and costs were considerably lower at this specific location in Denmark.

These first findings should be interpreted with great care, though. The iiP data were derived from the very first groups given and may vary as the program is further developed and scaled up. Data on consecutive groups and on multiple locations in multiple countries will be systematically added. Moreover, the benchmark comparison seems fitting, but it is a less strong design than propensity score matching of these data (which we are currently performing) or randomized controlled trials. Nonetheless, it seems worth noticing that this first evidence meets initial criteria of several national standards.

The present findings are meant as a first building block while establishing larger and more rigorous studies of iiP effects. In addition, it is meant to illustrate how integrated assessment and evaluation in iiP may contribute to a continuous loop of learning and improvement as the program is further developed.

Leijten, P., Gardner, F., Landau, S., Harris, V., Mann, J., Hutchings, J., ... & Scott, S. (2018).
Research review: Harnessing the power of individual participant data in a meta-analysis of the
benefits and harms of the incredible years parenting program. Journal of Child Psychology and
Psychiatry, 59(2), 99-109.
Menting, A. T., de Castro, B. O., & Matthys, W. (2013). Effectiveness of the Incredible Years parent
training to modify disruptive and prosocial child behavior: A meta-analytic review. Clinical Psychology
Review, 33(8), 901-913.
Menting, A. T., de Castro, B. O., Wijngaards-de Meij, L. D., & Matthys, W. (2014). A trial of parent
training for mothers being released from incarceration and their children. Journal of Clinical Child &
Adolescent Psychology, 43(3), 381-396.
Posthumus, J. A. (2009). Preventive effects and cost-effectiveness of the Incredible Years program
for parents of preschoolers with aggressive behavior. Doctoral Dissertation, Department of Child and
Adolescent Psychiatry, University Medical Centre Utrecht, the Netherlands.