Opening the Black Box: Using Process Evaluation Measures to Assess Implementation and Theory Building

Authors: Karachi T.W.¹; Abbott R.D.²; Catalano R.F.²; Haggerty K.P.²; Fleming C.B.²

Source: American Journal of Community Psychology, Volume 27, Number 5, October 1999 , pp. 711-731(21)

Publisher: Springer

Abstract:

The past decade has seen increasing recognition in prevention science of the need to move away from a black box approach to intervention evaluation and toward an approach that can elaborate on the mechanisms through which changes in the outcomes operate (Chen & Rossi, 1989; Durlak & Wells, 1997; Spoth et al., 1995). An approach that examines issues of program implementation is particularly critical in the design of efficacy studies of school-based preventive interventions. Numerous preventive intervention strategies are now delivered within the schools, often by regular classroom teachers. The extent to which teachers faithfully deliver a particular curriculum or incorporate instructional strategies emphasized by an intervention is a critical question for the overall project evaluation. This article illustrates the utilization of process measures from a multicomponent school-based prevention program to examine implementation of a teaching staff development intervention, and the program's underlying theoretical basis. Given the nested study design, the analyses utilize hierarchical linear models (Bryk & Raudenbush, 1992) to examine changes in teaching strategies by condition and investigate the hypothesized relationships between teaching practices and student behaviors based on the program's theoretical framework. Results suggest that teaching practices in two of the six intervention focus areas were positively impacted in the first 18 months of the project. Findings also support th relationships between teachers' instructional practices and students' behaviour.

Keywords: program implementation; school-based prevention

Language: English

Document Type: Research article

Affiliations: 1: Social Development Research Group, University of Washington, Seattle, Washington 98195 2: Social Development Research Group, University of Washington, School of Social Work

Publication date: 1999-10-01

• In this: publication
• By this: publisher
• In this Subject: Public Health
• By this author: Karachi T.W. ;  Abbott R.D. ;  Catalano R.F. ;  Haggerty K.P. ;  Fleming C.B.

Website © Publishing Technology. Article copyright remains with the publisher, society or author(s) as specified within the article.

• Terms and conditions
• Privacy policy
• Information for advertisers