Cross site evaluation of the Strategic Prevention Framework State Incentive Grants: Analysis of state-level intervening variables

BACKGROUND: CSAP has been conducting cross site evaluation of the Strategic Prevention Framework (SPF) State Incentive Grants (SIG) program since 2004.  SPF lays out five ordered steps to guide states and communities through a logical planning process for substance abuse prevention programs.  First two cohorts of juristictions that was awrded the grants completed implementations of the program and reported enough data to report substantive preliminary results. This paper describes the use of logic models by states in planning prevention projects.

METHODS:  Most states examined behavioral outcomes such as alcohol consumption and measured intervening variables (IVs) such as attitudes and norms and perceived risk of harm.  We compared pre- and post-intervention community averages on intervening variables and outcomes and computed change scores.  We then used non-parametric tests to assess statistically significant changes and differences. 

RESULTS:  Our analyses revealed that communities with SPF SIG funding generally reported more positive changes in IVs than negative or no changes.  Despite extensive heterogeneity in our sample data, 20.3% of tests of association between intervening variables and outcomes were statistically significant in a positive direction, whereas only 4.2% were negatively statistically significant, essentially a 5:1 ratio of positive to negative associations.  

CONCLUSIONS:  While challenging, communities that have created positive changes in intervening variables most often also saw positive changes in outcomes.  We conclude that the use of logic models helped states align prevention interventions with corresponding measures of change leading to stronger evaluations of cause and effect.