Choosing a Model
The model that I would choose to evaluate this program is Scriven’s consumer-oriented approach to evaluation based on what he labelled goal-free evaluation (Stufflebeam and Shinkfield, 2007). The looks at all of the effects and outcomes of the program regardless of the stated goals and objectives making it an appropriate model when the goals are not known (Stufflebeam and Shinkfield, 2007). The summative evaluation provides the consumer with an independent assessment of the value, merits, worth and significance of the program including comparison with other program options (Scriven, 1991).
I feel this is an appropriate model for evaluating the Aboriginal Prenatal Exercise Program because although the relationship between physical activity and health in pregnancy is mentioned, the specific goals of the program are not identified. The program was developed based on a survey of pregnant Aboriginal women not on predetermined goals of the program developers. Therefore it seems appropriate to have a client-oriented approach to focusing on what the program is doing rather than the developer’s expectations. This approach focuses on the client and the human experience of the program. The program had a number of components to it in addition to the exercise including nutritional snacks, informational resources and socializing that may have resulted in unexpected outcomes that only the clients would be able to identify. One of the strengths of this evaluation method is that by not focusing on the intended goals it allows for the identification of unexpected outcomes (positive or negative).
This program is finished and so it is a summative evaluation. It was identified as part of a pilot study and so the purpose of evaluating it now would probably be to determine if similar programs developed now or in the future would have value for this particular target population. Scriven’s (1991) evaluation checklist emphasizes determining the value of both the process and the outcomes and once the merits, worth and significance of the program has been determined, comparing the program using a cost-benefit analysis to possible alternatives available. Given this program was a whole package of services (free childcare, bus tickets and bathing suits, etc.) this kind of comprehensive evaluation looking at the value of the process as well as all of the outcomes is important. How it compares to other options will be determined by the clients and takes into consideration the value of all aspects of the program.
References
Stufflebeam, D.L. & Shinkfield, A.J. (2007). Evaluation theory, models, & applications. San Francisco, CA: Jossey-Bass A Wiley.
Scriven, M. (1991). Key evaluation checklist [on-line]. Retrieved Janaury 25, 2011 from www.wmich.edu/evalctr/checklists.
Kim,
ReplyDeleteI like the point you make when you say "by not focusing on the intended goals it allows for the identification of unexpected outcomes (positive or negative)." I too felt that Scriven's approach would be of value to this case study.
...and I like that you mention that cost/benefit analysis and evaluation should be considered of the components that were designed to remove barriers to participation. Good job, Kim.
ReplyDeleteKim you have made a solid assessment before choosing Scriven's model and I support the decision you have made. Yes as the program has reached a self-determined terminal point a summative evaluation is appropriate. As Laura mentioned you are one of the few folks who identified the potential of conducting a cost-benefit analysis and this is always a major concern with NGOs. Good work.
ReplyDeleteJay