“Evaluation can be viewed as a structured process that creates and synthesizes information intended to reduce the level of uncertainty for stakeholders about a given program or policy. It is intended to answer questions or test hypotheses, the results of which are then incorporated into the information bases used by those who have a stake in the program or policy” (James C.McDavid and Laura R.L. Hawthorn, (2006), p.3)
This was my place to start when looking at assignment two. I really needed to know why it was important to evaluate such a program and what happens to the information obtained. It is essential to see some of the following, I think, when looking at evaluation:
· Is the program effective?
· Is the program meeting the outcomes or goals?
· Should the program continue? Is it successful?
· What are the benefits of the program?
· Has there been any important development or changes that need to be made to the program? Etc., etc.
Looking at the article on the prenatal exercise program for urban Aboriginal women, I would use the Scriven method because it has a simple approach to evaluation while still being able to focus on the goals that need to be obtained.
The prenatal exercise program had a purpose, to decrease gestational diabetes mellitus (GDM) among urban Aboriginal women by providing an exercise program as well as to educate the participants about GDM. I did find it hard to decide what they would want to evaluate, the program, the success, if the program actually reduced the rate of GDM?? I don’t feel the program was clear in what they wanted to achieve as the end result, or at least defining the specific goals.
There is a reason for the program and the offerings of the course are clearly documented. I feel they could have been more specific in the program in terms of what would make it a success (each participant participate 100% of the time; look at diet, regular checkups, amount of exercise, etc.) “Initiatives to promote regular physical activity among Aboriginal women during their childbearing years could play a major role in optimizing healthy pregnancies and in reducing rates of type 2 diabetes in future generations” (Klomp, H., Dyck, R., and Sheppard, S. (2003), p. 1). What is defined or accounts as ‘regular’ physical activity?
I would also assume that in order for this program to continue, feedback would need to be obtained as well as data ensuring that the program is and has been a success. Following Scriven’s model, the formative feedback would provide information during the program for any changes that would need to be made. Summative evaluation would occur at the end of the program to see if the original goals have been met and if the program has been effective.
Although I think the program had good intentions I felt the program wasn’t clearly defined in terms of what the end outcome or result would be. Was it to inform and educate participants, provide an exercise program to a target group, collect data on participants who were in the program, or all the mentioned? I think an evaluation using Scriven’s method would help the program in terms of letting them know what changes they need to make in order to achieve the desired success and to also obtain the necessary data to make the changes.
James C. McDavid & Laura R.L. Hawthorn. (2006). Program Evaluation & Performance Measurement: An Introduction to Practice. Thousand Oaks, CA: Sage Publications.
Hi Lisa,
ReplyDeleteThis question came to mind as I read your post. Did this spontaneous initiative not have clear goals or do the evaluators of it not define the goals for measuring. Could have they had clear goals for the main part of the study and this is an articles that discusses another interesting thing they did. I wonder if this is only part of the article with the findings missing.
Thoughts?
Glenys
I actually wondered that too after I 'googled' the ariticle. It seemed like there was a bit more information, tables with findings, etc. I am assuming that this was the 'brief' article and that the goals may have been a bit more clear in the original plan...I hope.
ReplyDeleteI would also hope that when people do evaluations that they include all the information?? I guess this would depend on if they were provided all the information. This is still very new to me.
I was thinking exactly your thoughts - that if the program planners had been more clear on their goals, then the outcomes could be measured and we could call it a success, or not. However, then I thought about what Glenys and I chatted about in terms of values and I think this is probably one of those cases where we may be imposing a value system on a culture that may measure success in different ways. Just food for thought!
ReplyDeleteWell done analysis!
I was wondering, Lisa, if the reason the goals were so unclear was because there was not a specific audience for the evaluation. The information shared would be different depending on who was asking for it, no? That way, only the information needed would have to be included - just what was pertinent to the audience. It does not state who requested the evaluation in the first place.
ReplyDeleteYeah Laura, I didn't even consider if there would or could be different cultural values....good point!
ReplyDeleteReally, if this program prevented one person from getting GDM it would be good.
Good point Kate! I didn't even think of that. Maybe they were just providing an overview of the program and the information provided would be sufficient?? I wonder who requested the evaluation??
ReplyDeleteLisa it is obvious that you needed to be prepared for addressing this assignment. Clearly reflection is also one of your strong suits. You ask many great questions about the purpose and design of the program. It is crucial for evaluators to involve themselves deeply to understand what they are evaluating and why before they get too far along in a PE. Scriven's model is an appropriate choice for your evaluation.
ReplyDeleteWell done.
Jay