Assignment 2: Description and evaluation of a prenatal exercise program for urban Aboriginal Women

Description and Evaluation of a prenatal exercise program for urban Aboriginal Women

“Evaluation can be viewed as a structured process that creates and synthesizes information intended to reduce the level of uncertainty for stakeholders about a given program or policy.  It is intended to answer questions or test hypotheses, the results of which are then incorporated into the information bases used by those who have a stake in the program or policy” (James C.McDavid and Laura R.L. Hawthorn, (2006), p.3)

This was my place to start when looking at assignment two.  I really needed to know why it was important to evaluate such a program and what happens to the information obtained.  It is essential to see some of the following, I think, when looking at evaluation:

·       Is the program effective?

·       Is the program meeting the outcomes or goals?

·       Should the program continue?  Is it successful?

·       What are the benefits of the program?

·       Has there been any important development or changes that need to be made to the program? Etc., etc.

Looking at the article on the prenatal exercise program for urban Aboriginal women, I would use the Scriven method because it has a simple approach to evaluation while still being able to focus on the goals that need to be obtained.   

The prenatal exercise program had a purpose, to decrease gestational diabetes mellitus (GDM) among urban Aboriginal women by providing an exercise program as well as to educate the participants about GDM.  I did find it hard to decide what they would want to evaluate, the program, the success, if the program actually reduced the rate of GDM??  I don’t feel the program was clear in what they wanted to achieve as the end result, or at least defining the specific goals.

There is a reason for the program and the offerings of the course are clearly documented.  I feel they could have been more specific in the program in terms of what would make it a success (each participant participate 100% of the time; look at diet, regular checkups, amount of exercise, etc.)  “Initiatives to promote regular physical activity among Aboriginal women during their childbearing years could play a major role in optimizing healthy pregnancies and in reducing rates of type 2 diabetes in future generations” (Klomp, H., Dyck, R., and Sheppard, S. (2003), p. 1).  What is defined or accounts as ‘regular’ physical activity?

I would also assume that in order for this program to continue, feedback would need to be obtained as well as data ensuring that the program is and has been a success.  Following Scriven’s model, the formative feedback would provide information during the program for any changes that would need to be made.  Summative evaluation would occur at the end of the program to see if the original goals have been met and if the program has been effective. 

Although I think the program had good intentions I felt the program wasn’t clearly defined in terms of what the end outcome or result would be. Was it to inform and educate participants, provide an exercise program to a target group, collect data on participants who were in the program, or all the mentioned?  I think an evaluation using Scriven’s method would help the program in terms of letting them know what changes they need to make in order to achieve the desired success and to also obtain the necessary data to make the changes.

James C. McDavid & Laura R.L. Hawthorn.  (2006).  Program Evaluation & Performance Measurement:  An Introduction to Practice.  Thousand Oaks, CA:  Sage Publications.

Assignment 1: Review of DEUSM Program Evaluation

DEUSM Program Evaluation
This is the first review I have attempted on a program evaluation.  I hope I am somewhat on track :)  Please feel free to provide feedback, suggestions or comments as I am sure it will be useful in the future.

The Medical Education Department made revisions to the existing program evaluation approaches at Dokuz Eylul University School of Medicine (DEUSM) in June of 2005.

After careful consideration they chose a mixed evaluation model that was systematic and multidimensional and was developed to meet their institutional needs.  I also found some of Scriven’s evaluations methods present after reading through the program evaluation.

The evaluation consisted of three main questions as the areas of inquiry based on the general education goals:

1.      What are the effects of the educational program on students and graduates?

2.      What are the effects of the educational program on trainers?

3.      Is the educational program being implemented as planned?

I think it is impressive when an institution conducts a program evaluation in order to make any necessary changes than in turn will make their program more successful.  After only one year of the evaluation the institution made important revisions to the educational program.  This was a case where the program evaluation was completed for a purpose, improving the current program. 

Some of the strengths and weaknesses I found for DEUSM’s evaluation are noted below:

Strengths:

·         Started the evaluation with a purpose.

·         Had 3 guiding questions for the evaluation.

·         Made necessary changes after only one year.

·         Had a variety of methods for data collection.

·         Had a place for data analysis and interpretation.

·         Results were used for program improvement.

·         Obtained qualitative and quantitative data.

·         Looking at An Overview of Evaluation Theories by Michael Scriven, the idea of allowing those who are being evaluated to participate in the evaluation has become increasingly popular.  This is more collaborative and participatory in terms of evaluation.  It makes sense to have the people being evaluated take part in the evaluation and this was the case for DEUSM.

Weaknesses:

·         Data collected for the most part was only once a year.

·         Due to time limitations some activities were moved to the next year.

Overall I feel the program evaluation was a success and met the needs of the institution.  They made effective changes based on the two year evaluation program that included revising their curriculum, reducing the frequency of exams, and a diversification of socio-cultural activities in order to meet the needs of the students.  Other content based changes were made and the school plans on continuing the program evaluation.

As stated earlier I found that the DEUSM program evaluation adheres to Scriven and Mix methods of evaluation.

Some of the key points of program evaluation according to Scriven that the DEUSM has used are:

•         Simple approach to evaluation (started with 3 questions)

•         Purpose of evaluation can be goals or roles (goal was to improve the current program)

•         Goals - outcomes of the program, reason for a program, really need to study the goals of a program (DEUSM questions 3, is the educational program being implemented as planned?)

Scriven also says that the formative part of evaluations gives feedback during the delivery of a program for immediate or future modification.  DEUSM started making modifications after the first year of evaluation.

 Scriven, (1991) also says that formative evaluation is typically conducted during the development or improvement of a program; in this case it was their medical program that they wanted to improve.  He also states that the summative part of the evaluation can evaluate the learning materials and learning process to see what changes need to be made for improvement which DEUSM did after one year.

Mix Methods

•         Rarely is a pure theory applied (DEUSM had questions that related to their program needs)

•         Usually a mixture.  Parts and pieces and then modifications to fit a particular situation. (had a variety of methods for data collection, analysis and interpretation.  Used questions that would guide their evaluation needs.).

I think the program evaluation completed by DEUSM was very thorough and useful.  They completed a program evaluation but also took important steps in making their findings functional and purposeful.  They actually did something with the findings in order to improve their current program.  They used a variety of ways to obtain valid information and data which in turn allowed for the needed changes.

Program Evaluation Test

Just a test to see if this is up and running :)