Pre-thoughts for Assignment 3

Assignment 3 'Pre-thoughts' – Evaluation Assessment

It is always a challenge trying to find a program that teachers find effective and students find engaging.  I have worked with several assistive technology programs, several interactive websites, various literacy programs, etc. but have not be able to find the mixture of technology, reading, writing and “play”….until now.

I am an elementary AISI teacher and work with three schools within our district.  I work with about 60 elementary teachers and hundreds of students.  We had the opportunity to apply for some AISI funds last year and ended up purchasing a program called WiggleWorks at two of my schools.  I had heard about this program from a fellow teacher and how it had been updated over the last several years, I was intrigued J

Since purchasing this program last year our district has now implemented the program and added it to all K-3 teacher laptops (not because 2 of my schools purchased it).  Some teachers are familiar with the ‘older’ version of WiggleWorks as it was popular years ago.  The newer, updated version has the curricular focus integrated with the technology focus.

I have chosen to evaluate the WiggleWorks program as it is a program I use frequently and it is now being implemented in our district.  I personally feel that WiggleWorks is an incredible program if used effectively.

“WiggleWorks combines powerful technology with leveled books and instructional materials to develop reading, writing, and language skills necessary for future reading success. This proven effective program helps teachers scaffold instruction and advance all primary students toward reading independence.”

http://teacher.scholastic.com/products/wiggleworks/index.htm

There are several components to WiggleWorks as well as instructional materials to support the implementation and success of the program.  I think it would be valuable to assess the program and how it is being used in elementary classrooms.

“Evaluation assessment – a systematic study of the options, including their strengths and weaknesses, when a program evaluation is being planned”  (McDavid & Hawthorn, 2006, p.439).

I think an evaluation would prove beneficial as our schools, admin., teachers and other district members would be able to see how the program is able to reach student needs and what, if any, changes need to be made.

WiggleWorks

I have decided to choose WiggleWorks for my case study. “WiggleWorks combines powerful technology with leveled books and instructional materials to develop reading, writing, and language skills necessary for future reading success. This proven effective program helps teachers scaffold instruction and advance all primary students toward reading independence.”

I currently work with three schools in our district.  Two of my schools have purchased the entire WiggleWorks program and one school is part of our new district mandate that has WiggleWorks on all K-3 teacher laptops. 

I hope to explore how the program is implemented in all three schools and what the feedback is between the two programs (purchased and implemented).

Image taken from:

http://teacher.scholastic.com/products/wiggleworks/index.htm

 

Assignment 2: Description and evaluation of a prenatal exercise program for urban Aboriginal Women

Description and Evaluation of a prenatal exercise program for urban Aboriginal Women

“Evaluation can be viewed as a structured process that creates and synthesizes information intended to reduce the level of uncertainty for stakeholders about a given program or policy.  It is intended to answer questions or test hypotheses, the results of which are then incorporated into the information bases used by those who have a stake in the program or policy” (James C.McDavid and Laura R.L. Hawthorn, (2006), p.3)

This was my place to start when looking at assignment two.  I really needed to know why it was important to evaluate such a program and what happens to the information obtained.  It is essential to see some of the following, I think, when looking at evaluation:

·       Is the program effective?

·       Is the program meeting the outcomes or goals?

·       Should the program continue?  Is it successful?

·       What are the benefits of the program?

·       Has there been any important development or changes that need to be made to the program? Etc., etc.

Looking at the article on the prenatal exercise program for urban Aboriginal women, I would use the Scriven method because it has a simple approach to evaluation while still being able to focus on the goals that need to be obtained.   

The prenatal exercise program had a purpose, to decrease gestational diabetes mellitus (GDM) among urban Aboriginal women by providing an exercise program as well as to educate the participants about GDM.  I did find it hard to decide what they would want to evaluate, the program, the success, if the program actually reduced the rate of GDM??  I don’t feel the program was clear in what they wanted to achieve as the end result, or at least defining the specific goals.

There is a reason for the program and the offerings of the course are clearly documented.  I feel they could have been more specific in the program in terms of what would make it a success (each participant participate 100% of the time; look at diet, regular checkups, amount of exercise, etc.)  “Initiatives to promote regular physical activity among Aboriginal women during their childbearing years could play a major role in optimizing healthy pregnancies and in reducing rates of type 2 diabetes in future generations” (Klomp, H., Dyck, R., and Sheppard, S. (2003), p. 1).  What is defined or accounts as ‘regular’ physical activity?

I would also assume that in order for this program to continue, feedback would need to be obtained as well as data ensuring that the program is and has been a success.  Following Scriven’s model, the formative feedback would provide information during the program for any changes that would need to be made.  Summative evaluation would occur at the end of the program to see if the original goals have been met and if the program has been effective. 

Although I think the program had good intentions I felt the program wasn’t clearly defined in terms of what the end outcome or result would be. Was it to inform and educate participants, provide an exercise program to a target group, collect data on participants who were in the program, or all the mentioned?  I think an evaluation using Scriven’s method would help the program in terms of letting them know what changes they need to make in order to achieve the desired success and to also obtain the necessary data to make the changes.

James C. McDavid & Laura R.L. Hawthorn.  (2006).  Program Evaluation & Performance Measurement:  An Introduction to Practice.  Thousand Oaks, CA:  Sage Publications.

Assignment 1: Review of DEUSM Program Evaluation

DEUSM Program Evaluation
This is the first review I have attempted on a program evaluation.  I hope I am somewhat on track :)  Please feel free to provide feedback, suggestions or comments as I am sure it will be useful in the future.

The Medical Education Department made revisions to the existing program evaluation approaches at Dokuz Eylul University School of Medicine (DEUSM) in June of 2005.

After careful consideration they chose a mixed evaluation model that was systematic and multidimensional and was developed to meet their institutional needs.  I also found some of Scriven’s evaluations methods present after reading through the program evaluation.

The evaluation consisted of three main questions as the areas of inquiry based on the general education goals:

1.      What are the effects of the educational program on students and graduates?

2.      What are the effects of the educational program on trainers?

3.      Is the educational program being implemented as planned?

I think it is impressive when an institution conducts a program evaluation in order to make any necessary changes than in turn will make their program more successful.  After only one year of the evaluation the institution made important revisions to the educational program.  This was a case where the program evaluation was completed for a purpose, improving the current program. 

Some of the strengths and weaknesses I found for DEUSM’s evaluation are noted below:

Strengths:

·         Started the evaluation with a purpose.

·         Had 3 guiding questions for the evaluation.

·         Made necessary changes after only one year.

·         Had a variety of methods for data collection.

·         Had a place for data analysis and interpretation.

·         Results were used for program improvement.

·         Obtained qualitative and quantitative data.

·         Looking at An Overview of Evaluation Theories by Michael Scriven, the idea of allowing those who are being evaluated to participate in the evaluation has become increasingly popular.  This is more collaborative and participatory in terms of evaluation.  It makes sense to have the people being evaluated take part in the evaluation and this was the case for DEUSM.

Weaknesses:

·         Data collected for the most part was only once a year.

·         Due to time limitations some activities were moved to the next year.

Overall I feel the program evaluation was a success and met the needs of the institution.  They made effective changes based on the two year evaluation program that included revising their curriculum, reducing the frequency of exams, and a diversification of socio-cultural activities in order to meet the needs of the students.  Other content based changes were made and the school plans on continuing the program evaluation.

As stated earlier I found that the DEUSM program evaluation adheres to Scriven and Mix methods of evaluation.

Some of the key points of program evaluation according to Scriven that the DEUSM has used are:

•         Simple approach to evaluation (started with 3 questions)

•         Purpose of evaluation can be goals or roles (goal was to improve the current program)

•         Goals - outcomes of the program, reason for a program, really need to study the goals of a program (DEUSM questions 3, is the educational program being implemented as planned?)

Scriven also says that the formative part of evaluations gives feedback during the delivery of a program for immediate or future modification.  DEUSM started making modifications after the first year of evaluation.

 Scriven, (1991) also says that formative evaluation is typically conducted during the development or improvement of a program; in this case it was their medical program that they wanted to improve.  He also states that the summative part of the evaluation can evaluate the learning materials and learning process to see what changes need to be made for improvement which DEUSM did after one year.

Mix Methods

•         Rarely is a pure theory applied (DEUSM had questions that related to their program needs)

•         Usually a mixture.  Parts and pieces and then modifications to fit a particular situation. (had a variety of methods for data collection, analysis and interpretation.  Used questions that would guide their evaluation needs.).

I think the program evaluation completed by DEUSM was very thorough and useful.  They completed a program evaluation but also took important steps in making their findings functional and purposeful.  They actually did something with the findings in order to improve their current program.  They used a variety of ways to obtain valid information and data which in turn allowed for the needed changes.

Program Evaluation Test

Just a test to see if this is up and running :)