Assignment 4 - Logic Model

I apologize that my logic model is small and hard to read.  I really did try to enlarge the text but didn’t seem to have the space.

The inputs or what we would invest in the WiggleWorks evaluation are staff, time, technology, materials, equipment and the district mandate.

The outputs which include activities and participation mainly relate to the teachers and students.  The activities are more focused on the PD/training and in-class support for the WiggleWorks implementation.  The AISI teacher also has to include Jing videos and other resources that will assist teacher training and using the technology.

The outcomes are based on what short/medium and long term impacts the WiggleWorks program and implementation will have on students and teachers.

There are assumptions that stakeholders want the best for this implementation and will work towards success.

Other than volunteers that may be needed to work with students, external factors are not a significant part to this evaluation.

The evaluation will be focused, data will be collected, analyzed and interpreted before a final report in written. 

Assignment 3 – Evaluation Assessment

Engage Stakeholders

Who should be involved?

District supervisors, AISI teacher and cohort, K-3 teachers, administration and students

How might they be engaged?

Stakeholders will be engaged by understanding the program and viewing how technology can be integrated into the classroom and curriculum.  Stakeholders will also see the importance of the PD and in-class support that is provided for the WiggleWorks program.

Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

The WiggleWorks program, the training/PD for teachers, and the implementation of WiggleWorks.

Who will use the evaluation? How will they use it?

Who/users                             How will they use the information?

District supervisors                 Determine if this mandate is successful.  To determine if it is a program that will continue to be used by all district K-3 teachers.

AISI teacher                             Look at the implementation of WW and to see if further PD is needed or in-class support.  Success, challenges, anything that would improve the implementation and overall success.  To make decisions about modifying how WW is implemented or the PD that is offered.

K-3 teachers                            How to successfully integrate WiggleWorks in the classroom and the curriculum. 

What questions will the evaluation seek to answer?

·         Is the WiggleWorks program successful at meeting the needs of teachers and students in terms of their literacy program?

·         Is WiggleWorks an effective program?

·         Are student’s literacy levels increasing?

·         Is the professional development provided to teachers enough?

·         Is the in-class support helping teachers with the implementation?

·         What is working well?  Successes

·         What needs to be changed or altered?

What information do you need to answer the questions?

What I wish to know?  Indicators – How will I know it?

Is the program a success?                              Survey, Observation, Teacher and student feedback, student literacy levels (are they improving)

Is the PD and in-class support sufficient?       Teacher feedback, observation of program implemented in the classroom.

                                                                        Teachers will have and increased confidence in the PD and in-class support has been effective.

When is the evaluation needed?

June 2011

What evaluation design will you use?

Case study as it will provide various qualitative measures. 

Collect the information

What sources of information will you use?

Existing information:  Printed material about the program, teacher guides, program testimonials, saved data from students, feedback from teachers and students that have been using the program for awhile.

People:  Program participants.  K-3 teachers, students

Pictorial records and observations:  AISI teacher will collect any photos or videos that document  learning.

What data collection method(s) will you use?

·         Survey

·         Interview

·         Observation

·         Case study

·         Photos/videos

·         Testimonials

·         Journal, log, diary

Instrumentation:  What is needed to record the information?

·         Survey (template)

·         Questions (template)

·         Journal for taking observation notes

·         Folders for collection case study information

·         Camera

·         Binder to collect and organize testimonials

When will you collect data for each method you’ve chosen?

Method	Before Program	During Program	Immediately After	Later
Survey	X	X	X
Interview	X	X	X
Observation		X
Case study	X	X	X
Photos/videos		X
Testimonials		X	X
Journal, log, diary		X

Will a sample be used?

Yes, I have been monitoring a class since September 2010. 

Pilot testing: When?  Where? How?

I will look at three classes from one school, and two classes from my other two schools.  I will follow these teachers and students, successes and challenges before continuing the implementation to every class.

Analyze and Interpret

How will the data be analyzed?

Data analysis methods:  Data will be organized according to school, grade and teacher.  Information will then be organized in pros/cons of programs as well as pros/cons of PD and in-class support.  I will only be using a few classes for this evaluation as time is limited and it will take a great deal of time to interpret the data.

Surveys, interview questions, photos, videos, observations, case study notes, etc. will all be put into a binder for organization.  A copy of each will also be scanned into a computer file for easier sharing of information and findings.

Who responsible: 

AISI teacher will organize the data collected.

How will the information be interpreted – by whom?

AISI teacher will organize data and do an initial interpretation.  Data will then be shared with AISI cohort, administration and district supervisors in order to get another perspective.  Program participants may also be ask to join the discussion in order to get their meaning or interpretation of the data collected.

What did you learn?  What are the limitations?

From doing this activity/plan alone I have learned time is of the essence.  It takes time to plan an evaluation, it will take time to conduct and evaluation and it will take time to interpret the completed evaluation.

I haven’t completed the evaluation yet so I am not sure what the outcome will be.

Use the Information

How will the evaluation be communicated and shared?

To Whom                                                       When/Where/How to present

Admin. supervisors                                         Upon completion a written report will be submitted.

AISI cohort                                                       In person at monthly meetings, provide updates, share short summary statements, visuals

Administration                                                Monthly updates in person providing a short summary as to what is happening, also a written report will be provided upon completion.

*All communication will be able to be shared at any time if someone wishes to review it.

All findings will be shared at the end of the evaluation during the interpretation process.

Next Steps?

Discuss and take suggestions about how to improve the program, if there needs to be any

changes, what will that look like, etc.

Manage the evaluation:

·         Human subject’s protection – FOIP forms, only first name and last initial will be used for students, as well as grade and last initial (1W, 2F) for teachers.

·         Management chart – keep a chart showing data of who is attending PD, who is using the program, when they are using the program (how often), what will take place at what time (surveys, interviews, etc.)

·         Timeline – provide a timeline to all involved so that they know what to expect and when, also this will keep the evaluation on time.

·         Responsibilities – A master list of what all responsibilities are and who is in charge of what, what needs to be done, by whom and when

Standards:

·         Feasibility – I don’t foresee any barriers or obstacles at this time.

·         Accuracy – I feel the methods of data collection will provide a thorough understanding of the WiggleWorks program and how it is being implemented.  I will refine and revise as necessary.

Pre-thoughts for Assignment 3

Assignment 3 'Pre-thoughts' – Evaluation Assessment

It is always a challenge trying to find a program that teachers find effective and students find engaging.  I have worked with several assistive technology programs, several interactive websites, various literacy programs, etc. but have not be able to find the mixture of technology, reading, writing and “play”….until now.

I am an elementary AISI teacher and work with three schools within our district.  I work with about 60 elementary teachers and hundreds of students.  We had the opportunity to apply for some AISI funds last year and ended up purchasing a program called WiggleWorks at two of my schools.  I had heard about this program from a fellow teacher and how it had been updated over the last several years, I was intrigued J

Since purchasing this program last year our district has now implemented the program and added it to all K-3 teacher laptops (not because 2 of my schools purchased it).  Some teachers are familiar with the ‘older’ version of WiggleWorks as it was popular years ago.  The newer, updated version has the curricular focus integrated with the technology focus.

I have chosen to evaluate the WiggleWorks program as it is a program I use frequently and it is now being implemented in our district.  I personally feel that WiggleWorks is an incredible program if used effectively.

“WiggleWorks combines powerful technology with leveled books and instructional materials to develop reading, writing, and language skills necessary for future reading success. This proven effective program helps teachers scaffold instruction and advance all primary students toward reading independence.”

http://teacher.scholastic.com/products/wiggleworks/index.htm

There are several components to WiggleWorks as well as instructional materials to support the implementation and success of the program.  I think it would be valuable to assess the program and how it is being used in elementary classrooms.

“Evaluation assessment – a systematic study of the options, including their strengths and weaknesses, when a program evaluation is being planned”  (McDavid & Hawthorn, 2006, p.439).

I think an evaluation would prove beneficial as our schools, admin., teachers and other district members would be able to see how the program is able to reach student needs and what, if any, changes need to be made.

WiggleWorks

I have decided to choose WiggleWorks for my case study. “WiggleWorks combines powerful technology with leveled books and instructional materials to develop reading, writing, and language skills necessary for future reading success. This proven effective program helps teachers scaffold instruction and advance all primary students toward reading independence.”

I currently work with three schools in our district.  Two of my schools have purchased the entire WiggleWorks program and one school is part of our new district mandate that has WiggleWorks on all K-3 teacher laptops. 

I hope to explore how the program is implemented in all three schools and what the feedback is between the two programs (purchased and implemented).

Image taken from:

http://teacher.scholastic.com/products/wiggleworks/index.htm