Assignment 3 – Evaluation Assessment

Engage Stakeholders

Who should be involved?

District supervisors, AISI teacher and cohort, K-3 teachers, administration and students

How might they be engaged?

Stakeholders will be engaged by understanding the program and viewing how technology can be integrated into the classroom and curriculum.  Stakeholders will also see the importance of the PD and in-class support that is provided for the WiggleWorks program.

Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

The WiggleWorks program, the training/PD for teachers, and the implementation of WiggleWorks.

Who will use the evaluation? How will they use it?

Who/users                             How will they use the information?

District supervisors                 Determine if this mandate is successful.  To determine if it is a program that will continue to be used by all district K-3 teachers.

AISI teacher                             Look at the implementation of WW and to see if further PD is needed or in-class support.  Success, challenges, anything that would improve the implementation and overall success.  To make decisions about modifying how WW is implemented or the PD that is offered.

K-3 teachers                            How to successfully integrate WiggleWorks in the classroom and the curriculum. 

What questions will the evaluation seek to answer?

·         Is the WiggleWorks program successful at meeting the needs of teachers and students in terms of their literacy program?

·         Is WiggleWorks an effective program?

·         Are student’s literacy levels increasing?

·         Is the professional development provided to teachers enough?

·         Is the in-class support helping teachers with the implementation?

·         What is working well?  Successes

·         What needs to be changed or altered?

What information do you need to answer the questions?

What I wish to know?  Indicators – How will I know it?

Is the program a success?                              Survey, Observation, Teacher and student feedback, student literacy levels (are they improving)

Is the PD and in-class support sufficient?       Teacher feedback, observation of program implemented in the classroom.

                                                                        Teachers will have and increased confidence in the PD and in-class support has been effective.

When is the evaluation needed?

June 2011

What evaluation design will you use?

Case study as it will provide various qualitative measures. 

Collect the information

What sources of information will you use?

Existing information:  Printed material about the program, teacher guides, program testimonials, saved data from students, feedback from teachers and students that have been using the program for awhile.

People:  Program participants.  K-3 teachers, students

Pictorial records and observations:  AISI teacher will collect any photos or videos that document  learning.

What data collection method(s) will you use?

·         Survey

·         Interview

·         Observation

·         Case study

·         Photos/videos

·         Testimonials

·         Journal, log, diary

Instrumentation:  What is needed to record the information?

·         Survey (template)

·         Questions (template)

·         Journal for taking observation notes

·         Folders for collection case study information

·         Camera

·         Binder to collect and organize testimonials

When will you collect data for each method you’ve chosen?

Method	Before Program	During Program	Immediately After	Later
Survey	X	X	X
Interview	X	X	X
Observation		X
Case study	X	X	X
Photos/videos		X
Testimonials		X	X
Journal, log, diary		X

Will a sample be used?

Yes, I have been monitoring a class since September 2010. 

Pilot testing: When?  Where? How?

I will look at three classes from one school, and two classes from my other two schools.  I will follow these teachers and students, successes and challenges before continuing the implementation to every class.

Analyze and Interpret

How will the data be analyzed?

Data analysis methods:  Data will be organized according to school, grade and teacher.  Information will then be organized in pros/cons of programs as well as pros/cons of PD and in-class support.  I will only be using a few classes for this evaluation as time is limited and it will take a great deal of time to interpret the data.

Surveys, interview questions, photos, videos, observations, case study notes, etc. will all be put into a binder for organization.  A copy of each will also be scanned into a computer file for easier sharing of information and findings.

Who responsible: 

AISI teacher will organize the data collected.

How will the information be interpreted – by whom?

AISI teacher will organize data and do an initial interpretation.  Data will then be shared with AISI cohort, administration and district supervisors in order to get another perspective.  Program participants may also be ask to join the discussion in order to get their meaning or interpretation of the data collected.

What did you learn?  What are the limitations?

From doing this activity/plan alone I have learned time is of the essence.  It takes time to plan an evaluation, it will take time to conduct and evaluation and it will take time to interpret the completed evaluation.

I haven’t completed the evaluation yet so I am not sure what the outcome will be.

Use the Information

How will the evaluation be communicated and shared?

To Whom                                                       When/Where/How to present

Admin. supervisors                                         Upon completion a written report will be submitted.

AISI cohort                                                       In person at monthly meetings, provide updates, share short summary statements, visuals

Administration                                                Monthly updates in person providing a short summary as to what is happening, also a written report will be provided upon completion.

*All communication will be able to be shared at any time if someone wishes to review it.

All findings will be shared at the end of the evaluation during the interpretation process.

Next Steps?

Discuss and take suggestions about how to improve the program, if there needs to be any

changes, what will that look like, etc.

Manage the evaluation:

·         Human subject’s protection – FOIP forms, only first name and last initial will be used for students, as well as grade and last initial (1W, 2F) for teachers.

·         Management chart – keep a chart showing data of who is attending PD, who is using the program, when they are using the program (how often), what will take place at what time (surveys, interviews, etc.)

·         Timeline – provide a timeline to all involved so that they know what to expect and when, also this will keep the evaluation on time.

·         Responsibilities – A master list of what all responsibilities are and who is in charge of what, what needs to be done, by whom and when

Standards:

·         Feasibility – I don’t foresee any barriers or obstacles at this time.

·         Accuracy – I feel the methods of data collection will provide a thorough understanding of the WiggleWorks program and how it is being implemented.  I will refine and revise as necessary.