Revised Teacher Survey

After carefully reading assignment 5 directions, I realized I missed the part about having various questions (open ended, scale rating, short answer, etc).  I revised my survey in order to make some improvements, however, I still wanted to keep similar questions and have it remain short and simple.

I will say that I did like the variety of questions in the revised survey much more than my original yes/no survey.  Even though the content/questions remained much the same, the variety in ways I asked them seemed to have provided much more information.

I definitely see the benefit in asking a variety of question types over the traditional yes/no survey.

Revised Teacher Survey

1.     What grade do you teach?

Kindergarten ___

Grade 1 ____

Grade 2 ____

Grade 3 ____

Other (ESL, DLT) ____

2.    Do you use WiggleWorks in your classroom?  (Resources, on the SMARTboard, etc.)

No ___

Yes ___

If answered yes, please check what best describes your use:

Sometimes _____

On a weekly basis _____

On a daily basis _____

As much as we can _____

3.    Do your students use WiggleWorks in the computer lab?

Yes ___       No ___

If yes, what is their comfort level when using the program on their own?

Confident ____

Work with a friend ____

Need some guidance at first _____

Unable to use WiggleWorks on their own _____

4.    Do you think there was enough PD provided for WiggleWorks?

Yes ___          No ___       Would like some more PD ____

5.    Do you find the resource box helpful?  If so, what do you use?

No ___

Yes ___  If yes, what do you use?

Books ___

Audio Books ___

Teacher Plan/Resource binder ___

6.    Is there a reason you are not using WiggleWorks in your classroom or in the computer lab?

7.    If you have been using WiggleWorks, what part do you use most?

Read ___

Read Aloud ___

Write ___

My Book ___

Magnet Board ___

8.     What part of WiggleWorks have your students been using most?

Read ___

Read Aloud ___

Write ___

My Book ___

Magnet Board ___

9.     Do you think WiggleWorks provided student engagement?

No ___

Yes ___ If yes, rate using the below scale, 1 being mildly engaged, 5 being thoroughly engaged.

1 ___  2 ___ 3 ___ 4 ___ 5 ____

10. Do you agree with the following statement:

"WiggleWorks combines powerful technology with leveled books and instructional materials to develop reading, writing, and language skills necessary for future reading success. This proven effective program helps teachers scaffold instruction and advance all primary students toward reading independence."

Not at all       Somewhat         Agree       Strongly Agree

11.   If you have been using WiggleWorks, will you continue to use it in the future?

N/A ___

Yes ___

No ___

12.  Additional comments/feedback.

Teacher Survey

Well, the experience I had with the teacher survey was a good one.  I initially thought no one would complete it, or maybe just a few.  I was surprised how many people completed the survey and how it created some great dialogue.  I had several teachers approach me to tell me they completed the survey, and then proceeded to provide more feedback about the program in person.

I know the PD that I provided, and the handful of teachers that I have worked closely with implementing the program, but you never know for sure who will use it while you are away.  I work at 3 schools and often have team meetings, so I am not always around.  It was great to see that many teachers are using the program and are very happy with the results.  It was also interesting to see how teachers that were provided the same PD used the program in different ways.

I decided to use Survey Monkey to stay consistent with what I have been using for the last two years.  Any time that I needed to collect data or use a small survey I have used Survey Monkey.  I knew teachers would be familiar with the style and format.  I also didn’t want to go over ten questions as I thought the more questions, the less answers I would receive.

My original survey was sent out as a link in an e-mail.  I explained that I just wanted some feedback about the WiggleWorks program, successes and roadblocks, that all answers would be anonymous; their participation/time/feedback would be greatly appreciated.

I know how I usually feel about surveys, well at least up until our last 809 class J  I have learned that communication is very important when administering a survey.  If participants have a better understanding why the information is being collected, they may take more time or be more honest when completing them. 

Here is a copy of the survey; again, the one I passed out to teachers was via e-mail, using Survey Monkey.

1.     Do you use WiggleWorks in your classroom?  (Resources, on the SMARTboard, etc.)

Yes ___       No ___

2.    Do your students use WiggleWorks in the computer lab?

Yes ___       No ___

3.    Do you think there was enough PD provided for WiggleWorks?

Yes ___       No ___

4.    Do you find the resource box helpful?  If so, what do you use?

No ___

Yes ___

Books ___

Audio Books ___

Teacher Plan/Resource binder ___

5.    Is there a reason you are not using WiggleWorks in your classroom or in the computer lab?

6.    If you have been using WiggleWorks, what part do you use most?

Read ___

Read Aloud ___

Write ___

My Book ___

Magnet Board ___

7.     What part of WiggleWorks have your student been using most?

Read ___

Read Aloud ___

Write ___

My Book ___

Magnet Board ___

8.     Do you think WiggleWorks provided student engagement?

Yes ___

No ___

9.     Do you agree with the following statement:

"WiggleWorks combines powerful technology with leveled books and instructional materials to develop reading, writing, and language skills necessary for future reading success. This proven effective program helps teachers scaffold instruction and advance all primary students toward reading independence."

Yes ___

No ___

10.  If you have been using WiggleWorks, will you continue to use it in the future?

N/A ___

Yes ___

No ___

Other than feedback about the program, I was also able to find out that most teachers (or at least the ones who came to talk to me) found the survey ‘short and sweet.’  They said that if it was any longer they may not have completed it, just because time is always an issue.  I also thought it was interesting that most teachers said they would rather explain their problems (especially dealing with technology) to a person and not to have to write it out on a survey.

I know I have gained some valuable knowledge regarding the WiggleWorks program and surveys by doing this assignment.  I also know that even though my initial survey had about 20 questions (almost double!), these 10 questions gave me the information I needed.

Student Survey

I wanted to include this survey because I wasn't able to create one survey that both students and teachers could answer.  I created a short survey for students (see below) and a short survey for teachers that will be posted next.

I used this survey with grade 1 and 2 students. It is a very simple and short survey, but I just wanted some feedback. 

The feedback was all positive and I assume some of that has to do with the fact that WiggleWorks is a new program.  I find sometimes the ‘newness’ (I may have just created a word!) alone engages students.  I am very interested to see if students feel the same way in June, and then again next school year.

I do think WiggleWorks is an engaging program and it is designed to help improve students reading and writing levels.  In saying that, it is a complimentary program and not a core resource.  Teachers are using the program in a variety of ways with students at this point.

I also think that it is too early to determine if the WiggleWorks program is actually improving students reading and writing levels.

I will post the teacher survey as well.

WIGGLEWORKS SURVEY

J - YES

L - NO

Do you use WiggleWorks in the computer lab?

J ____

L ____

Do you like using WiggleWorks in the computer lab?

J ____

L ____

Do you like when your teacher uses WiggleWorks on the SMARTboard?

J ____

L ____

Do you like reading with WiggleWorks?

J ____

L ____

Do you like writing with WiggleWorks?

J ____

L ____

Does WiggleWorks help you with reading and writing?

J ____

L ____

Does WiggleWorks make reading and writing fun?

J ____

L ____

Draw a star by your favorite sections. 

Students did have help completing this survey. The questions were read aloud so that young readers would understand what was being asked.

Assignment 4 - Logic Model

I apologize that my logic model is small and hard to read.  I really did try to enlarge the text but didn’t seem to have the space.

The inputs or what we would invest in the WiggleWorks evaluation are staff, time, technology, materials, equipment and the district mandate.

The outputs which include activities and participation mainly relate to the teachers and students.  The activities are more focused on the PD/training and in-class support for the WiggleWorks implementation.  The AISI teacher also has to include Jing videos and other resources that will assist teacher training and using the technology.

The outcomes are based on what short/medium and long term impacts the WiggleWorks program and implementation will have on students and teachers.

There are assumptions that stakeholders want the best for this implementation and will work towards success.

Other than volunteers that may be needed to work with students, external factors are not a significant part to this evaluation.

The evaluation will be focused, data will be collected, analyzed and interpreted before a final report in written. 

Assignment 3 – Evaluation Assessment

Engage Stakeholders

Who should be involved?

District supervisors, AISI teacher and cohort, K-3 teachers, administration and students

How might they be engaged?

Stakeholders will be engaged by understanding the program and viewing how technology can be integrated into the classroom and curriculum.  Stakeholders will also see the importance of the PD and in-class support that is provided for the WiggleWorks program.

Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

The WiggleWorks program, the training/PD for teachers, and the implementation of WiggleWorks.

Who will use the evaluation? How will they use it?

Who/users                             How will they use the information?

District supervisors                 Determine if this mandate is successful.  To determine if it is a program that will continue to be used by all district K-3 teachers.

AISI teacher                             Look at the implementation of WW and to see if further PD is needed or in-class support.  Success, challenges, anything that would improve the implementation and overall success.  To make decisions about modifying how WW is implemented or the PD that is offered.

K-3 teachers                            How to successfully integrate WiggleWorks in the classroom and the curriculum. 

What questions will the evaluation seek to answer?

·         Is the WiggleWorks program successful at meeting the needs of teachers and students in terms of their literacy program?

·         Is WiggleWorks an effective program?

·         Are student’s literacy levels increasing?

·         Is the professional development provided to teachers enough?

·         Is the in-class support helping teachers with the implementation?

·         What is working well?  Successes

·         What needs to be changed or altered?

What information do you need to answer the questions?

What I wish to know?  Indicators – How will I know it?

Is the program a success?                              Survey, Observation, Teacher and student feedback, student literacy levels (are they improving)

Is the PD and in-class support sufficient?       Teacher feedback, observation of program implemented in the classroom.

                                                                        Teachers will have and increased confidence in the PD and in-class support has been effective.

When is the evaluation needed?

June 2011

What evaluation design will you use?

Case study as it will provide various qualitative measures. 

Collect the information

What sources of information will you use?

Existing information:  Printed material about the program, teacher guides, program testimonials, saved data from students, feedback from teachers and students that have been using the program for awhile.

People:  Program participants.  K-3 teachers, students

Pictorial records and observations:  AISI teacher will collect any photos or videos that document  learning.

What data collection method(s) will you use?

·         Survey

·         Interview

·         Observation

·         Case study

·         Photos/videos

·         Testimonials

·         Journal, log, diary

Instrumentation:  What is needed to record the information?

·         Survey (template)

·         Questions (template)

·         Journal for taking observation notes

·         Folders for collection case study information

·         Camera

·         Binder to collect and organize testimonials

When will you collect data for each method you’ve chosen?

Method	Before Program	During Program	Immediately After	Later
Survey	X	X	X
Interview	X	X	X
Observation		X
Case study	X	X	X
Photos/videos		X
Testimonials		X	X
Journal, log, diary		X

Will a sample be used?

Yes, I have been monitoring a class since September 2010. 

Pilot testing: When?  Where? How?

I will look at three classes from one school, and two classes from my other two schools.  I will follow these teachers and students, successes and challenges before continuing the implementation to every class.

Analyze and Interpret

How will the data be analyzed?

Data analysis methods:  Data will be organized according to school, grade and teacher.  Information will then be organized in pros/cons of programs as well as pros/cons of PD and in-class support.  I will only be using a few classes for this evaluation as time is limited and it will take a great deal of time to interpret the data.

Surveys, interview questions, photos, videos, observations, case study notes, etc. will all be put into a binder for organization.  A copy of each will also be scanned into a computer file for easier sharing of information and findings.

Who responsible: 

AISI teacher will organize the data collected.

How will the information be interpreted – by whom?

AISI teacher will organize data and do an initial interpretation.  Data will then be shared with AISI cohort, administration and district supervisors in order to get another perspective.  Program participants may also be ask to join the discussion in order to get their meaning or interpretation of the data collected.

What did you learn?  What are the limitations?

From doing this activity/plan alone I have learned time is of the essence.  It takes time to plan an evaluation, it will take time to conduct and evaluation and it will take time to interpret the completed evaluation.

I haven’t completed the evaluation yet so I am not sure what the outcome will be.

Use the Information

How will the evaluation be communicated and shared?

To Whom                                                       When/Where/How to present

Admin. supervisors                                         Upon completion a written report will be submitted.

AISI cohort                                                       In person at monthly meetings, provide updates, share short summary statements, visuals

Administration                                                Monthly updates in person providing a short summary as to what is happening, also a written report will be provided upon completion.

*All communication will be able to be shared at any time if someone wishes to review it.

All findings will be shared at the end of the evaluation during the interpretation process.

Next Steps?

Discuss and take suggestions about how to improve the program, if there needs to be any

changes, what will that look like, etc.

Manage the evaluation:

·         Human subject’s protection – FOIP forms, only first name and last initial will be used for students, as well as grade and last initial (1W, 2F) for teachers.

·         Management chart – keep a chart showing data of who is attending PD, who is using the program, when they are using the program (how often), what will take place at what time (surveys, interviews, etc.)

·         Timeline – provide a timeline to all involved so that they know what to expect and when, also this will keep the evaluation on time.

·         Responsibilities – A master list of what all responsibilities are and who is in charge of what, what needs to be done, by whom and when

Standards:

·         Feasibility – I don’t foresee any barriers or obstacles at this time.

·         Accuracy – I feel the methods of data collection will provide a thorough understanding of the WiggleWorks program and how it is being implemented.  I will refine and revise as necessary.